This certainly looks like it: or certainly a Sebaceous gland has become infected.
What to do: get yourself a small bottle of Liquid Antiseptic intended to treat skin grazes and insect bites (in the UK the Brand is TCP), cream and lotion versions of antiseptic are of no use.
Dab a little of the undiluted Liquid Antiseptic directly onto the bump 4 or 5 times a day, for 3 or 4 days and this should certainly reduce in size. When it starts to subside you can reduce application of the antiseptic to 1 or 2 times a day until it has completely healed.