It sounds like your skin is reacting adversely to something it is contact with on an occasional basis.
This could be a soap or shower gel or other body wash you use from time to time. Also, it depends whether you apply anything to your penis when you masturbate: baby oil or lotion, skin cream or lotion, or anything else that you apply before, during or after you masturbate … ?