Tried turning up maximum bounces to 12, still there but less.
You might want to try changing the Ray Visibility settings instead. (Although turning up the maximum bounces might be better in cases where you need these settings turned on)
The two settings that I turn off to help me get rid of these artifacts are Glossy and Transmission.
You would find these settings under Your light source > Object > Cycles Settings.
With both settings on:
Hopefully this helps you! :D