I was wondering if the samples matter at all when you're baking a normal map. I don't think so, but I just want to make sure.
No, samples have no effect.
A sample is a measurement done by shooting a ray into the scene. These rays scatter randomly inside the scene, so the more samples the more accurate information about the scene.
Normal map needs only 1 sample because there is no scattering. From every pixel of the normal map mapped on your lowpoly in the scene you need to send only 1 ray till it hits the highpoly, and store the surface direction into the normal map pixel. There is no additional random scattering after that, you get the exact finite value just there.