If you want to give the whole material a structured surface, put it into both. In over 90% of all cases, this is probably what you want.
But there are others:
If you want to material to look like the paint or glossy part is smooth and even and below that, there is a bumpy surface (application example: Wood finish in a car interiour), then add the normal map only to the diffuse shader of the shader.
If you want the material to look like it has a smooth general surface but a slightly scratched paint or top layer, it might make sense to only connect it to the glossy shader (probably the less frequently used case).
