My hair particle system always spawns sideways like in the picture.
What to do?
This is normal. Some people create their source object/group lying down so that it's upright when Blender uses it, so you could simply rotate your source object.
However, you can fix it in the Particle System settings. Select the particle system tab and find the Rotation section. There is a setting "Initial Orientation", which is probably set to "None" or "Velocity/Hair". Try changing this to "Global Z". (depending on the orientation of your original object, you might need to use a different one).