If you are confused about something I said, I can clarify some more.
In ParticleSettings for the emitter, turn on Rotation and set Initial Rotation to Normal. In my example I also set gravity to 0, as shown, as it seems you did also.
Orient your particle object in edit mode in a way that its local y-axis points to the forward direction you like it to have. You can see its local axis by turning on Axis in Display tab of the object properties for the particle object.
You might also want to have a look over the manual section on particle rotation.