When you convert a particle system, the new objects all use the same mesh data. This is the same as making a linked duplicate. When you join two of these objects together, every other object using that same mesh will show the same duplication.
Blender seems to recognise this, so that if you select all of the particle objects and Join them in one go they will be joined into one object without the duplication. The original object you used for the particle system will receive the same duplication so you may want to separate it first.
In the mesh data properties there is a number between the mesh name and the 'F' button (only shown if it is greater than 1). This number indicates how many objects are using this mesh data. By selecting the original object used as a particle you can click this number to separate the object from the others preventing it getting the same duplication when you join all of the others.