In the top right of your screenshot is the outliner, I can see seven armatures there including some that are not visible in the viewport. That would indicate that while in object mode you are duplicating the entire armature, maybe while expecting to duplicate a single bone, which you should do in edit mode.
The first armature is the parent of several mesh objects while the others are duplicate armatures that can be deleted. You may need to select them in the outliner and delete them, enabling the little eye icon will make them visible in the viewport.
I don't know that each rig is the same, you may have made alterations to one of the duplicates that you expect to be in your initial armature. You can hide/show one rig at a time to see which bones are part of each armature.
Once you remove these duplicates you should find it easier to work on your rig.