Going for a pretty general answer here, bear with me or skip to the end if you just want a yes or no.
So, while most if not all engines will offer some form of culling (not rendering/using what is not seen) you will still see detrimental drops in your framerate if your model has a significant polycount. By itself when testing you might be fine yes but games are meant to be dynamic and when you throw particles, physics and post processing in the mix it's a lot more for your system to process now.
For a quick example, fire up Blender and add the default cube, right click and select it, it is selected instantly! Now subdivide it say 10 times, add a subsurf modifier and set the view level to 6 and try again. It takes a while, this is a rather exaggerated example but it's more or less what your engine which is also a 3d software does as it has to iterate over the vertices in the mesh. Now imagine doing that on a moving character and having to do matrix math for every frame to calculate the new points for vertices to be at.
Even if your engine does support culling or some other magical algorithm to discard hidden geometry or automatic level of detailing (LOD) you would still most likely be hit with load times at some point whether when launching your game for the first time , loading assets or saving depending on how you go about it.
"Real" world example now, if your model has 100, 000 vertices, this is what is loaded into your engine of choice, if your armor is 20, 000 vertices and covers say 50% of your characters body. Your final polycount would be 70, 000 vertices instead of 120, 000 if you delete the hidden parts of the character model.
In any 3d modelers workflow there is usually an entire step dedicated to this called retopology where the simplest base mesh that can support the high details from a much more detailed and higher poly count model (usually sculpt). See https://cgcookie.com/course/introduction-to-retopology/
Even better, if you can generate normal and or displacement maps you can also see your polycount drop even more significantly. This is what pretty much every game studio does. If you're making a game you can't imagine that everyone will have a "good" machine to run it and most games ship with several different models depending on the resolution and or settings you are playing at.
For example here is a somewhat popular and funny screenshot of Skyrim being forced to run on the lowest possible setting. In game detailed grass and tree leaves are merely planes and that shiny detailed axe is a blob with some good textures.
Counter Strike Go
So in conclusion, people will try every trick in the book for lower geometry while still having a good look so if it isn't being seen and you can animate or get a similar effect, remove it.