You have multiple shapes, which are concave, and what you write about compositing suggests that this is a problem for a much more complex scene that what you're showing us. Backface culling, or anything related, isn't going to work here:
It will still give us your "x-ray effect" when we look through concavity, or we look through two meshes. (It will also affect your shadows when you don't have manifold meshes, which is why these shadows look funny, but that's not a big deal in most situations.)
There is something we might be able to do that will work better:
Here, rather than using a Transparent BSDF to create my basic alpha, I'm using an IOR 1.0 Refraction BSDF. That turns my rays into transmission rays, so I can make further hits, backface or frontface, this object or another, into full transparency. To animate, I'd just be animating the mix factor from diffuse->refraction.
Refraction doesn't quite act right in Blender with regards to shadows, so we can improve this a bit by replacing our straight refraction BSDF with "ghetto glass": we'll mix between refraction and transparency on the basis of "is shadow ray":
This technique requires that you not be using refraction elsewhere in your scene, because these materials will be invisible to any kind of refraction rays, not just the rays they themselves generate. Refraction is not necessarily a frequently used effect, so this might meet your needs.