The problem in your scene is caused by the bone structure. Take the L side for example:
Since you are using two bones for each upperarm for some reason (which is unlike normal human), the constrained bone
DEF-upper_arm.01.L doesn't follow the rotation of its control bone or even
DEF-upper_arm.02.L, which would lead to the problem. As shown below:
There would be a few ways to fix this, but I'm really not sure why you structed and set the control bone for upper arm like this (I assume you are trying to simulate the natural-looking muscle stretch). However, I think at least an additional Copy Rotation constraint on
DEF-upper_arm.01.L is needed here.
Here is my try, FYI: