I tried adding weight, removing weight and removing doubles. The mesh popping out won't go away. Any ideas?
To trouble shoot, enable Adjust edit cage to modifier result of the Armature modifier.
Then you can select the problem vertices in edit mode and examine the problem vertices.
This vertice should not be assigned to the chest. It should be mainly assigned to the UpperArm.R, currently it isn't.
To fix this go into weight paint mode, select the upper arm bone and add weight to the vertice.