The weights probably aren't proper. Click on the mesh and press ctrl + tab, then click the bone you're moving. The more red an area is, the more it's influenced by that bone. Play around with it until you achieve the desired result.