The child object follows the parent when I move it, but when I change the parent object's scale the child not only changes it's scale as expected, but moves, and keeps moving. How to solve that?
The issue you are facing is just how parenting works. Let me try to explain.
Take this example:
In these two images I have a monkey parented to a cube. Notice the location of the monkey. The only thing I change is the scale of the parent cube. (Fun fact if you cleared the parent on the monkey now, it would jump back to the top image.)
In this type of parenting the parent's scale is a multiplier for the child object's location and dimensions, its just how it works.
From your question it sounds like you want to be able to scale the parent, and have the child copy the scale, but stay in the same location. To do that you need to use a Child Of Constraint (also detailed in my parenting answer.)
On still using the same example as above, put a Child Of Constraint on the monkey. You want to uncheck all the Location and Rotation check boxes. Now only the scale will be changed.