Hit the Set inverse button in the constraint settings
Set Inverse
By default, when you parent your owner to your target, the target becomes the origin of the owner’s space. This means that the location, rotation and scale of the owner are offset by the same properties of the target. In other words, the owner is transformed when you parent it to your target. This might not be desired! So, if you want to restore your owner to its before-parenting state, click on the Set Inverse button.
https://www.blender.org/manual/rigging/constraints/relationship/child_of.html