The problem is created by the IK bone pivot not align to the child bones in edit mode. Use the cursor to snap all the bone pivot to the same location. Also make sure that your bones are rotated so that it is perpendicular to the front view as much as possible - at rest.
Attached is are the bones rotated and all the IK pivots of the bone aligned properly.
Snapping the bone pivot onto the same location do the following - In edit Mode.
- Select the end bone nub.
- Use CtrlS choose "cursor to selected"
Select the other bones hit CtrlS choose " Selection to Cursor"
If required , go to pose mode and reset all rotation and transformation using AltG and AltR
Below, you can see 3 spherical nub aligned. The 2 red ones and the big green one needs to all be aligned in one spot using the "Cursor to selection" then " selection to cursor" method.
The answer is sort of correct. Hawkenfox meant to say "Shift S", not "Ctrl S" while in edit mode on an IK bone. This should solve any confusion. Ctrl S is used to save the file. At least it is in Blender 2.78a.
I just had the same problem, even by applying "with automatic weights" the armature would be out of the mesh, then I realized that I hadn't applied my mesh's modifiers, such as subdivision surface, I clicked on the apply button then I went to object mode and selected my character then shift selected my armature and ctr+p then "with automatic weights" that was it.