You are on the right track to understanding the solution...
Actually an IK bone generally doesn't "follow" FK bones, but in some rigify characters you have controls to determine if IK or FK controls a limb, and whether to snap align one to another at frames where you want to switch character bones being driven by IK or not:
Here's what you can do to an armature so that you can switch and have IK follow FK and not have a circular logic that could cause problems.
First define an armature (or armature object level) Custom Property for each limb set (arms, legs, eyes):
copy this as a new driver:
Use this to negate the IK Influence of IK over hand or foot FK bone where you find this IK constraint, so that FK/IK influence doesn't fight one another:
Then you can add a copy position to the IK bone, and drive its influence factor to disable based on some IK->FK factor. Note that I change the bone end to be used from 0 to 1.
Now when you turn the armature IK influence between 0 and 1 you'll see proper alignment, and can keyframe the appropriate gone. When you want to change animation control to either IK or FK, be sure to add keyframes for both bones.
This will somewhat mimic Rigify style controls.