Thanks to @SergeL's answer, we can duplicate another copy of armature and bake it, then reassign it the newly created action to the armature.
In my case, I need to remove the ik contraint in bones for making another new animation, so I keep a backup of original armature and continue work on duplicated copy of armature after done baking action. At this point, we still can select which armature we wanna apply onto your 3D object, like this:

Here's the steps to work on new action without deleting your ik bones:
- Duplicate selected armature to new copy.
- Bake action using the duplicated armature (pose menu > animation > Bake action... under pose mode)
- It should automatically assign the action to current duplicated armature.
- Remove all ik contraints in duplicated armature.
- Select your 3D object and goto its modifier. Select the duplicated armature in "Object" field under its armature modifier.
- And now, you can work on new animation in a new action, without the ik bones constraints while keeping it as backup.