I linked a group in from another file, it was a character with armature. After linking it I realize that I don't have control over the armature so I deleted it and decided that I want to append it in instead. But it just won't appear after appending. I tried other groups, in different files, it appears that this is persistent. I can't append any group as long as I have: 1. linked it 2. then delete it. 

How can I solve this problem?