I tried different FBX options but Mixamo says it can not indentify the armature.
You should be able to setup the armature in Mixiamo. Make sure your character is in a standard pose with both arms extended. Mixamo has problems with characters with short arms and legs. It works best with standard humanoid looking character because it's easier to identify the difference between the different joints. Once you've setup the armature you can click the "Download" button.