If I switch to cycles it renders as I would expect it to, and it looks fine in solid draw mode. I have also tried different materials so I think it's a geometry problem. I have also checked that the normals are facing outside as well. Can anyone explain what I'm doing wrong?
So I think I have found a solution but I still can't explain what is actually going on( might be a bug in the model importer). When I import a model, it generates automatic materials. These materials work perfectly with cycles, but not with Eevee. Any changes made to the default material seem to produce the same z-fighting in the model. So I deleted the generated materials and manually remade them( with exactly the same node setup ) and BAM, no more glitchy renders.
I can only assume this is due to a bug in the generated material.