The point is you have rotated the mesh in order to align it to the reference image.
But this is difficult to do that and to align the mesh with its origin (which is used by the mirror as point of symmetry).
Generally, reference images are rotated and scaled before so that they are aligned to the 3D view axis.
But you can correct it:
- Reset the Z rotation to 0°
- Snap the 3D cursor at the object origin (in object mode, having the mesh selected: ShiftS then 'cursor to origin')
- Enter edit mode
- Activate the mirror
- Set the 3D cursor as pivot point
- Select all the vertices
- And try to rotate/move the vertices so that the mirrored part is well aligned and mirrored
Once done, come back to object mode and rotate to check if your mesh is OK comparing to the reference.
Your blend file, a bit (but not perfectly) corrected :