I have been playing around with camera tracking and I'm happy enough with the tracking and alignment of the markers with the footage when viewed through the camera.
The problem is, when I am not looking through the camera I can see that the grid isn't aligned to the camera.
How do you correctly align the grid to this tracked camera? I have set a floor and two points for scale, also clicked setup tracking scene, but this just added cube and plane which were on the grid.
If all your trackers are the floor use the "set X axis" and "set Y axis" and redo "setup tracking scene".
It would be better if you had a few trackers on the wall or fence and you'd set those as "wall" in the orientation tab.
You can manually re-orient your solved scene in the 3D viewport by rotating the camera. Just select the camera rotate it about 90 degrees on X. The trackers are parented to the camera, so they will follow all movement.
If you have a good camera solve (with errors under 0.3) it shouldn't be difficult to line all of your tracker marks at 0 on the z axis on a side or front view as they should be in one plane.
In this example the camera is solved but nor oriented correctly:
Manually rotating the camera to bring the right orientation (note the tracking points at the side view at the bottom, those are trackers placed on the ground):
Bringing the camera down to make the trackers on the ground be roughly at 0 on the Z axis:
