I need to track the camera in a very long panning shot. Because the start frame and end frame don't have common reference points, I am only able to get fragmented track points. I can detect a camera pattern in the graph view but when I try to build this, the average error is too high. Is it still possible to track a shot like this?
Yes, this is possible, although you will need to use the tripod solver. Also, you will have a difficult time doing a 3d scene reconstruction, because blender can't track depth in a panning-only shot. Also, make sure there is no zooming, as blender doesn't handle that correctly as of yet. Finally, make sure you have a decent number of solid keyframes (10ish) at each frame, and that the trackers are getting disabled AS SOON AS they begin to deviate (or be "obscured" by the edge of the screen).
Here are is the "Ultimate" BSE answer on motion tracking (required reading for anyone interested in blender's motion tracker):
When the markers move out of the visible frame you should disable them so they aren't considered when solving. Inversely you can also have markers disabled before they come into view and enable them only while they are visible.
You can find the enabled property in the marker properties panel P, when disabled a marker is shown as a plain box without the resize handles or centre point. Like other tracking properties, this is automatically keyframed as you change it.
You can also use box select B to select all the markers that are off screen and use ⇧ ShiftD to disable the selected markers.
The same can be done in the clip editors graph view by selecting the keyframes you want to disable and pressing ⇧ ShiftD.
There is also a margin setting that can automatically disable the marker as it reaches the edge of the image, this can be set for selected markers as well as the new marker settings.
I just made an add-on specifically for tracking long shots with a constantly evolving environment:
It allows you to batch-track "steps" of a specified number of frames any number of times at once (e.g. 15 steps of 20 frames each) without degrading performance on long shots. It also lets you have an overlap between each step by using an offset, so that the overall tracking doesn't have any cut.
Please read its Readme for instructions on how to use it and don't hesitate to ask me if you have any further questions.
I've used it on a ~5000 frames shot with a camera fixed on a car and my results turned out very good. The add-on could still use several improvements to ease and accelerate the process even more so you can expect updates to come.
This is still very fresh and not bullet proof yet, so please use with precautions and report issues if you find any.