I think this has to do with having lots of tracking points vs memory that is slowing me down to a craw.
I have 16GB ram, and 11GB swap. I selected detect features, giving me a lot of tracking points, (per youtube I found out this feature) when I run it to track of course it runs fine for a while until my RAM is almost used up, then slows to a craw.
I too have proxied my video to 50% as well.
from what I have learned on YouTube the more tracking points the better. How do I utilize this feature in blender to select the points for me and still be able to use them all while tracking something over 150 frames. the limit that they all use for training purposes.
this too is a per-recorded movie by someone else.