0
$\begingroup$

When I press Solve Camera Motion for a second time, I get a dramatically worse solve error despite only minor automatic refinements to values.

Why would this be? Unticking Keyframes before pressing Solve Camera Motion for a second time seems to solve the issue but I'd like to understand why. Why is it OK to let Blender determine the keyframes for the first solve but letting it do so on subsequent passes has terrible results?

Note: I'm interested in understanding the process here (why the solve error would get unexpectedly far worse), even though in my particular example below the initial solve error is great/low and doesn't actually need improvement.

Here are my steps in details:

1. I tell Blender to work everything out itself, i.e. I have Keyframe and all the Refine options checked:

enter image description here

2. I get a great/low solve error:

enter image description here

And it chooses the following keyframes:

enter image description here

3. I press Solve Camera Motion again (I even temporarily unticked Keyframe and set the A and B values back to what they were before, in case they affected things even when greyed out - however, I don't think they do). This time I get a terrible/high solve error:

enter image description here

And it chooses slightly different keyframes:

enter image description here

To me it looks like the Focal Length etc. have only been refined a bit. Why would solving again (and allowing it to choose the keyframes again) produce a dramatically worse result when I expected it would either achieve nothing or slightly refine the result?

This feels like a bug or perhaps I'm missing something that actually is changing dramatically between the two solves?

My .blend file and the background clip can be found here on GitHub.

Any insights on what's going on here would be much appreciated.

Improve this question
$\endgroup$
4
  • $\begingroup$ I don't have an answer for you, but your first result was very good, with error being less than half a pixel. That's better than ideal. Why do you want to solve again after getting such a good result? $\endgroup$
    – Alexandre Marcati
    Nov 16, 2021 at 18:53
  • $\begingroup$ Yes, the first result was great here but I'm interested in the mechanics of what's going on. That the result changes so much implies there's something I'm missing about the solving process. And I'd like to understand what that is, independent of the initial solve error in this particular situation (rather than treating it as magic that often works but sometimes produces very unexpected results). $\endgroup$
    – George Hawkins
    Nov 16, 2021 at 20:34
  • $\begingroup$ I see. Just to make sure I understood: you solved once and it gave the first result, then you didn't change any settings and solved again and the result was different. Or did you change some settings? If so, what did you change? $\endgroup$
    – Alexandre Marcati
    Nov 16, 2021 at 20:48
  • $\begingroup$ I didn't change anything, I just let Blender further refine the values itself, i.e. I left all the Refine checkboxes ticked. $\endgroup$
    – George Hawkins
    Nov 17, 2021 at 18:02

1 Answer 1

Reset to default
1
$\begingroup$

The root cause of the troubles

is very small parallax on the footage, meaning that the camera is not looking at the scene from different angles. Instead it rotates around itself.

This is so called "tripod camera" situation.

enter image description here

On such video there is simply not enough information to carry out calculations from math/geometric standpoint.

Particular problems and what can be improved:

1.please notice the solver cannot detect principal point of the camera correctly - these values are very far from the center of the image which is obviously not the real camera geometry.

enter image description here

That causes the situation that after the first solve the solution actually worsens, not improves. Small solution error is an illusion, the solver falls into a local minimum of noise data.

Hit "Set center" for defaults and uncheck "Refine - Optical center".

It will prevent the "89 pix error on the next solve".

2.Try to find the two frames with the maximum parallax (means maximum angle difference, angle around objects on the scene, not around the camera) and set "Frames A and B" on them. Set "Focal length" to obviously wrong value, check only "Refine - Focal length" and see if the solver can find reasonable focal length. If it can't - then uncheck all the "Refine" options - they make no sense in this stuation. My opinion - there is no chance with this footage to calculate "Refinements"

3.Look at the Blender's console. Although the output is far from being descriptive it may give you some clue about how the solver is doing with different parameters while experimenting.

enter image description here

The right way with this kind of footage

1.try to find out the parameters of the camera used for filming (or best guess). "Focal length" is the most important, the others are not so much (use center for the principal point and zeroes for radial distortion coefficients (K1,...). Even with focal length you don't have to be extremely precise. Even with 1,2,... 10... mm off you still have a chance for a good scene reconstruction.

Please notice, that focal length is kind of a relative parameter and it's usually written assuming "standard" 36mm camera sensor even if the real sensor is order of magnitude smaller.

Example: My OnePlus phone, 4mm real focal length, translated to 36mm sensor it is 28mm. So I would use 36mm sensor x 28mm focal length in blender.

2.check "Tripod" checkbox to enable appropriate algorithm

Solving error will be bigger, about 1.2 - 1.5 pix, however real scene geometry will be much more correct.

enter image description here

Appendix

You can determine the parameters of the camera with good (not very bad:)) accuracy using other methods (other than automatic refinement) if any single frame on the footage contains known geometric shapes, - straight lines. (your footage kind of does).

1.to find out "linear" camera geometry (focal length and maybe principal point) you can use f-spy application.

2.to find out "non-linear" geometry (meaning radial and tangential distortion) you can use manual camera calibration workflow in Blender. The approach is to load an image (a frame from the footage) with straight lines on it, draw straight lines on the overlay, enable calibration helpers and try different coefficients (K1,...) interactively looking at the resulting (un-)bending of the lines.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for that super detailed answer. I had actually tried the Tripod option and found that it gave worse solve errors than not ticking it - so (foolishly) I gave up on it, on the assumption that a lower solve error must always be a good thing. I was using a DSLR so I do know the sensor size and focal length (23.1mm and 18mm respectively). I presume if I enter 23.1mm for Sensor Size then I can use the unadjusted focal length - 18mm - as the Focal Length value, i.e. I don't need to calculate the 35 mm equivalent focal length and use that? $\endgroup$
    – George Hawkins
    Nov 17, 2021 at 18:00
  • $\begingroup$ OK - I ticked Tripod, unticked the Refine options, reset the Optical Center to (960, 540) px and the Lens Distortion values to 0 and set Sensor Size and Focal Length to their correct values (23.1mm and 18mm). And get a terrible solve error - 4.90px. I really put a lot of effort into cleaning up the tracks and markers so this seems a depressingly bad result. If I keep Tripod but also check all the Refine options then I get an apparently much better solve error (but with refined values for focal length and optical center that are no longer true to what they really were). $\endgroup$
    – George Hawkins
    Nov 17, 2021 at 18:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .