How to determine lens undistortion values for motion tracking?

Question

In the camera setting in motion tracking, how do I tweak the K1, K2 and K3 value properly. I was told that these values are not necessary to tweak but I do want to adjust it, just for the sake of perfection.

Answer 1

All camera lenses in the real world create some kind of distortion. CG imagery does not have the imperfections of glass.

Setting the distortion values for the lens is crucial if you are trying to do accurate scene reconstruction and integration with 3D generated elements.

To determine the values for a particular lens you can shoot a picture of something you know has straight lines (It doesn't have to be a grid, but it must have straight lines).

Use the annotation tool to draw straight lines to use as reference . Find the beginning and ending points of lines that should be straight and draw a line that connects them (To draw straight lines use CtrlD and Left Mouse Button).

Enable Calibration on the Display tab.

Play with the K1, K2 settings and distort the pencil lines until they match the curvature of the distorted lines in your image.

To correct Barrel distortion use Negative numbers on K1, for Pincushion use positive values.

Settings in Blender 2.8X

Keep in mind that most distortion is radial and distorts the image starting from the optical center of the lens (that might not coincide with the center of the image) If you find that the drawn line distort correctly on the top of the frame and not on the bottom (or vice versa, or on one side of the frame and not in the other) you might need to manually find the optical center in the camera settings. The slider on the left will move the optical center horizontally, the one on the right will move it vertically.

To check the result in your image you can use the Render Undistorted option.

Use the same values for images made with the same lens. If you change lenses, you will have to determine the values for each one.

Currently there is no way in blender to use images made with a variable focal length during the shot. In other words don't use a zoom during the shot.

How To Make Use Of The Distortion Information

To use the distortion information in the compositor use a Movie distortion node (not to be confused with the Lens Distortion node) available in Shift + A > Distort > Movie Distortion

The node needs to use the information from the clip you used to determine the distortion.

To create an undistorted version of the image use the node with the Undistort option enabled after the video clip, and before the nodes to combine with the 3D scene.

Depending of your needs, there is another way to use the distortion values of the lens, and that is to leave the video footage alone and apply de distortion to the 3D scene generated in blender. In that case you'd plug the Movie Distortion node to the rendered layers using the Distort option.

Note

There are currently two choices to undistort.

Divisions will only give you two parameters to play: K1 and K2. Most lenses can be corrected with these.

For more complex lens distortion.

Polynomial will give you an additional K3.

To understand the difference between the two you can read this IPOL article about division models, but for the mathematically challenged it may suffice to quote:

The main advantage of the division model is the requirement of fewer terms than the polynomial model for the case of severe distortion. Therefore, the division model seems to be more adequate for wide-angle lenses.

for basic info on K1, K2, K3 and image center (or optical axis) read:

https://en.wikipedia.org/wiki/Distortion_(optics)

Blender uses algorithms based on OpenCV. You might want to read more in the following link: https://docs.opencv.org/2.4/doc/tutorials/calib3d/camera_calibration/camera_calibration.html

Other Note When undistorting it is likely that you'll run into problems with the edges of the frame:

It is possible that the undistorted image shows repeated pixels at the edge of the frame:

Or that the edges of the image are missing.

Read these links:

Solve camera motion in Blender: exact camera data or refinement?

Motion tracking. The edges of undistorted footage are cropped, and so are the elements of the 3D scene

For an alternative way to export undistortion information to other software like Nuke, Natron, Fusion, etc. read this link:

https://openvisualfx.com/2020/05/05/nuke-lens-distortion-in-blender-2-90/

and

Why is my footage smearing out towards the border on rendering an image? 3