9
$\begingroup$

I have question what is blender's camera projection model.

I want to use blender to my computer vision experiment which need to compare real photo and CG. For real photo, I used camera matrix K for perspective projection, like this

 K= [f 0 Px 0]
    [0 f Py 0] 
    [0 0  1 0]

(Px and Py is center coordiate of 2D image, start from Top-Left to Bottom Right)

So I can easily transform 3D vertex to 2D pixel coordinate. To compare this matrix, I need to know how blender make camera matrix K in it's internal rendering process. It seems that blender's camera has 3 parameters (f (focal length), near plane, far plane. Ignoring shift x and y).

How to make blender's camera projection matrix which transform 3D vertex to 2D image coordinate from these 3 paremeters?

$\endgroup$
2
  • $\begingroup$ Related: blender.stackexchange.com/q/9203/599 $\endgroup$
    – gandalf3
    Aug 19, 2014 at 6:16
  • $\begingroup$ I checked that post and find perspetive matrix generation method, but it is complete same with OpenGL's perspective matrix expression. It has left, right, top, bottom, near, far parameters. But I don't know how actually blender's camera settings converted to that parameters. Do you know any thing that? $\endgroup$ Aug 19, 2014 at 13:52

1 Answer 1

17
$\begingroup$

The following Python code builds a calibration matrix K commonly used in computer vision from a Blender camera. You can then use the generated K matrix to compare against your own.

import bpy
from mathutils import Matrix

def get_calibration_matrix_K_from_blender(camd):
    f_in_mm = camd.lens
    scene = bpy.context.scene
    resolution_x_in_px = scene.render.resolution_x
    resolution_y_in_px = scene.render.resolution_y
    scale = scene.render.resolution_percentage / 100
    sensor_width_in_mm = camd.sensor_width
    sensor_height_in_mm = camd.sensor_height
    pixel_aspect_ratio = scene.render.pixel_aspect_x / scene.render.pixel_aspect_y
    if (camd.sensor_fit == 'VERTICAL'):
        # the sensor height is fixed (sensor fit is horizontal), 
        # the sensor width is effectively changed with the pixel aspect ratio
        s_u = resolution_x_in_px * scale / sensor_width_in_mm / pixel_aspect_ratio 
        s_v = resolution_y_in_px * scale / sensor_height_in_mm
    else: # 'HORIZONTAL' and 'AUTO'
        # the sensor width is fixed (sensor fit is horizontal), 
        # the sensor height is effectively changed with the pixel aspect ratio
        pixel_aspect_ratio = scene.render.pixel_aspect_x / scene.render.pixel_aspect_y
        s_u = resolution_x_in_px * scale / sensor_width_in_mm
        s_v = resolution_y_in_px * scale * pixel_aspect_ratio / sensor_height_in_mm

    # Parameters of intrinsic calibration matrix K
    alpha_u = f_in_mm * s_u
    alpha_v = f_in_mm * s_v
    u_0 = resolution_x_in_px*scale / 2
    v_0 = resolution_y_in_px*scale / 2
    skew = 0 # only use rectangular pixels

    K = Matrix(
        ((alpha_u, skew,    u_0),
        (    0  ,  alpha_v, v_0),
        (    0  ,    0,      1 )))
    return K

if __name__ == "__main__":
    # Insert your camera name below
    K = get_calibration_matrix_K_from_blender(bpy.data.objects['Camera'].data)
    print(K)

Notes

  • I've tested this with cycles and many scenes, and it works.
  • The updated code is now well-tested with non-unit aspect ratio in different scenes.
  • Non-zero skew is not supported in blender AFAIK.
  • The principal point assumed fixed at the center of the image. If this is not so, this code needs to be updated to take camera shift into account (see How to set a principal point for a camera)
  • Lens distortion is taken into account separately, see my answer to Distortion Coefficients and Camera intrinsic of blender's cameras?
  • I'm ignoring your last column containing all zeros.
  • The code is written explicitly for documentation purposes. I apologize for the long names and extra variables but it really pays off for debugging careful parameter conversions.
  • To get the full "projection matrix which transforms 3D vertex to 2D image coordinates" you mention in the end of your question, see my answer to 3x4 camera matrix from blender camera.
$\endgroup$
10
  • $\begingroup$ Thanks for detailed reply, rfabbri. I tested your code and found some problem about fy (alpha_v) in non-uniform aspect image (1280x960). I projected mesh to camera, the x of projected position is correct, but y is looks incorrect. I modified your code to aspect_ratio_resolution = resolution_x_in_px / resolution_y_in_px s_v = resolution_y_in_px * scale * aspect_ratio / aspect_ratio_resolution / sensor_height_in_mm Then I got correct result. But I don't know what I did. Can you explan about this? $\endgroup$ Oct 6, 2015 at 3:50
  • $\begingroup$ Also I found that the original and modified code does not work for non uniform pixel aspect (non-square pixel). $\endgroup$ Oct 6, 2015 at 4:07
  • $\begingroup$ @KichangKim what is your sensor fit setting? (next to sensor size: Auto, Horizontal or Vertical)? $\endgroup$
    – rfabbri
    Oct 27, 2015 at 23:32
  • $\begingroup$ @KichangKim I've updated the code (and tested it). Thanks. The pixel aspect ratio is a render context setting, technically not really a physical camera parameter but part of the projection model which takes into account the pixel aspect ratio to be displayed. I am certain that the example you tried had a vertical sensor fit; this reverses the way the aspect ratio influences pixel coordinate computation in blender (I checked this against blender's internal code logic in addition to testing it). If the fit is vertical, the aspect is inverted so it needs to be applied to x. $\endgroup$
    – rfabbri
    Oct 28, 2015 at 3:33
  • 1
    $\begingroup$ I (think I) fixed the problems with this function in my modified version. $\endgroup$
    – Daniel
    Oct 8, 2018 at 12:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .