I have question what is blender's camera projection model.

I want to use blender to my computer vision experiment which need to compare real photo and CG. For real photo, I used camera matrix K for perspective projection, like this

 K= [f 0 Px 0]
    [0 f Py 0] 
    [0 0  1 0]

(Px and Py is center coordiate of 2D image, start from Top-Left to Bottom Right)

So I can easily transform 3D vertex to 2D pixel coordinate. To compare this matrix, I need to know how blender make camera matrix K in it's internal rendering process. It seems that blender's camera has 3 parameters (f (focal length), near plane, far plane. Ignoring shift x and y).

How to make blender's camera projection matrix which transform 3D vertex to 2D image coordinate from these 3 paremeters?

  • Related: blender.stackexchange.com/q/9203/599 – gandalf3 Aug 19 '14 at 6:16
  • I checked that post and find perspetive matrix generation method, but it is complete same with OpenGL's perspective matrix expression. It has left, right, top, bottom, near, far parameters. But I don't know how actually blender's camera settings converted to that parameters. Do you know any thing that? – Kichang Kim Aug 19 '14 at 13:52
up vote 13 down vote accepted

The following Python code builds a calibration matrix K commonly used in computer vision from a Blender camera. You can then use the generated K matrix to compare against your own.

import bpy
from mathutils import Matrix

def get_calibration_matrix_K_from_blender(camd):
    f_in_mm = camd.lens
    scene = bpy.context.scene
    resolution_x_in_px = scene.render.resolution_x
    resolution_y_in_px = scene.render.resolution_y
    scale = scene.render.resolution_percentage / 100
    sensor_width_in_mm = camd.sensor_width
    sensor_height_in_mm = camd.sensor_height
    pixel_aspect_ratio = scene.render.pixel_aspect_x / scene.render.pixel_aspect_y
    if (camd.sensor_fit == 'VERTICAL'):
        # the sensor height is fixed (sensor fit is horizontal), 
        # the sensor width is effectively changed with the pixel aspect ratio
        s_u = resolution_x_in_px * scale / sensor_width_in_mm / pixel_aspect_ratio 
        s_v = resolution_y_in_px * scale / sensor_height_in_mm
    else: # 'HORIZONTAL' and 'AUTO'
        # the sensor width is fixed (sensor fit is horizontal), 
        # the sensor height is effectively changed with the pixel aspect ratio
        pixel_aspect_ratio = scene.render.pixel_aspect_x / scene.render.pixel_aspect_y
        s_u = resolution_x_in_px * scale / sensor_width_in_mm
        s_v = resolution_y_in_px * scale * pixel_aspect_ratio / sensor_height_in_mm

    # Parameters of intrinsic calibration matrix K
    alpha_u = f_in_mm * s_u
    alpha_v = f_in_mm * s_v
    u_0 = resolution_x_in_px*scale / 2
    v_0 = resolution_y_in_px*scale / 2
    skew = 0 # only use rectangular pixels

    K = Matrix(
        ((alpha_u, skew,    u_0),
        (    0  ,  alpha_v, v_0),
        (    0  ,    0,      1 )))
    return K

if __name__ == "__main__":
    # Insert your camera name below
    K = get_calibration_matrix_K_from_blender(bpy.data.objects['Camera'].data)
    print(K)

Notes

  • I've tested this with cycles and many scenes, and it works.
  • The updated code is now well-tested with non-unit aspect ratio in different scenes.
  • Non-zero skew is not supported in blender AFAIK.
  • The principal point assumed fixed at the center of the image. If this is not so, this code needs to be updated to take camera shift into account (see How to set a principal point for a camera)
  • Lens distortion is taken into account separately, see my answer to Distortion Coefficients and Camera intrinsic of blender's cameras?
  • I'm ignoring your last column containing all zeros.
  • The code is written explicitly for documentation purposes. I apologize for the long names and extra variables but it really pays off for debugging careful parameter conversions.
  • To get the full "projection matrix which transforms 3D vertex to 2D image coordinates" you mention in the end of your question, see my answer to 3x4 camera matrix from blender camera.
  • Thanks for detailed reply, rfabbri. I tested your code and found some problem about fy (alpha_v) in non-uniform aspect image (1280x960). I projected mesh to camera, the x of projected position is correct, but y is looks incorrect. I modified your code to aspect_ratio_resolution = resolution_x_in_px / resolution_y_in_px s_v = resolution_y_in_px * scale * aspect_ratio / aspect_ratio_resolution / sensor_height_in_mm Then I got correct result. But I don't know what I did. Can you explan about this? – Kichang Kim Oct 6 '15 at 3:50
  • Also I found that the original and modified code does not work for non uniform pixel aspect (non-square pixel). – Kichang Kim Oct 6 '15 at 4:07
  • @KichangKim what is your sensor fit setting? (next to sensor size: Auto, Horizontal or Vertical)? – rfabbri Oct 27 '15 at 23:32
  • @KichangKim I've updated the code (and tested it). Thanks. The pixel aspect ratio is a render context setting, technically not really a physical camera parameter but part of the projection model which takes into account the pixel aspect ratio to be displayed. I am certain that the example you tried had a vertical sensor fit; this reverses the way the aspect ratio influences pixel coordinate computation in blender (I checked this against blender's internal code logic in addition to testing it). If the fit is vertical, the aspect is inverted so it needs to be applied to x. – rfabbri Oct 28 '15 at 3:33
  • Thanks for updating code. I found that small mistake in your code, duplicating pixel_aspect_ratio calculation in HORIZONTAL and AUTO mode :) – Kichang Kim Nov 20 '15 at 3:12

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.