What is blender's camera projection matrix model?

Question

I have question what is blender's camera projection model.

I want to use blender to my computer vision experiment which need to compare real photo and CG. For real photo, I used camera matrix K for perspective projection, like this

 K= [f 0 Px 0]
    [0 f Py 0] 
    [0 0  1 0]

(Px and Py is center coordiate of 2D image, start from Top-Left to Bottom Right)

So I can easily transform 3D vertex to 2D pixel coordinate. To compare this matrix, I need to know how blender make camera matrix K in it's internal rendering process. It seems that blender's camera has 3 parameters (f (focal length), near plane, far plane. Ignoring shift x and y).

How to make blender's camera projection matrix which transform 3D vertex to 2D image coordinate from these 3 paremeters?

Answer 1

The following Python code builds a calibration matrix K commonly used in computer vision from a Blender camera. You can then use the generated K matrix to compare against your own.

import bpy
from mathutils import Matrix

def get_calibration_matrix_K_from_blender(camd):
    f_in_mm = camd.lens
    scene = bpy.context.scene
    resolution_x_in_px = scene.render.resolution_x
    resolution_y_in_px = scene.render.resolution_y
    scale = scene.render.resolution_percentage / 100
    sensor_width_in_mm = camd.sensor_width
    sensor_height_in_mm = camd.sensor_height
    pixel_aspect_ratio = scene.render.pixel_aspect_x / scene.render.pixel_aspect_y
    if (camd.sensor_fit == 'VERTICAL'):
        # the sensor height is fixed (sensor fit is horizontal), 
        # the sensor width is effectively changed with the pixel aspect ratio
        s_u = resolution_x_in_px * scale / sensor_width_in_mm / pixel_aspect_ratio 
        s_v = resolution_y_in_px * scale / sensor_height_in_mm
    else: # 'HORIZONTAL' and 'AUTO'
        # the sensor width is fixed (sensor fit is horizontal), 
        # the sensor height is effectively changed with the pixel aspect ratio
        pixel_aspect_ratio = scene.render.pixel_aspect_x / scene.render.pixel_aspect_y
        s_u = resolution_x_in_px * scale / sensor_width_in_mm
        s_v = resolution_y_in_px * scale * pixel_aspect_ratio / sensor_height_in_mm

    # Parameters of intrinsic calibration matrix K
    alpha_u = f_in_mm * s_u
    alpha_v = f_in_mm * s_v
    u_0 = resolution_x_in_px*scale / 2
    v_0 = resolution_y_in_px*scale / 2
    skew = 0 # only use rectangular pixels

    K = Matrix(
        ((alpha_u, skew,    u_0),
        (    0  ,  alpha_v, v_0),
        (    0  ,    0,      1 )))
    return K

if __name__ == "__main__":
    # Insert your camera name below
    K = get_calibration_matrix_K_from_blender(bpy.data.objects['Camera'].data)
    print(K)

Notes

• I've tested this with cycles and many scenes, and it works.
• The updated code is now well-tested with non-unit aspect ratio in different scenes.
• Non-zero skew is not supported in blender AFAIK.
• The principal point assumed fixed at the center of the image. If this is not so, this code needs to be updated to take camera shift into account (see How to set a principal point for a camera)
• Lens distortion is taken into account separately, see my answer to Distortion Coefficients and Camera intrinsic of blender's cameras?
• I'm ignoring your last column containing all zeros.
• The code is written explicitly for documentation purposes. I apologize for the long names and extra variables but it really pays off for debugging careful parameter conversions.
• To get the full "projection matrix which transforms 3D vertex to 2D image coordinates" you mention in the end of your question, see my answer to 3x4 camera matrix from blender camera.