# Combine depth images from two cameras using the world matrix

In my simple scene I have a very simple model and two cameras. From each camera I know the external parameters Pi (from bpy.data.objects['Cam0'].matrix_world) and both have the same internal parameters K. Additionally, I have one rendered depthmap (z-buffer) per camera.

From the depthmap I should be able to generate a point cloud X, with Xi = Pi^-1 xi, where x are the pixel scaled with its depth. However, if I combine the point clouds, they do not match, because they one is rotated away from the other.

Is P the used projection matrix (excluding K), especially is the rotation of the camera correct in terms of what I want to achieve?

EDIT: I have uploaded a example that illustrates my problem. From the setting in the blend file I have generated two depthmaps (are both contained as exr and as ascii txt). Further, the Matlab example visualizes the rotation errror. Example

• If the point clouds rotate apart than the transformation is wrong, when Xi are in world coordinates what is the rotation center? Perhaps you could add a .blend containing your cameras and the point clouds. From my understanding Pi would transform say a cube at the origin to the position and orientation of the camera Pi^-1 would apply the inverse transformation. – stacker Dec 10 '13 at 7:41
• See also this paper scholar.lib.vt.edu/theses/available/etd-12232009-222118/… page 13pp. – stacker Dec 10 '13 at 7:46
• I have uploaded an example. – Masala Dec 10 '13 at 9:52
• You are doing all the calculations in mathlab , I couldn't convert the ascii files with meshlab (meshlab.sourceforge.net) for use in blender. I would have give it a try if I could do it in blender instead of mathlab (I don't have). Now the question seems to be off-topic. – stacker Dec 10 '13 at 10:53
• The problem can also be visualized (with the same scripts) with Octave, which is open source. – Masala Dec 10 '13 at 14:24