0
$\begingroup$

I am currently doing a computer vision task and got stuck on the following problem. I need ground truth values for my sequence. I have a nice sequence where the camera moves trough my scene and captures the RGB-frames. Now, I need a corresponding frame for every RGB-frame, but instead of the RGB-values it should store the distance. I know, that you can get the total depth (euclidean distance from camera to real-world-object) by connecting the 'Depth'-output of the Render Layer Node to a File Output Node in the Compositing workspace using the EXR file format. But I just need the Z-component of the distance to the camera and I don't want to convert it afterwards with the camera parameters (already did it, but I need a cleaner workflow).

I stumbled upon this function: bpy_extras.view3d_utils.region_2d_to_location_3d , but could find almost nothing about how to use it properly. I don't know what I should give it as input. Has anyone a solution or maybe already used the function and can explain me how i would use it in the default Blender setup (just a Cube, Camera and Light) and if it does what I expect it to do (giving the x,y,z and then I can strip the unnecessary information)? I already tried using the world_to_camera_view function, but this only works for object-parameters like vertices and not for the whole surface to create a dense map. I also know, that you can render the scene with a Cycles material to store x,y,z in the RGB-channels, but there you only get world-coordinates and you have to change the materials after you rendered the real sequence. I really need just a frame with z-distance to the camera for every pixel in the frame.

I would be really grateful if someone could help me, because I've been trying to do that for days now. I posted the same question before, here at stackoverflow, but I thought it might be more suitable to post it here now, because it is blender-specific.

EDIT: Pinhole Model I am interested in getting z for every pixel camcoord on my image plane. The way I am doing it currently is, that I get the depth depth for camcoord from Blender and then I use the Thales' theorem to get z with my known focal length and the known principal point. My problem with this solution is, that I don't get the actual true values for z because of the discrete pixels. A simple plane that is parallel to my camera (image plane) at the z-distance of 10 metres will give me values between 9.98 and 10.02 metres after my current approach.

$\endgroup$
4
  • $\begingroup$ Related blender.stackexchange.com/questions/28557/… Raycast into a scene, get the object and local location of hit if hit. Feeding global loc back into world to camera "should" give correlating results. Also see questions relating to camera projection matrix. $\endgroup$
    – batFINGER
    May 29, 2020 at 15:04
  • $\begingroup$ An ortho camera would give depth from camera plane? $\endgroup$
    – batFINGER
    May 29, 2020 at 15:11
  • $\begingroup$ The question you mentioned looks promising. I will try it. I think using an ortho-camera would change the camera parameters and therefore the depths wouldn't correlate to the pixels of my RGB-images anymore. $\endgroup$
    – wuff.
    Jun 1, 2020 at 9:49
  • $\begingroup$ * with ortho cam scaled to match frustum of persp camera $\endgroup$
    – batFINGER
    Jun 1, 2020 at 13:55

2 Answers 2

3
$\begingroup$

The Camera Data node will give you a shader which will return the values you are looking for in the pixels of an image.

Here's a calibration scene. The camera is at World (0,0,1), the planes are at Z = 0, 1, 2, 3.

enter image description here

With this shader set-up, the pixel-value will be Z-component of the vector from the camera to the shading-point. ( You could also have shaded the planes using the Texture Coordinate node, in the Object space of the camera, and taken the Z component of that)

enter image description here

The nodes in the 'Illustration Only' frame are there to bring the lengths into a 0-1 range.. you wouldn't use those. Ensure the render output is linear, (set the Color Management to 'Raw'), and save to a floating-point format, probably, as you say, EXR, (in which case, as @susu has commented, color transforms will be bypassed.)

$\endgroup$
2
  • 1
    $\begingroup$ EXR will bypass color transforms so it doesn't matter what settings are used for color management, it will be linear data. To ensure linear output on display referred formats use Raw. Standard will curve data by adding a "gamma curve". $\endgroup$
    – susu
    Jun 1, 2020 at 15:24
  • $\begingroup$ @susu Thanks. Edited. Wow. The more I read on this, the more confused I get. $\endgroup$
    – Robin Betts
    Jun 1, 2020 at 16:41
0
$\begingroup$

I think you could easily build a shader which calculates any relation between world position and camera position as well as orientation/direction. The shader node graph gives you the option to access the camera position/direction and the world position of any shading point. Additionally there are Vector Math nodes which help you to calculate whatever value you have in mind.

https://docs.blender.org/manual/en/latest/render/shader_nodes/input/camera_data.html https://docs.blender.org/manual/en/latest/render/shader_nodes/converter/vector_math.html https://docs.blender.org/manual/en/latest/render/shader_nodes/input/geometry.html

If you need more help on this I would ask you to elaborate on your issue with a small drawing so I can better understand which component of the depth you would like to see in the pass.

$\endgroup$
1
  • $\begingroup$ just edited my question with a drawing. My problem is not to get the position of the camera or the position of an object. My shading point you mean the object-point that is getting represented in the image-plane when rendering the image? How can I call every shading point for my 1920×1080 px? $\endgroup$
    – wuff.
    Jun 1, 2020 at 10:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .