2
$\begingroup$

I am currently trying to extract a depth map created by Cycles and read it in Python to undo the distortion introduced by Cycles. Let me tell you what I've done so far step-by-step.

  1. I cannot use Eevee for the rendering of the depth map as it doesn't properly support headless rendering on a cluster and I cannot fake a display (https://devtalk.blender.org/t/blender-2-8-unable-to-open-a-display-by-the-rendering-on-the-background-eevee/1436/16)
  2. Unfortunately, Cycles introduces radial distortion due to its pinhole camera model which I can undo with some postprocessing(Cycles generates distorted depth). This, however, creates my need to read the EXR file containing the depth map for some postprocessing afterwards.
  3. The OpenEXR package works wonderfully and I'd be happy with it, were it not for the fact that I cannot install it on said cluster because there are outdated gcc-libraries I have no control over.
  4. That's why I tried cv2's and imageio's EXR reader contained in the imread methods, the first returns None, the second returns a zero array. Curiously enough, imageio can read the very depth image if I first load it with OpenEXR, transform it into numpy and save it with imageio.imwrite as EXR, so it's not a general problem with EXR and imageio.

Now to my questions:

  1. Do you know what makes Blender's EXR output incompatible with imageio and cv2? I tried different/no compression, both half and full float, RGB or RGBA etc as Blender output node options.

  2. Is there a reliable way to intercept the depth map output before saving to disk? That would also help as I wouldn't need to read the EXR to begin with. Apparently, the only way I know via a Viewer node and bpy.data.images['Viewer Node'] does not work reliably headless (seriously, Blender).

  3. Is there a way to do some fancy undistortion in the compositing tree? If I could simply put my small code snippet to undistort it into some kind of node in the compositing tree, that would also make things easier. Of course it still needs to be as fast.

These are my code snippets:

#This works perfectly
#Z is the channel's name in blender, for some reason Blender adds '.V'
depth_exr = OpenEXR.InputFile('{}/depth/depth{}.exr'.format(basepath, frame_num))
depth_img = np.reshape(array.array('f', depth_exr.channel('Z.V')), (720,1280))

#This returns a zero array
depth_img2 = imageio.imread('{}/depth/depth{}.exr'.format(basepath, frame_num))
#This returns None; -1 is the mode for not changing/casting the input values to int
depth_img3 = cv2.imread('{}/depth/depth{}.exr'.format(basepath, frame_num), -1)

#However, this works well
imageio.imwrite('test_depth.exr', depth_img, flags=0x0001)
plt.imshow(imageio.imread('test_depth.exr'))

#-> 0x0001 is the code for EXR_FLOAT in saving images: https://github.com/imageio/imageio/issues/356

#Undistort the image based on pixel distance to center cx, cy and focal length in pixel units f_pix
#cx, cy, f_pix are from the true camera intrinsics extracted from Blender
indizes = np.mgrid[:depth_img.shape[0], :depth_img.shape[1]]
d_pix = np.sqrt((indizes[1, :, :] - cx)**2 + (indizes[0, :, :] -cy)**2)

depth_undistorted = np.cos(np.arctan(d_pix/f_pix)) * depth_img

A minimal example can be found here. It will generate the depth0001.exr in /tmp/ and you can try to import in Python with the functions above.

I seriously feel like Blender is actively sabotaging its scientific use. I'd seriously appreciate any help.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.