After looking at the following image from your paper, it seems to me as if your image is basically a sprite map, so I propose the following workflow to get you there.

Proposed Workflow:

I'm certain that you could mimic the polyhedron shape in blender to set active planes (viewports) on programatically, and use an UV coordinate mapping per sprite location to get a depth map per the corresponding camera location/angle (position).

Set up a highly subdivided plane for each view, then attach the Displacement Modifier to it, along with a custom UV layout to the sprite sheet that corresponds with the appropriate viewpoint.

I would then add a global driver to attempt to control the displacement modifier strength per projection, and when they get close to each other...

Apply the modifier, Join the objects together, and remove duplicate verts.

After looking at the following image from your paper, it seems to me as if your image is basically a sprite map, so I propose the following workflow to get you there.

Proposed Workflow:

I'm certain that you could mimic the polyhedron shape in blender to set active planes (viewports) on programatically, and use an UV coordinate mapping per sprite location to get a depth map per the corresponding camera location/angle (position).

Set up a highly subdivided plane for each view, then attach the Displacement Modifier to it, along with a custom UV layout to the sprite sheet that corresponds with the appropriate viewpoint.

I would then add a global driver to attempt to control the displacement modifier strength per projection, and when they get close to each other...

Apply the modifier, Join the objects together, and remove duplicate verts.

Uploaded a Video