OpenGL 3D ViewPort frame capture with transparency (no checkered pattern)

Question

I attempt to use Blender 2.82 (currently I cannot change version), to render a robot hand holding an object, so that the hand creates an occlusion on the object, but the hand itself should not be visible in the final image, only the object. See attached screenshot. From the resulting OpenGL/Eevee render in a 2nd Blender window with a 3D viewport, I capture depth and color information of the object, and I need the alpha channel to be saved as well (where there is background or the occlusions). Side note not relevant to the question: the hand is animated live using an update via a python script reading ROS information, and the OpenGL captured render is sent back to ROS (similar to what the past Morse did with BGE).

I achieved almost all my goals (although slow at 10 Hz) and the scripts are available there for those interested.

However, I cannot get the alpha channel to be captured in the script. What happens is that the background is transparent in the viewport, one sees the checkered pattern, but the glReadPixels always captures alpha = 255 (or 1 in Blender terms) while the color are the color of the checkered pattern (which is confusing as the image in ROS looked to have transparent when in fact it has just the Blender checkered pattern instead).

data: [51, 51, 51, 255, 52, 52, 52, 255, 52, 52, 52, 255, 52, 52, 52, 255, 52, 52, 52, 255, (...) ]

instead of being at least this way

data: [51, 51, 51, 0, 52, 52, 52, 0, 52, 52, 52, 0, 52, 52, 52, 0, 52, 52, 52, 0, (...) ]

If I save the Viewport Render Image to png, I get the alpha channel saved and no checkered pattern.

To prepare these scripts I followed several useful threads and ended with this one from which I extracted what I needed and not doing GPU Rendering as it was slower (double rendering the scene) or had issues with my update of the posture. Workbench render did not permit to do the same transparency result, and Cycles was way too slow for my needs.

I searched several threads regarding the checkered pattern and the closest I found tried to do an homogeneous checkered pattern which won't help me (unless I make it a chroma-key green or something to later retrieve the alpha from chromakey, but this is hacky) All other alpha/transparency threads are related to getting it to work at all in the viewport, which I already have.

I also tried several options on which buffer to read from bgl.glReadBuffer GL_BACK or GL_FRAMEBUFFER, etc... or to which to attach to using bgl.glBindBuffer

I tried to use a hold_out shader for the World surface, which did not help.

I tried look for all possible things related to transparency and alpha that I could find (button/options/menus) but in the end it seems OpenGL captures the viewport final render (with checkered pattern) and not the rendered image before display.

I would be glad to get some ideas where to look at, if this is possible at all from bpy and bgl without going to saving a png from bpy as this would be even slower than 10 Hz I achieve now (for depth and color).

thanks

Here is the screenshot of what I managed to get, image is seen in an external display (ROS) but the checkered pattern IS color info, NOT the external display showing where there is alpha using its own checkered pattern.

Answer 1

For people facing this issue, I should say it is solved since 2.83, version that is also okay for my scripts.

I found that out while trying a chroma-key hacky work-around and realizing the green was not a uniform rgb(0,255,0) but some rgb(1,255,1).

It turns out user dither settings in post-processing was not considered for OpenGL live rendering in the viewport (always set to 1.0 in 2.82 see this ) and only set to user settings for png image saving. Then I found that starting from 2.83lts the user dither setting was taken into account in OpenGL live rendering and also the alpha values were now captured by glReadPixels.

So I fell on an old bug that was uncovered before but solved inbetween by some other commit.