I attempt to use Blender 2.82 (currently I cannot change version), to render a robot hand holding an object, so that the hand creates an occlusion on the object, but the hand itself should not be visible in the final image, only the object. See attached screenshot.
From the resulting OpenGL/Eevee
render in a 2nd Blender window with a 3D viewport, I capture depth and color information of the object, and I need the alpha channel to be saved as well (where there is background or the occlusions).
Side note not relevant to the question: the hand is animated live using an update via a python script reading ROS information, and the OpenGL captured render is sent back to ROS (similar to what the past Morse did with BGE).
I achieved almost all my goals (although slow at 10 Hz) and the scripts are available there for those interested.
However, I cannot get the alpha channel to be captured in the script. What happens is that the background is transparent in the viewport, one sees the checkered pattern, but the glReadPixels
always captures alpha = 255 (or 1 in Blender terms) while the color are the color of the checkered pattern (which is confusing as the image in ROS looked to have transparent when in fact it has just the Blender checkered pattern instead).
data: [51, 51, 51, 255, 52, 52, 52, 255, 52, 52, 52, 255, 52, 52, 52, 255, 52, 52, 52, 255, (...) ]
instead of being at least this way
data: [51, 51, 51, 0, 52, 52, 52, 0, 52, 52, 52, 0, 52, 52, 52, 0, 52, 52, 52, 0, (...) ]
If I save the Viewport Render Image to png, I get the alpha channel saved and no checkered pattern.
To prepare these scripts I followed several useful threads and ended with this one from which I extracted what I needed and not doing GPU Rendering as it was slower (double rendering the scene) or had issues with my update of the posture.
Workbench
render did not permit to do the same transparency result, and Cycles
was way too slow for my needs.
I searched several threads regarding the checkered pattern and the closest I found tried to do an homogeneous checkered pattern which won't help me (unless I make it a chroma-key green or something to later retrieve the alpha from chromakey, but this is hacky) All other alpha/transparency threads are related to getting it to work at all in the viewport, which I already have.
I also tried several options on which buffer to read from bgl.glReadBuffer
GL_BACK or GL_FRAMEBUFFER, etc... or to which to attach to using bgl.glBindBuffer
I tried to use a hold_out
shader for the World surface, which did not help.
I tried look for all possible things related to transparency and alpha that I could find (button/options/menus) but in the end it seems OpenGL captures the viewport final render (with checkered pattern) and not the rendered image before display.
I would be glad to get some ideas where to look at, if this is possible at all from bpy and bgl without going to saving a png from bpy as this would be even slower than 10 Hz I achieve now (for depth and color).
thanks
Here is the screenshot of what I managed to get, image is seen in an external display (ROS) but the checkered pattern IS color info, NOT the external display showing where there is alpha using its own checkered pattern.