What i'm trying to achieve is to stream an arbitrary camera view in blender to a v4l device.
What i've done so far
After reading the community post about streaming the Eevee view on Windows and linux, I adapted the spout addon for blender in order to work with the virtualvideo package. In short, I use a draw handler in Blender to read the pixels the 3D view and send it to the virtual v4l device:
scene = context.scene dWIDTH = camera.texshare.capture_width dHEIGHT = camera.texshare.capture_height view_matrix = context.scene.camera.matrix_world.inverted() projection_matrix = context.scene.camera.calc_matrix_camera( context.evaluated_depsgraph_get(), x=dWIDTH, y=dHEIGHT) # Defined somewhere else, just here for complete example offscreen = gpu.types.GPUOffScreen(dWIDTH, dHEIGHT) offscreen.draw_view3d( scene, context.view_layer, context.space_data, context.region, view_matrix, projection_matrix) bgl.glDisable(bgl.GL_DEPTH_TEST) draw_texture_2d(offscreen.color_texture, (0, 0), dWIDTH, dHEIGHT) # Read pixels and store them in the buffer bgl.glReadPixels(0, 0, dWIDTH, dHEIGHT, bgl.GL_RGB, bgl.GL_UNSIGNED_BYTE, buffer) # Send next frame fvd.send_next_frame(buffer)
What is working
When streaming is enabled, the virtualvideo streams the pixel buffer to the v4l device and i can watch the "stream" with ffplay (or something else):
What isn't working
bgl.glReadPixelsfunction takes about 300 to 400 ms to render one frame which results in a slow frame rate.
By using the view and projection matrices, the output is not exactly the "camera view", even the camera mesh gets rendered into the view
The rendered image somehow gets displayed in my blender viewport (See image below).
I somehow managed to flip the image upside down, but that's the least of my problems :D
What i'm asking
Is there a way to grab and redirect the already created (and very performant) camera view from blender? This does not even necessarily have to be done via the Python interface of Blender, if there is a good solution in C++ or similar I would be very grateful!
If not, is there an alternative for the
bgl.glReadPixelsfunction that works faster?
Why does the rendered image show up in the viewport and how can i disable this behavior?