Quickly & continuously generate images without rendering?

Question

I know the title is a little ambiguous but bear with me for a minute

On one hand I have blender with a 3D model and a server script

s = socket(AF_INET, SOCK_STREAM)
s.setsockopt(SOL_SOCKET,SO_REUSEADDR, 1)
s.bind(("127.0.0.1",6100))

...

s.listen()
conn, addr = s.accept()
data= get_data(conn)
if(data=="move_arm"):
  move("arm")
  bpy.ops.render.render(write_still=True, animation=False)

...

and on the other hand I have another software that sends blender instructions on how to move that model via sockets

const client = new net.Socket(); // node.js taken for example

...

client.connect(6100, "127.0.0.1",()=>{
    client.write("move_arm");
});

but the problem is that rendering is very slowly compared to the continuous instructions being received,
So is there any better way to also generate the image(s) continuously to keep up?

Answer 1

Found a solution!

I had to directly get the image being generated in the buffer using the gpu module

You can trim the code down to this:

import bpy,gpu,time

def draw():
    global to_draw
    if to_draw:
        framebuffer = gpu.state.active_framebuffer_get()

        viewport_info = gpu.state.viewport_get()
        width = viewport_info[2]
        height = viewport_info[3]

        framebuffer_image.scale(width, height)

        pixelBuffer = framebuffer.read_color(0, 0, width, height, 4, 0, 'FLOAT')
        
        pixelBuffer.dimensions = width * height * 4
        framebuffer_image.pixels.foreach_set(pixelBuffer)
        framebuffer_image.filepath_raw = "/.../MyImage.png"
        framebuffer_image.save()
        to_draw=False

def update_draw():
    global to_draw
    bpy.ops.wm.redraw_timer(type='DRAW_WIN_SWAP', iterations=1)
    to_draw = True
    bpy.ops.wm.redraw_timer(type='DRAW_WIN_SWAP', iterations=1)

if __name__ == "__main__":
    global to_draw
    to_draw = False
    
    if "color_buffer_copy" in bpy.data.images:
        framebuffer_image = bpy.data.images["color_buffer_copy"]
    else:
        framebuffer_image = bpy.data.images.new("color_buffer_copy" , 32, 32, float_buffer=True,alpha=True)
    draw_handler=bpy.types.SpaceView3D.draw_handler_add(draw, (), 'WINDOW', 'PRE_VIEW')

    bpy.data.objects['Cube'].rotation_euler[0] = 1.5
    update_draw()
    
    time.sleep(10) # just added for an example
    
    bpy.data.objects['Cube'].rotation_euler[1] = 3
    update_draw()
    
    bpy.types.SpaceView3D.draw_handler_remove(draw_handler, 'WINDOW')

But the caveat to this is that you have to keep the GUI open
i.e. you cannot run this script from the command line in --background

but if anyone has an alternate solution which runs from the command line, please do post it