4
$\begingroup$

I have some rendered or processed pixel data in a bgl.Buffer() object and need it in a numpy array but that proccess seems to take very long. Here is what I have tried so far:

buffer = bgl.Buffer(bgl.GL_BYTE, WIDTH * HEIGHT * 4)
# render something into it
imageDataNp = np.empty(WIDTH * HEIGHT * 4, dtype=np.float32)

# now some benchmarks
imageDataNp = np.array(buffer, dtype=np.float32) # 7.05s
imageDataNp = np.asarray(buffer, dtype=np.float32) # 7.00s
imageDataNp = np.fromiter(buffer, dtype=np.float32) # 3.29s

imageDataNp = np.array(buffer.to_list(), dtype=np.float32) # 4.69s
imageDataNp = np.asarray(buffer.to_list(), dtype=np.float32) # 4.71s
imageDataNp = np.fromiter(buffer.to_list(), dtype=np.float32) # 2.80s

# create an image datablock as secondary buffer
if not IMAGE_NAME in bpy.data.images:
    bpy.data.images.new(IMAGE_NAME, WIDTH, HEIGHT, float_buffer=True)
image = bpy.data.images[IMAGE_NAME]
image.scale(WIDTH, HEIGHT)

image.pixels.foreach_set(buffer)
image.pixels.foreach_get(imageDataNp) # both lines together take only 2.05s

# answer from stackexchange by Sanoronas
buffer_list = bytes(buffer.to_list())
imageDataNp = np.frombuffer(buffer_list, dtype=np.float32) # both lines together take 1.28s

It appears like even setting and getting to and from a Blender image datablock as a "secondary" buffer ist faster than going directly from bgl.Buffer() to a numpy array.

One possible reason I can think about is that bgl.Buffer() is at the time of 2.92 still not supporting the numpy buffer proctocol as described in this post.

Is there a way to achieve something similar using only Python? I see that the Cookie Cutter addon by CG Cookie ships with an file bgl_ext.py which includes a function def np_array_as_bgl_Buffer(array) which appears to do exactly the complement to what I am trying to achieve.

An example file with the full code from above can be found here.

$\endgroup$

2 Answers 2

7
$\begingroup$

I already talked to Gottfried directly, but I am sure that will be interesting for others as well: The fastest way to copy the image data for now seems to take a slight detour over the OpenGL calls. Here is a complete benchmarking example, which does the following (you need to provide an image yourself):

  1. Create a numpy array of the appropriate size
  2. Create a bgl.Buffer using the numpy array as a template
  3. Copy the pixel data via OpenGL from the image's OpenGL bindcode into the Buffer.
  4. Get a flipped view of the numpy array, which now contains the image data which was copied into the Buffer object.
  5. Save the numpy array via PIL to a PNG image in the blend-files directory. This step takes long and is only to demonstrate that the numpy array really contains the data.

This approach takes about 12 ms for a 4096 x 4096 RGBA image. The same approach can be taken with the GPUOffscreen.color_texture, if you rendered to an offscreen.

import bpy, bgl, os
import numpy as np
import timeit
from PIL import Image

# returns a flipped view into the given numpy array
def from_image_to_numpy(image, ndarray):
    
    # prepare image for OpenGL use
    if image.gl_load(): raise Exception()

    # set image texture to active texture
    bgl.glActiveTexture(bgl.GL_TEXTURE0)
    bgl.glBindTexture(bgl.GL_TEXTURE_2D, image.bindcode)
    
    # then we pass the numpy array to the bgl.Buffer as template,
    # which causes Blender to write the buffer data into the numpy array directly
    buffer = bgl.Buffer(bgl.GL_BYTE, ndarray.shape, ndarray)
    bgl.glGetTexImage(bgl.GL_TEXTURE_2D, 0, bgl.GL_RGBA, bgl.GL_UNSIGNED_BYTE, buffer)
    bgl.glBindTexture(bgl.GL_TEXTURE_2D, 0)
    
    # start time measurement
    start = timeit.default_timer()
    
    # this is needed to flip the image in Y direction due to the differently
    # defined origins of OpenGL and BPY/PIL (top-left vs. bottom-left)
    # NOTE: Could also use np.flip, but that is a bit slower (still < 0.1 ms though)
    flipped_ndarray = ndarray[::-1, :, :]

    # return the flipped array
    return flipped_ndarray

# SOME TESTING CODE
# +++++++++++++++++++++++++++++++++
if __name__ == '__main__':

    # PREPARE THE IMAGE AND NUMPY ARRAY
    # +++++++++++++++++++++++++++++++++
    # get an image from the Blender data blocks
    IMAGE_NAME = "Untitled"
    image = bpy.data.images[IMAGE_NAME]

    # create the numpy array
    ndarray = np.empty((image.size[1], image.size[0], image.channels), dtype=np.uint8)

    # SOME BENCHMARKING OF THE FUNCTION
    # +++++++++++++++++++++++++++++++++
    # define number of repeated executions to take the timing from
    EXEC_NUMBER = 100
    
    # test code to get the average execution time for from_image_to_numpy()
    print("Execution of from_image_to_numpy() takes %.3f ms (avergage of %i executions)" % (timeit.timeit("from_image_to_numpy(image, ndarray)", number=EXEC_NUMBER, globals=locals()) / EXEC_NUMBER * 1000, EXEC_NUMBER))

    # COPY FROM IMAGE TO NUMPY ARRAY
    # +++++++++++++++++++++++++++++++++
    # load the image data into a numpy array
    ndarray = from_image_to_numpy(image, ndarray)

    # SAVE THE ARRAY TO FILE
    # +++++++++++++++++++++++++++++++++
    # directory path of the blend file
    path = bpy.path.abspath(os.path.dirname(os.path.dirname(os.path.realpath(__file__))))

    # start time measurement
    im = Image.fromarray(ndarray)
    im.save(path + "/bpy_image_to_numpy.png", format="PNG", quality=90, optimize=True)```
$\endgroup$
6
$\begingroup$

You can use the function np.frombuffer. However you first have to convert the bgl Buffer to a bytes list.

buffer_list = bytes(buffer.to_list())
imageDataNp = np.frombuffer(buffer_list, dtype=np.float32)

With your example file this method took 1.69s in contrast to 1.8s for your workaround.

$\endgroup$
1
  • $\begingroup$ Thank you, this is already a big improvement. $\endgroup$ May 3, 2021 at 7:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .