1
$\begingroup$

One of the most time taking part while working with images in Blender is to get pixels to numpy arrays and then to assign them back. As Blender supports multi-thread render I've thought it could be possible to make an image processing faster using concurrent.futures like this (the script adds 10 images, 10 numpy arrays and assigns arrays as images' pixels):

import bpy, concurrent.futures, time
import numpy as np

def test(counter, multi):
    t1 = time.perf_counter()
    x = 1920
    y = 1080
    px = x*y*4
    images = bpy.data.images
    counts = [i for i in range(counter)]
    all_images = []
    for i in counts:
        num = np.ones(px, dtype = 'f')
        name = 'imgtest_'+str(i)
        new = images.new(name, x, y)
        all_images.append(new)
    if multi:
        def pixels(img):
            img.pixels[:] = num[:]
        with concurrent.futures.ThreadPoolExecutor() as executor:
            executor.map(pixels, all_images)
    else:
       for img in all_images:
           img.pixels[:] = num[:]
    t2 = time.perf_counter()
    print(t2-t1)

test(10, True)
test(10, False)

But it gives almost no time benefit. Also I've tried to split pixels lists to slices and process them with threading, but it doesn't speed up processes as well. I've never used threading before. Am I doing something wrong here or does Blender just not support threading for such tasks?

Improve this question
$\endgroup$
4
  • $\begingroup$ Take a look at E-Cycles addon, and see how it works $\endgroup$
    – cxnt
    Commented Apr 14, 2020 at 10:52
  • $\begingroup$ Thank you! Right now I can't afford it. $\endgroup$
    – Andrey Sokolov
    Commented Apr 14, 2020 at 16:26
  • 1
    $\begingroup$ All the multithreading in blender is via c/c++ routines called via python. $\endgroup$
    – Bert VdB
    Commented Apr 15, 2020 at 12:23
  • $\begingroup$ Thank you! But still I don't understand: does it mean that Python built-ins threading modules are inefficient for processing Blender data and it can be only done by changing Blender source code? $\endgroup$
    – Andrey Sokolov
    Commented Apr 15, 2020 at 12:36

1 Answer 1

Reset to default
3
+100
$\begingroup$

First of all, Cycles render is an additional application with its own kernel, written in c/c++. It is ''glued'' to Blender by several Python scripts, well I think so.

Secondly, you can use Python modules like threading and multiprocessing to process data simultaneously. They are built-ins in Python interpreter, so there shouldn't be any problems if you'll decide to distribute your script.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you. concurrent.futures.ThreadPoolExecutor() from the script, which is also in built-ins, does the same as threading module, but for some reason it doesn't give any speed advantages. I've also tried concurrent.futures.ProcessPoolExecutor() (the same as multiprocessing module) for multiprocessing but it returns errors, something about "-c language was not found". I am afraid it may be because Blender just doesn't provide a possibility to process several images (or its other data blocks) simultaneously. And this is what I need to know exactly. Or a simple script proving opposite. $\endgroup$
    – Andrey Sokolov
    Commented Apr 14, 2020 at 16:15
  • 1
    $\begingroup$ @AndreySokolov, can't say anything about an error you've got using concurrent. By the way, you may not see any speed increment (sometimes speed lost) because of Pythons' GIL. Also It's a mutex included in a standard python implementation for safe memory management (to avoid race condition, etc). In short GIL runs one "thread" for a short period of time, stops it, runs another one, stops it, runs first one again, and so on... $\endgroup$
    – Artem
    Commented Apr 15, 2020 at 15:51
  • 1
    $\begingroup$ @AndreySokolov, maybe you can try to cheat a little by using threads outside python. Write your main image processing function(which you'd like to multithread) in good old C language using <pthread.h>. Then create a python binding to it and use in your script. It's a complicated task. You must compile code for different OS, in case of package distribution and overall be familiar with C $\endgroup$
    – Artem
    Commented Apr 15, 2020 at 16:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .