How to monitor render process in a thread-safe manner?

I'm currently writing an addon, the purpose of which requires it to render several parts of active scene sequentially, preferably with UI-responsiveness intact. The problem is the UI freezes if I just directly call bpy.ops.render.render(animation=True). I get UI-responsiveness by changing the execution context to "INVOKE_DEFAULT", but then the code won't wait until a render process finished before executing the next one.

Using threading Module

Looking for an alternative, I then use Python's threading.Thread object to encapsulate render function, like this:

def _init_render_thread(self):
target=bpy.ops.render.render, kwargs={'animation':True})


... and check the thread object's is_alive method at interval set by a Timer object. Unfortunately it's prone to occasional random crashing, as any thread-related bugs are, so I'm still stuck with this method.

Chaining Handlers

Based on jesterKing's answer, I then chain the render execution using a scene-global property and render handlers with the following code. By chaining, the operator no longer needs to be modal. It's finishing early, then leaving completion to these handlers:

@persistent
def render_pre_handler(dummy):
props = bpy.context.scene.oha_layout_tools

if props.render_marker_infos:
rmi = props.render_marker_infos.pop(0)
marker_scene_settings(bpy.context, rmi)

@persistent
def render_complete_handler(dummy):
props = bpy.context.scene.oha_layout_tools

if props.render_marker_infos:
rmi = props.render_marker_infos.pop(0)
marker_scene_settings(bpy.context, rmi)

bpy.ops.render.render('INVOKE_DEFAULT', animation=True)
else:
bpy.app.handlers.render_pre.remove(render_pre_handler)
bpy.app.handlers.render_complete.remove(render_complete_handler)
props.marker_infos.clear()
props.render_marker_infos.clear()


It still doesn't work. The render_complete handler promptly executes even while the render thread itself is still in progress.

Is there any thread-safe way to monitor render process, other than spawning new Blender process?

• You should check blender.stackexchange.com/a/445/97 . The modal operator + python timer events is the way to go (as I posted in the comment to this answer) – jesterKing Jul 5 '13 at 17:27
• Um, I already done that, modal operator and timer object. As stated in my question. – Adhi Jul 5 '13 at 17:34
• I see. modal operators are still the way to go. I posted my answer with handler usage. – jesterKing Jul 5 '13 at 20:03
• Looking at your code, in order for you to use a modal operator the way @jesterKing is suggesting, you would actually have to have two modal operator classes (one for your script as a whole like you have it now, and one just for the render threading) – Gwen Jul 8 '13 at 5:08
• Maybe there's some mistake in it, but I did wrote two modal operators in a previous commit. The crash occurences were just as frequent and random... – Adhi Jul 8 '13 at 8:46

You can add handlers for render complete process: http://www.blender.org/documentation/blender_python_api_2_67b_release/bpy.app.handlers.html#bpy.app.handlers.render_complete

Use that to flip a flag that you can check in your main code that can check when the current render is ready before starting a new one.

• I still fail to use this. I need custom property to sequentialize render processes, but each modification to the property in an operator can't be seen outside that operator (including render_post handler), as if it never change. Maybe the monitoring must be done entirely within the modal operator, but I don't know how yet. – Adhi Jul 8 '13 at 4:27

Your handlers look fine, but you definitely need to look into other ways to run multiple processes than using the threading module. Python's threading module is not fully compatible with how Blender operates and is a guaranteed way to generate lots of ugly errors farther down the road.

Other options you should consider:

• Blender's Modal Operators: These are not the same as the threading module, but are instead essentially Blender's own implementation of threading. Look at the Operator Modal template file in Blender's Text Editor for an example of how to use this. This is the recommended way of multithreading in Blender!
• The multiprocessing module: Allows you to utilize multiple GPU/CPU cores for your rendering. This can have the additional advantage of allowing you to greatly speed up your rendering times by rendering multiple images in parallel.
• The subprocess module: This is by far the simplest way to thread, but only supports command-line operations. If you are willing to do command-line rendering, this should work.
• The concurrent.futures module: This module is a high-level way of operating parallel threads, that essentially allows you to "queue" jobs to be completed. I haven't used this yet, since it's new in Python 3.x, but I recommend you take a look at it to see if it does what you're looking for.
• I used subprocess to spawn Blender processes to render in the background, an old solution to another addon. IMHO it's horrible from user interaction standpoint, but I think I'll use this if all else fails. The addon must work, elegant or not... – Adhi Jul 8 '13 at 5:32
• @Gwenn Isn't threading not recommended for CPU-bound processes because of python's Global Interpreter Lock? and multiprocessing also not recommended because it will create multiple instances of blender? – Napster Apr 18 '17 at 15:45

My current solution to this problem, ugly as it is, is to have a modal operator watch output file's size at interval. Size doesn't change = render is finished, which works here because I'm only rendering audio files, so the process should be fast enough. Here's the relevant chunk of the code:

def check_render_file(self, context):

# ... initialization ...

if not self.prev_stat:
self.prev_stat = os.stat(context.scene.render.filepath)

return {'PASS_THROUGH'}

cur_stat = os.stat(context.scene.render.filepath)

if self.prev_stat.st_size != cur_stat.st_size:
self.prev_stat = cur_stat

return {'PASS_THROUGH'}

# ... cleanup, next render ...

return {'FINISHED'}

def modal(self, context, event):
if event.type == 'TIMER':
return self.check_render_file(context)

return {'PASS_THROUGH'}