0
$\begingroup$

I have 100s of scenes with simple primitive objects at random locations (not in a sequence) in each of these scenes. For example if I have 2 objects in a scene, the only thing that will vary across all 100s of scenes is the location, orientation of these 2 objects. I have file generated with all these locations and orientations to load into blender from. Currently in my project I am sequentially loading each location, orientation detail of a scene and rendering it one after the other.

Is there a way to generate these 100 scenes in parallel? I have been looking at GPU rendering/cluster based rendering but failed to find any concrete that I can use.

Any pointer will be appreciated. Thanks in advance!

$\endgroup$
  • $\begingroup$ I don't think rendering them in parallel would help you if you're just doing it on your one machine. I think a command line batch script would probably be better. Are all the scenes in one blend file, or do you have multiple blend files. If so, what naming convention are you using? $\endgroup$ – Todd McIntosh Jul 28 '15 at 18:38
  • $\begingroup$ I am not using any blend file. I am setting things up using a python script, that sets the scene with camera, light and I am reading the file with the object poses into a list. Running the list in a loop, I am creating objects in the scene and rendering it. Clearing the objects again and creating them based on the next location and so on. Hope the process is clear now. $\endgroup$ – desinghkar Jul 28 '15 at 19:06
  • $\begingroup$ Hmmm, I think that will make it tough to split up or parallelize. If you could render each scene from the command line with a startup script and an input argument, you might have better luck at splitting it all up. $\endgroup$ – Todd McIntosh Jul 28 '15 at 21:52
  • $\begingroup$ Also if you could arrange for each scene to be loaded for each frame *Scene 1 on Frame 1", it might make it easier to distribute the render job as well. That way if you trigger a render job to do Frames 1-5 it will render out Scenes 1-5. $\endgroup$ – Todd McIntosh Jul 28 '15 at 21:55
  • $\begingroup$ Yes, I think the one scene per frame idea is good, because it keeps all the scene logic inside the blender file script, but allows the frame ranges of each distributed render job to be batched. $\endgroup$ – Todd McIntosh Jul 28 '15 at 21:57
1
$\begingroup$

Ok, here's my first draft of an answer. I'll revise it as we talk this out more. If this seems to be going in the right direction, I can add further coding samples.

The key to being able to distribute your render jobs between many render nodes is being able to easily tell the render node what scenes to render.

Currently, from what you've described you would have to add some sort of input argument value to tell the script what scene to load and then render out one frame.

I think if you could modify your script to run on a per frame basis, then map the frame number to scene number, that would make it easier to distribute the job to many machines without having to worry about input values for the scene number.

So the file arrangement would be:

  • GenScene.blend file blank file ready for scene setup
  • Startup python script ("GenSceneFrame.py") would be run once for every frame of the animation. Let's assume we have 100 scene's to render, we'll set the animation range to 1-100
  • SceneData.csv file that contains object setup data for each scene

These 3 files would be packaged together in one folder (or with subfolders if you want but you'd have to be vigilant about paths).

You would have initate the render job from the command line so that you can get it run the python script on startup which is where the frame handler will be created.

It would look something like this:

blender -b GenScene.blend -s 1 -e 100 -a -P GenSceneFrame.py

Let's say for convenience sake that you're going to use a commercial render farm service. They would need to be able to launch your render job with the ability to pipe in the startup python script reference.

You would upload the folder as a Zip file, set the render job for frames 1-100. Then the animation job would be distributed between however many render nodes the service uses, or however many you opt to pay for.

Each frame render runs the internal script and sets up the scene for the current frame number and renders out one frame image. At the end of the job you should have 100 rendered images, one for each scene.

$\endgroup$
  • $\begingroup$ Thanks a lot for the detailed answer. So there is a slight bit of information I think is missing in my question. When I said rendering a scene, I care about a particular frame (say 500 after physics simulation), so current focus is on rendering a zbuffer of this particular frame. Hence I have 100 scenes and I have to render out 100 zbuffer images. However I might need the animation of the physics simulation as well, but thats not the current focus of my project. $\endgroup$ – desinghkar Jul 29 '15 at 16:20
  • 1
    $\begingroup$ Ok, well then you will definitely need to use the command line with Scene number solution in order to have multiple renders going on multiple render nodes. One command line call would set up on scene, bake x number of frames and then render out the desired frame image. $\endgroup$ – Todd McIntosh Jul 29 '15 at 16:22
  • $\begingroup$ Adding to that, my project has lot of iterations and in each iteration my program calls blender to generate 100s of scenes according to specific object locations. So I am not sure if I can use the render farm. So i wanted to do it on a local machine to which i have access all the time. $\endgroup$ – desinghkar Jul 29 '15 at 16:23
  • $\begingroup$ If you are only using 1 machine, then I'm not sure trying to parallel the renders would be helpful. The physics sims generally use multicore processing, so doing it one by one would probably be the way to go. Do you not have this process already automated with scripts? I thought you did. $\endgroup$ – Todd McIntosh Jul 29 '15 at 16:25
  • $\begingroup$ I think the only way to parallelize the process you already have would be to use multiple machines, either locally or remotely. Your launch app would have to be able to call blender headless commands for each machine. Then if they all rendered back to a shared drive, you'd get all the frames in one place. $\endgroup$ – Todd McIntosh Jul 29 '15 at 16:30
1
$\begingroup$

As you have a script creating all the scenes, I can think of three approaches, the render farm or way you distribute the render will influence which suits you.

One is to save a file after each scene is created. At the end of your scene creation loop, instead of rendering you save the scene out to it's own file - bpy.ops.wm.save_as_mainfile(filepath='scene'+str(index)). These individual files can be submitted as separate tasks to the render farm that you are using.

Another is to submit each render task after it is created. This will depend on the render farm you are using, if there is an addon that allows you to submit the task from blender then you can add that operator to your script, instead of bpy.ops.render.render()

The third approach would be to alter your script to only contain the tasks you have inside your loop. Use sys.argv to get a variable passed in when starting blender from the cli and use it as the index value instead of an index range as you loop through. Each machine that starts rendering will be given a different value or range of values to render.

blender -b --python myscript.py -- 5

myscript.py could contain -

import bpy
from sys import argv

SceneIdx = argv[argv.index('--') + 1]

mydata = ReadData('file.dat')
bpy.data.objects['Cube'].location = mydata[SceneIdx]
bpy.ops.render.render()
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.