4
$\begingroup$

I've got a Ubuntu 14.04 headless server running right now for 3d rendering on Blender's Cycles rendering engine, across 4 GPUs.

Specs:

(4) ASUS GTX 970s, 4GB
1300W PSU
Core i7
16GB RAM
250GB SSD

Animations are rendering beautifully across the 4 GPUs in Cycles, but my question is concerning parallelizing processes to those cards:

Is it possible for me to have 4 separate blender processes running in parallel targeting one specific GPU each?

Running all 4 cards as "one" card (CUDA_MULTI_0 in a .py setup) is not actually a full 4x speed increase over running just one card, though it is still quite fast, and amazing i might add having come from mostly CPU rendering. Anyway, it turns out to be more like 3x speed, and i want to maximize render times further by rendering one animation frame per card in parallel, giving me a true 4x speed.

Theoretically, it should just be a matter of running 4 separate blender processes on the command line, while having python scripts setup to target specific cards like this:

$ blender -b file.blend -P cuda1.py -s 1 -e 10 -a &
$ blender -b file.blend -P cuda2.py -s 11 -e 20 -a &
$ blender -b file.blend -P cuda3.py -s 21 -e 30 -a &
$ blender -b file.blend -P cuda4.py -s 22 -e 40 -a &

and the cuda[x].py scripts look like this, where each one is targeting a specific CUDA device:

import bpy, _cycles

bpy.context.scene.cycles.device = 'GPU'
bpy.context.user_preferences.system.compute_device_type = 'CUDA'

# this is different in each cuda[x].py file, CUDA_0, CUDA_1, CUDA_2, CUDA_3
bpy.context.user_preferences.system.compute_device = 'CUDA_0'

So, this all technically works right now, but here's the big issue i'm trying to figure out. When i do this, only the first blender process gets GPU access. Processes 2-4 default back the CPU, ignoring the GPU instructions. All four cards work just fine, and as I mentioned, one blender process using the CUDA_MULTI_0 device in the python script is leveraging all four cards as expected.

Is there some known limitation to blender, or CUDA/NVIDIA for that matter, that only lets a single processes access any or all the GPUs at one time, that maybe i'm just not aware of?

Any advice on this is greatly appreciated.

$\endgroup$
2
$\begingroup$

It turns out all I needed was to update the NVIDIA driver to v346.72 for the ASUS GTX 970 Strix cards to be able to handle multiple processes in tandem.

Update - Oct 2019

To those wondering if there was a performance gain in doing this: yes, there was.

I don't have the specifics with me as this was done a few years back, but I do remember the render times breaking down like this simplified example:

1 GPU renders 1 frame of an animation in 4 minutes, so in theory bulking up to 4 GPUs should render that one frame in 1 minute. This wasn't what I was seeing in reality though. The actual result took more like 1.2 minutes per frame.

So for a 200 frame animation, the math broke down like this:

All 4 GPUs working as one:

  • (200 frames / 1 Combined GPU) = 200 frames * 1.2 minutes per frame = 240 mins total

Splitting up the GPUs to independently work on a queue of frames:

  • (200 frames / 4 GPUs) = 50 frames per GPU * 4 minutes per frame in parallel = 200 mins total
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.