1
$\begingroup$

We're running a headless renderserver (Threadripper 1950x + 2x RADEON VII + 128GB RAM). It's running Ubuntu 20.04 LTS and has blender 2.82, 2.80 and 2.79 installed.

In 2.82 we're seeing the CPU being utilized 100% for the complete duration of the render. I'm struggling to find out why.

OUR SETUP

In our setup, our designers can press an 'upload' button in a Blender addon, this copies the file to the render server. They can than specify in browser whether or not they want to use GPU or CPU. In case of CPU, Cycles is running in CPU mode on the threadripper. In case of GPU, both RADEON VII's are selected and Cycles runs in GPU/OPENCL mode.

this used to work great. As we could also schedule 2 renders at the same time (one on the CPU, and one on the GPU). If the scene was to big to fit in GPU memory, there is an automatic failover that makes it use CPU (non OPENCL) instead.

CURRENT PROBLEM

When uploading a file made with blender 2.82 we still select 'GPU' in our browser. It detects that we're using 2.82 and starts executing the following script:

echo "2.82 running"
echo "renderType: $renderType"
/app/blender2_82/blender --background "$INPUT_DIR$filename" --python "/app/renderer/$renderType.py" --python "$borderScript"

In this:

  • the $renderType comes from the information provided by our webclient (it equals GPU)
  • $INPUT_DIR$filename refers to the correct path
  • $renderType.py specifies a scripts that fixes the right rendertype and devices (so it actually loads the GPU.py script (see below)
  • $borderScript is able to change whether or not the entire frame is rendered and is created automatically by our system (see below)

The GPU.py script:

import bpy
from bpy import data as D
from bpy import context as C
from mathutils import *
from math import *

print("GPU SCRIPT EXECUTED")

for scene in bpy.data.scenes:
    scene.render.tile_x = 256
    scene.render.tile_y = 256
    scene.render.engine = 'CYCLES'
    scene.cycles.tile_order = 'CENTER'
    scene.cycles.device = 'GPU'
    print("engine: " + scene.render.engine)
    print("device: " + scene.cycles.device)

print("Compute device type: " + bpy.context.preferences.addons['cycles'].preferences.compute_device_type)
print("Compute device type: " + bpy.context.preferences.addons['cycles'].preferences.compute_device_type)
print("DEVICE USAGE: ")
for device in bpy.context.preferences.addons['cycles'].preferences.devices:
   print("  - " + device.name + " : " + str(device.use))

the borderscript:

import bpy
#bpy.context.scene.render.border_min_x = 0
#bpy.context.scene.render.border_max_x = 1
#bpy.context.scene.render.border_min_y = 0
#bpy.context.scene.render.border_max_y = 1
#bpy.context.scene.render.use_border = True

#print("BORDER SCRIPT EXECUTED")

def add_subframe_to_fo_path(scene):
  scene.node_tree.nodes['File Output'].base_path = str('/app/output/') + 'Job' + str('01305') + '-' + str('D1022_Londerzeel_EXT2.blend.2020.07.27-10.48.59') + '/parts'
  #for file_slot in scene.node_tree.nodes['File Output']:
  #  file_slot.path=file_slot.path + '_Job' + str('01305') + '_Task' + str('1374')

for scene in bpy.data.scenes:
  bpy.app.handlers.render_pre.append(add_subframe_to_fo_path(scene))

bpy.ops.render.render(animation=False)

I know both scripts are executed by the logs and the fact that it actually starts rendering:

2.82 running
renderType: GPU
Blender 2.82 (sub 7) (hash 375c7dc4caf4 built 2020-03-12 05:30:40)
Read prefs: /home/charmeleon/.config/blender/2.82/config/userpref.blend
found bundled python: /app/blender2_82/2.82/python
Read blend: /app/ingested/D1022_Londerzeel_EXT2.blend.2020.07.27-10.48.59
GPU SCRIPT EXECUTED
engine: CYCLES
device: GPU
Compute device type: OPENCL
Compute device type: OPENCL
DEVICE USAGE:
  - AMD Radeon VII : True
  - AMD Radeon VII : True
  - AMD Ryzen Threadripper 1950X 16-Core Processor : False

Logs

It even loaded the kernel changes for OpenCL rendering:

Fra:8 Mem:10586.62M (0.00M, Peak 13052.30M) | Time:00:50.82 | Mem:4862.77M, Peak:4909.48M | Scene, View Layer | Updating Device | Writing constant memory
Fra:8 Mem:10586.62M (0.00M, Peak 13052.30M) | Time:00:50.82 | Mem:4862.77M, Peak:4909.48M | Scene, View Layer | Updating Device | Writing constant memory | Compiling render kernels
Fra:8 Mem:10586.62M (0.00M, Peak 13052.30M) | Time:00:50.82 | Mem:4862.77M, Peak:4909.48M | Scene, View Layer | Updating Device | Writing constant memory
Fra:8 Mem:10621.82M (0.00M, Peak 13052.30M) | Time:00:50.82 | Mem:4862.77M, Peak:4909.48M | Scene, View Layer | Rendered 0/150 Tiles, Denoised 0 tiles

Yet my processor is completely pegged and my GPU fans aren't even spinning. enter image description here

Some addendums:

  • Due to the fact that GPU uses 256 tiles, that the CPU is pegged and that we can start a parrallel job on the CPU we've got a performance that less than 25% of what we had when using 2.80 or 2.79.

  • Our Radeon VII's have 16GB of graphical memory, so below any of the memory readouts in the logs. I used to think our scenes were just to big and it started 'swapping' to system memory, but that does not appear to be the case.

$\endgroup$
1
$\begingroup$

I can only tell you how I have it running on our slurm cluster with CUDA, but the script also seems to support OpenCL.The GPU.py script seems fundamentally different from yours.

./blender \
        -b "[blendfile]" \
        -o "./video/frame#######" \
        -P ../../../gpu.py \
        -s [startframe] \
        -j [frame jumps, to render with multiple thread] \
        -a

gpu.py

import bpy

def enable_gpus(device_type):
    preferences = bpy.context.preferences
    cycles_preferences = preferences.addons["cycles"].preferences
    cuda_devices, opencl_devices = cycles_preferences.get_devices()

    if device_type == "CUDA":
        devices = cuda_devices
    elif device_type == "OPENCL":
        devices = opencl_devices
    else:
        raise RuntimeError("Unsupported device type")

    activated_gpus = []

    for device in devices:
        if device.type == "CPU":
            device.use = False
        else:
            device.use = True
            activated_gpus.append(device.name)

    cycles_preferences.compute_device_type = device_type
    bpy.context.scene.cycles.device = "GPU"

    return activated_gpus


enable_gpus("CUDA")
| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.