0
$\begingroup$

I'm running blender 2.92 via python on Ubuntu 18.04.5 bionic with RTX 2080 Ti GPUs.

I'm iteratively generating images using cycles. After I generate around 300 images the error in the following comes up and blender crashes. The RTX 2080 has 11GB and each rendering process only takes around 1GB in nvidia-smi so I don't think it's a GPU memory error.

Fra:1 Mem:123.13M (Peak 145.52M) | Time:00:00.16 | Mem:0.00M, Peak:0.00M | Scene, RenderLayer | Loading render kernels (may take a few minutes the first time)
Failed to load CUDA kernel from '/playpen3/home/jmincho/workspace/blender-2.92.0-linux64/2.92/scripts/addons/cycles/lib/kernel_sm_75.cubin' (File not found)

Refer to the Cycles GPU rendering documentation for possible solutions:
https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html

Failed to load CUDA kernel from '/playpen3/home/jmincho/workspace/blender-2.92.0-linux64/2.92/scripts/addons/cycles/lib/filter_sm_75.cubin' (File not found)
Invalid handle in cuModuleGetFunction( &functions.adaptive_stopping, cuModule, "kernel_cuda_adaptive_stopping") (device_cuda_impl.cpp:600)
Invalid handle in cuModuleGetFunction( &functions.adaptive_filter_x, cuModule, "kernel_cuda_adaptive_filter_x") (device_cuda_impl.cpp:602)
Invalid handle in cuModuleGetFunction( &functions.adaptive_filter_y, cuModule, "kernel_cuda_adaptive_filter_y") (device_cuda_impl.cpp:604)
Invalid handle in cuModuleGetFunction( &functions.adaptive_scale_samples, cuModule, "kernel_cuda_adaptive_scale_samples") (device_cuda_impl.cpp:606)
Writing: /tmp/base_scene.crash.txt
Unable to save '/tmp/base_scene.crash.txt': Too many open files
scandir: Too many open files

I don't know what actually causes the error. I confirmed scripts/addons/cycles/lib/kernel_sm_75.cubin exists in the mentioned path. As Too many open files is mentioned, I tried deleting files in /tmp/ but it didn't help either. Any tips or suggestions would be helpful. Thanks.

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ There appears to be an issue with too many open file descriptors, so Blender is unable to both open the CUDA kernel and write the crash log. You would have to check which program is opening an unreasonable number of files on your system or if there is a limit set that is too small. askubuntu.com/questions/181215/… $\endgroup$
    – Robert Gützkow
    Commented May 12, 2021 at 15:08
  • $\begingroup$ @RobertGützkow My previous max file descriptor limit (ulimit -n) was 1024. I increased it to 70000 and now the script works. Thanks! $\endgroup$
    – J-min
    Commented May 12, 2021 at 21:56

1 Answer 1

Reset to default
0
$\begingroup$

As @RobertGützkow mentioned, it was because too many files were opened. Blender was unable to open the CUDA kernel after it reached the system's maximum file descriptor limit. ulimit -n gave me 1024. I increased the limit by ulimit -n 70000 and the error disappeared.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .