I'm just trying to set up my workstation for Blender and got caught by a weird problem:
Installing just a single GPU (AMD Radeon RX Vega 64) in the machine works fine with Blender. The GPU shows properly via "clinfo", is shown properly in Blender in the System Tab as available OpenCL device and works fine (and fast) for rendering with Cycles.
Now installing a second, identical GPU Blender suddenly doesn't find any OpenCL-GPU at all. The System tab in the User Preferenes shows no available OpenCL GPUs, at all. Curiously "clinfo" still finds both GPUs just fine.
- CPU: AMD Ryzen 7 1700
- GPU: 2x AMD Radeon RX Vega 64
- OS: Ubuntu 18.04
- Driver: amdgpu-pro-18.50-725072-ubuntu-18.04
- Blender: both v2.79b and v2.80 beta
Installing just one GPU everything works fine in Blender, installing both none is useable.
Anybody any clue, what is going wrong and how to get both GPUs working with Blender?
With the help of the Blender source-code I was able to figure out a little bit more information:
With both GPUs installed, when the Cycles code calls "clGetPlatformIDs()" during initialization, this function returns with error-code -1001, i.e. CL_PLATFORM_NOT_FOUND_KHR from the "cl_khr_icd"-extension; apparently meaning that the OpenCL ICD-loader library is not able to find any usable OpenCL ICD libraries.
Compiling and running the simple "clewTest" from https://github.com/OpenCLWrangler/clew, which seems to perform the same CLEW/OpenCL initialization as the Cycles renderer, works fine, though. In this case "clGetPlatformIDs()" returns successfully and the OpenCL ICD lib can be used.
In both cases the same OpenCL ICD loader lib is used (confirmed in both cases via "dlinfo()" at runtime).
Both cases behave the same no matter if AMD's ICD-loader lib (from the AMDGPU-PRO driver package) or the generic open-source Khronos ICD-loader lib is used.
On Linux all OpenCL ICDs are registered via files in sub-directories of "/etc/OpenCL". These are the same in all cases, all registered ICD-libs are also present in all cases and identical.
Again, with just one GPU installed, everything works fine in all cases and Blender/Cycles can access and use the GPU via OpenCL.
Does Blender maybe somehow change search paths for dynamic libraries or other environment variables, which might prevent the OpenCL ICD libs from being found or loaded successfully by the ICD-loader? Something must be different for Blender, because "clewTest" works fine using the same API-calls...
Who is actually using multiple (AMD?) GPUs via OpenCL in Blender running on Linux successfully?
(While I found a large number of posts from people using multiple AMD-GPUs, most seem to be using Windows).
(I posted my question as guest without login, at first, and didn't see a way to edit this post later. Therefore I posted additional information as answer. Sorry about that.)
I'm just using the source-code to try to figure out what is going wrong on my system and so far I can't see that Blender itself is doing anything wrong or strange. On the contrary: so far it seems that Blender is doing exactly the same as "clewTest", but while for the this later succeeds, for Blender it doesn't, for some reason.
I guess my next step will be to take the source-code of the generic Khronos ICD-loader from GitHub, enable tracing in that and try to figure out why it can't find any ICDs when Blender is trying to use it with 2 GPUs installed.
I guess it's something mis-configured on my system in some way.
Looking online there seem to be thousands of people using Blender on multi-GPU machines. While most of them seem to prefer nVidia-cards with CUDA and/or are running Blender on Windows (I don't have any Windows to test with), which might work fine, I suppose there must be others out there using Blender with multiple AMD GPUs on Linux, though, and I was hoping that these people might provide some hints how they got it working (or maybe just the info "worked for me just fine out of the box", which at least would prove my assumption that something is mis-configured on my system).
One point I'm wondering about: With both GPUs installed, both only run in PCIe-x8-mode on my system. Unfortunately I can't force PCIe-x8-mode with just one GPU installed (and "clewTest" works with two GPUs installed), but has anyone out there maybe had issues with AMD GPUs running in PCIe-x8-mode for Cycles' OpenCL-rendering?
CONLUSION (sort of)
After some more experiments it turns out that the OpenCL-ICD-lib itself from the AMDGPU-PRO driver package refuses to start with Blender in a Dual-GPU configuration (return error-code -32: CL_INVALID_PLATFORM).
It does work, though, if I start Blender with the "-b" option to run it as a background-process without GUI.
So probably some weird kind of interaction between 2xGPU + OpenGL + OpenCL which the Linux AMDGPU-PRO OpenCL-driver itself doesn't like... Bummer...
FINALLY WORKING!!! (in June 2019)
Since my original post I updated - my OS to Ubuntu 18.04.2 with HWE (Linux Kernel 4.18) - AMD GPU driver to: amdgpu-pro-19.20-812932-ubuntu-18.04.tar.xz
I don't know which of these two updates made the difference, but now my Dual RX Vega 64 setup is working fine, now, both in Blender v2.79b as well as different v2.80 beta-versions.
(And the speed-up of the Cycles Renderer compared to a single RX Vega 64 is between 50-100% depending on the scene.)