2
$\begingroup$

I'm trying to find out if pooling VRAM between multiple cards is actually possible--I've received conflicting information.

In a recent chat with NVidia tech support about my Quadro RTX 5000, I was told there is no way for me to pool vram with multiple cards.

However the Blender 2.9 User Guide says this:

Distribute Memory Across Devices

Allocates resources across multiple GPUs rather than duplicating data, effectively freeing up space for larger scenes. Note that in order for this option to be available, the GPUs must be connected together with a high bandwidth communication protocol. Currently only NVLink on Nvidia GPUs is supported.

Specifically, I'd love to pool VRAM while using Optix GPU rendering with Cycles. I would consider purchasing another Quadro RTX if I know for sure that it works.

Does anyone have definitive information on this?

$\endgroup$
1
  • 2
    $\begingroup$ This differential for this feature is D7426. You may want to ask Patrick Mours or Brecht van Lommel on Blender Chat or DevTalk if it would work for your intended use case. $\endgroup$ Commented Sep 22, 2020 at 19:33

1 Answer 1

2
$\begingroup$

TLDR: it is possible; Blender team should know better, as well as tech people at Nvidia.

The ability to pool memory depends on the implementation of the software using it for Geforce cards, and I understood that it can be automatic (I understand managed by the driver) with profesionnal display cards ie., Quadro (generations before Ampere), and later RTX A6000. For the non display professional/enterprise cards (A100, in DGX Station A100, DGX A100), it happens with a multiway nvlink.

Cards however must be of the same generation, The NVlink is updated from generation to generation and is not backward/forward compatible.

Test with Turing generation Geforce

Memory pooling works with Turing generation Geforce cards (2080, 2080Ti, Titan RTX), in GNU/Linux and Windows 10.

reference: https://www.pugetsystems.com/labs/articles/NVLink-on-NVIDIA-GeForce-RTX-2080-2080-Ti-in-Windows-10-1253/

Comments on cuda 11.2 and pooled memory: Stream-ordered memory allocator

One of the highlights of CUDA 11.2 is the new stream-ordered CUDA memory allocator. This feature enables applications to order memory allocation and deallocation with other work launched into a CUDA stream such as kernel launches and asynchronous copies. This improves application performance by taking advantage of stream-ordering semantics to reuse memory allocations, using and managing memory pools to avoid expensive calls into the OS. The new asynchronous memory allocation and free API actions allow you to manage memory use as part of your application’s CUDA workflow. For many applications, this reduces the need for custom memory management abstractions, and makes it easier to create high-performance custom memory management for applications that need it. Moreover, this feature makes it easier to share memory pools across entities within an application.

cudaMallocAsync(&ptr, size, stream); // Allocates physical memory
kernel<<<...,stream>>>(ptr);
cudaFreeAsync(ptr, stream);          // releases memory back into a pool
cudaMallocAsync(&ptr, size, stream); // Reuses previously freed pointer
kernel<<<...,stream>>>(ptr);
cudaFreeAsync(ptr, stream);          // releases memory back into a pool
....                                 // Executes other work in the stream

As shown in this example, CUDA 11.2 introduces new stream-ordered versions of cudaMalloc and cudaFree—called cudaMallocAsync and cudaFreeAsync—which take a stream as an additional argument. The first call to cudaMallocAsync in the example allocates memory from the OS, but the subsequent call to cudaFreeAsync does not free it back to the OS. Instead, the memory is stored in a pool maintained by the CUDA driver, which allows the second call to cudaMallocAsync to reuse the memory previously freed, if it is of sufficient size.

reference: https://developer.nvidia.com/blog/enhancing-memory-allocation-with-new-cuda-11-2-features/

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .