CUDA Vulkan interconnection: validity of const device pointer address

Accelerated Computing CUDA CUDA Programming and Performance
1

Dear CUDA experts,

I have a question regarding the interconnection between CUDA and Vulkan textures.

I process off-screen rendered frames in CUDA using a Vulkan texture pointer.
The Vulkan texture pointer is used to call cudaImportExternalMemory with a properly filled cudaExternalMemoryHandleDesc struct where the file descriptor is retrieved with the 3rd party API Unigine::ResourceExternalMemory::getFdHandle().
From this a device pointer is obtained and utilized in our CUDA image post processing.

For the same Vulkan texture (the off-screen render target/frame buffer is allocated only once at the beginning), I want to process several consecutive frames (can vary from 60 to 30 FPS) over a given period of time.
Ideally, I want to create the external memory handle in CUDA only once as the device pointer seems
to point to a constant address.
What we don’t know: how valid is the assumption that the address of the device pointer and therefore
the address of the underlying frame buffer is constant? Is it possible that these addresses change
over time - for instance to accommodate other applications running on the GPU?

Your help is highly appreciated, thanks a lot in advance!

Best wishes.
Philipp

System specification: Kubuntu 22.04, CUDA 11.8, GeForce RTX 3050

2

Can someone maybe point us in the right direction?
We would be glad to discuss this topic further.
Thanks in advance.

3

Any help would be highly appreciated

6

Hi Philipp, got this hint from a devtech colleague…:

Vulkan images are allocated at a fixed virtual device address and importing them into CUDA also gives the/a fixed address. In fact, one should just import the memory once at the beginning, rather than doing that again every frame. So their assumption is correct.

Do hope this answers your question?

Regards
-Frank

7

Hi Frank,
apologies for the late reply. This answers my question, thanks a lot!

To sum up further insights from discussions with experts:
The same behavior with a fixed virtual device address also applies to DirectX12.
Older graphics APIs like DirectX11 or OpenGL can behave differently - here, the OS / driver can perform reallocations dynamically.
CUDA experts, please feel free to correct any of the above mentioned behavior in case I misunderstood something.

Best regards
Philipp

8

Perhaps there is a way to see for each frame, whether the address stayed constant, without going through all the (expensive) steps. By it you could safeguard your program and still profit from this optimization.