Set CUDA device corresponding to D3D11 device

We have a multi-GPU setup, where we run one instance of an app per GPU. The app is based on a 3D engine running on Direct3D 11, but we do some custom computation using CUDA. How do we deterministically set the correct CUDA device, which corresponds to the same GPU as the Direct3D 11 device used by the engine? The GPU-s in the machine are of identical type, thus they have the same name. The CUDA devices seem to be numbered differently, as in the 3D engine. For some strange reason we get correct behaviour if we always set the CUDA device to 1 (currently we only have 2 cards), but would like to either understand why, or even better: have a solution, which is guaranteed to work correctly everywhere.

Something I’ve tried is getting a DXGI adapter interface from the D3D device, and comparing the properties with the ones I get when enumerating the CUDA devices. DXGI_ADAPTER_DESC2 contains an “AdapterLuid”, but this doesn’t seem to correspond the luid-s in the CUDA device enumeration - they are of a different format.

One promising way of making sure CUDA devices are enumerated in a deterministic order seems to be setting CUDA_DEVICE_ORDER to PCI_BUS_ID. However, I still don’t know in which order they are evaluated in the engine - the maker claims it’s the same order as the order in which GPU-s are listed in the task manager (these are not ordered by PCI bus ID).

Any tips?

This is a standard need in CUDA/Graphics interop on windows. Have you looked at the GetDevice/GetDevices functions in D3D11 interop API and/or the D3D11 interop sample codes to see how they do it?

1 Like

Many thanks! That function somehow escaped our attention, and it was just what we needed.