We have a multi-GPU setup, where we run one instance of an app per GPU. The app is based on a 3D engine running on Direct3D 11, but we do some custom computation using CUDA. How do we deterministically set the correct CUDA device, which corresponds to the same GPU as the Direct3D 11 device used by the engine? The GPU-s in the machine are of identical type, thus they have the same name. The CUDA devices seem to be numbered differently, as in the 3D engine. For some strange reason we get correct behaviour if we always set the CUDA device to 1 (currently we only have 2 cards), but would like to either understand why, or even better: have a solution, which is guaranteed to work correctly everywhere.
Something I’ve tried is getting a DXGI adapter interface from the D3D device, and comparing the properties with the ones I get when enumerating the CUDA devices. DXGI_ADAPTER_DESC2 contains an “AdapterLuid”, but this doesn’t seem to correspond the luid-s in the CUDA device enumeration - they are of a different format.
One promising way of making sure CUDA devices are enumerated in a deterministic order seems to be setting CUDA_DEVICE_ORDER to PCI_BUS_ID. However, I still don’t know in which order they are evaluated in the engine - the maker claims it’s the same order as the order in which GPU-s are listed in the task manager (these are not ordered by PCI bus ID).
Any tips?