System with Tesla V100 + RTX 4090?

Sorry if this isn’t the best place to post this, it’s the closest forum category that made sense.

I have a spare V100 on an eGPU, it’s working fine. I have a Windows 11 system with a 4090 (internal) & a 2080ti (eGPU). The V100 will not work at the same time as the other 2 since they’re GeForce whereas the V100 is a datacenter class GPU. I understand that NVIDIA doesn’t package their driver sets to support both at the same time. I had at one point successfully had a Quadro RTX 8000 GPU (workstation class) on that 2nd eGPU so certain driver sets did work with with both GeForce and workstation GPUs.

Is there any way to make this work? Just trying to get additional AI inference capacity on the system. Nothing heavy duty, just LM Studio. In the worst case, would GPU pass-through work if one set or the other were passed to a VM with compatible drivers there?

Note that I’ve tried the datacenter 582.70 drivers which includes all 3 drivers, but the system is not allowing the GeForce GPU drivers to load alongside the V100. Interestingly, depending on how I connect the eGPU to control enumeration, if the V100 enumerates first it works fine but the other 2 don’t and if one of the other 2 enumerate first then they work and the V100 doesn’t. I don’t understand why NVIDIA would provide a driver package that contains drivers for all 3 but not allow them to run together.

A windows driver will enumerate (i.e. be recognized for system use) when the ID of the device in question is listed in something called an INF file in the driver. Windows WHQL certification complicates this a bit, but that is the basic notion. (WHQL should not do anything except line up with the INF specified IDs, for a proper driver package.)

NVIDIA does indeed make drivers with INFs that don’t cover every possible device. However the driver that comes with the CUDA toolkit installer will be the least restrictive driver, usually. So my advice first of all is that you use a CUDA toolkit installer to try to make this work, vs. relying on windows auto-installed drivers or anything else.

This could be an explanation as to why different drivers seem to either support or not support specific GPUs or specific GPU families.

If you have a driver package that will work with any of the 3 GPUs individually, then NVIDIA, to my knowledge, does not further restrict its use when all 3 are present together. In my experience, the issue then would be that the system itself is having trouble enumerating (plug-and-play PnP enumeration) all 3 devices at the same time. This is not a NVIDIA issue but a platform issue (some combination of the system BIOS and the OS).

Its fairly easy to identify this problem. You would usually see a yellow “bang” (warning icon) on the device in the list in device manager, if the system is having trouble making the device ready for the GPU driver to use. The system itself needs to allocate and properly assign resources to the device, and if you have a yellow bang, then one possible reason is that the system could not allocate resources properly. In that situation, the driver cannot fix it or use such a device. But it is not because the driver is actively preventing the device from being used.

As it happens, I’m on the cusp of retirement from NVIDIA. The login that I use will become deactivated at that point, and it won’t be possible for me to use this persona to comment on these forums. But with a bit of searching on these forums or on the internet, you can find various reports of people trying these things, their experiences, and comments similar to my own above.

The only corrective advice I generally offer is that if your system BIOS is not currently at the latest version offered by the motherboard manufacturer, then its usually good practice to update it to the latest. So update the BIOS to latest, then try an appropriate CUDA toolkit. By “appropriate”, I mean you’ll need one that supports all 3 GPUs. The V100 is compute capability 7.0, the 2080ti is compute capability 7.5, and the 4090 is compute capability 8.9. So using a R580 driver that comes with a CUDA toolkit install would be a good choice, if your 582.70 is recognizing the V100.

None of this should be construed as any sort of guarantee, however.

Regarding your question about VMs, if your base OS is Windows, I wouldn’t expect any VM system on top of windows (such as DosBox) to work correctly. If your base setup is not windows but say a proper VM environment such as VMWare or whatever it is called now (Omnissa?), then that is a separate topic. I think that would be a jungle to hack through, and I don’t have expertise or advice there. The usual advice from NVIDIA for people wanting to use virtualized environments would be to use a NVIDIA system like vPC or vWS. I think getting those 3 devices to work in a single VM environment like that would be difficult or impossible. NVIDIA vGPU based hypervisor solutions like vPC or vWS do not work with consumer grade GPUs.

That sounds exactly like a platform resource allocation issue, not something fixable or under the control of NVIDIA drivers.