Nvidia-smi "No devices were found" - VMWare ESXI Ubuntu Server 20.04.03 with RTX3070

Did you pass through any advanced config parameters from ESXi to get it working?

You made more progress than I on Ubuntu. There is a package on Debian called nvidia-driver and v460 was the version it “picked” for me. That was what I got working.

Nice catch on the driver,

I did, I used this YouTube video - PCI pass-through & Folding at Home on the VMware appliance (VMware vSphere ESXi 7) Jason Meers - YouTube

PCI pass-through & Folding at Home on the VMware appliance (VMware vSphere ESXi 7) Jason Meers

hypervisor.cpuid.v0 False
pciPassthru.use64bitMMIO True
pciPassthru.64bitMMIOSizeGB 32 - Single GPU option.

Memory - Check All guest memory (All locked)
Memory Limit - Unlimited

Boot Option EFI

Hi there, I met the same problem and my thread is Nvidia-smi shows "No devices were found", and dmesg shows "rm_init_adapter failed, device minor number 0"

By following @generix 's advice, I reinstall a Ubuntu 20.04 under ESXi, do passthrough as @twine.david said (except using EFI boot), downgrade the driver to 460.84, and hold my linux driver to 5.4.0-100-generic, now I can get my GPU in nvidia-smi as follows:

Because new users can’t upload more than 1 pictures, I split it up.

Interestingly, I don’t know why another hypervisor with nvidia-driver 470 can recognize GPU.

And I try another bare metal with Ubuntu 20.04, kernel 5.13.0-27-generic, nvidia-driver 510.47, GPU is GTX Titan X and it works. But the same conbination of kernel and driver is not work on ESXi.

As generix said, it seems like ESXi can’t support nvidia-driver >= 470 greatly.

Furthermore, to confirm kernel 5.13 can work with low version nvidia-driver, I made a bare metal with Ubuntu 20.04, kernel 5.13.0-28-generic, nvidia-driver 465.31, GPU is RTX 3090, it still works.

Because the apt source stopped to provide nvidia-driver under 470, I download .run file from nvidia website and install manually. To prevent the driver auto upgrade, I stopped autoupgrade function of apt.

Hope it can helps! <3

Did you install the driver via a .run file? I don’t see a v460 available via apt search.

EDIT: Missed your second post where you mentioned .run file.

I agree it seems to be ESXi/Nvidia related and not hypervisor related as I got it to pass through on Proxmox, even with the latest 510 drivers.

Yep. It looks like apt merged drivers from 418-470 to 470. I downloaded a 460 driver from Official Advanced Driver Search | NVIDIA

Hi. Does anyone find a different solution other than using a lower version driver? The VM I’m using is managed by university so I couldn’t really turn off the auto upgrade.

My current situation is very similar to @DPCFish. One VM works perfectly fine with GTX 1080. But the other one with the same VM setting and Titan RTX failed to initialize the driver.

$ lspci | grep VGA can see the device.
$ dmesg | grep -i nvrm on 470.103 driver returns
NVRM: GPU 0000:03:00.0: RmInitAdapter failed! (0x26:0x56:1253)
on 510 driver returns error code 1463.

It seems work on workstation level card.


A4000 + Ubuntu 20 + ESXi 7.0U3

A5000 + Ubuntu 18 + ESXi 7.0U3

hello,I met the same question when i use 3080 on ubuntu 20.04.4 ESXi6. 7.U2 . nvidia-driver is 470.57.01. I want to use ubuntu20.04.3 and nvidia-driver6-460. Do you have installation package about ubuntu 20.04.3.iso and nvidia-driver-460 ?could you email to me (qq:1210586191)?

Hi,

I have just the same problem with my 1650. Trying right now with the 460 driver.

Update: 460 driver (first disable nouveau) works. I hope there wil be a fix fast for the new drivers.

Any Updates?
all Drivers above 470.57 wont work with ESXI → Ubuntu and my RTX 30er Series.
on Windows VMs no Problems.
need the new drivers (5XX)

i have the same issue, need working 510 drivers for ESXi

What changes were made in 470.86 because 470.82.01 works but 470.86 upwards doesn’t?

I’m just here to throw some wood on the fire.

Guess what? The beta release of 515.43.04 open source driver works with Ampere cards, but the proprietary driver shipped in the same installer does not. Hmmmmmmmmmmmmmmmmmmm

I am in the same position as others in the thread trying to use a 3090 in ESXi 7.0U3c with DirectPath I/O (PCIe passthrough). I confirm that the 470.82 drivers were the last proprietary drivers to work until 515.43.04/kernel-open I also want to reiterate that the properietary kernel driver 515.43/04/kernel still fails with the same errors as discussed in this thread.

Per the instructions here, install with: ./NVIDIA-Linux-x86_64-515.43.04.run -m=kernel-open and make sure to include options nvidia NVreg_OpenRmEnableUnsupportedGpus=1 in /etc/modprobe.d/nvidia

I lowkey suspect this whole “bug” is making it difficult to do this by design, and whatever chicanery NVidia is up to in the proprietary driver isn’t gonna fly on open source. Hooray for open source!

$ dmesg | grep vmware
[ 0.000000] vmware: TSC freq read from hypervisor : 2992.968 MHz
[ 0.000000] vmware: Host bus clock speed read from hypervisor : 66000000 Hz
[ 0.000000] vmware: using clock offset of 18001067827 ns
[ 5.414545] systemd[1]: Detected virtualization vmware.

$ nvidia-smi
Thu May 19 16:38:19 2022
±----------------------------------------------------------------------------+
| NVIDIA-SMI 515.43.04 Driver Version: 515.43.04 CUDA Version: 11.7 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce … Off | 00000000:03:00.0 Off | N/A |
| 30% 38C P8 35W / 350W | 5MiB / 24576MiB | 0% Default |
| | | N/A |
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 2942 G /usr/lib/xorg/Xorg 4MiB |
±----------------------------------------------------------------------------+