Failed to initialize NVML: Unknown Error

Hello

I want to enable VGPU on my VMware environment. I encounter an issue and run out of troubleshooting methods.

environment:

  1. virtualization:
    VMware ESXi 8.0.2, 22380479
    vCenter server 8.0.2.00100

2.hardware:
Supermicro SYS-1029U-TR4
GPU NVIDIA TU104GL [Quadro RTX 4000]

  1. Drivers and modules for NVIDIA on ESXi

esxcli software vib list | grep -i nvd

NVD-VMware_ESXi_8.0.0_Driver 535.154.02-1OEM.800.1.0.20613240 NVD VMwareAccepted 2024-01-22 host
nvdgpumgmtdaemon 535.154.02-1OEM.700.1.0.15843807 NVD VMwareAccepted 2024-01-22 host

  1. Errors:

nv-hostengine -d

Error: Failed to initialize NVML

nvidia-smi

Failed to initialize NVML: Unknown Error

Problem:
I’ve installed latest VIB and encountered the errors above. Problem is that I can’t find why.
I try checking logs, dmesg, read throuh nvidia-bug-report.sh

I can successfuly load NVIDIA kernel module, see the device both on ESX and vSphere.

How can I troubleshoot better?

Hi,
what are you trying to achieve? You cannot use vGPU for a GPU that doesn’t support vGPU!

Regards
Simon

You’re right - I wanted to use it for vGPU, but I checked more and it’s confirmed that rtx4000 doesn’t support vGPU. Still, I’m a bit dissapointed that there’s no sufficient info when troubleshooting