No GPU Profiles - nvidia-smi error

Followed the Cisco guide to installing everything for our M10 GPUs. When the cards were intially installed, nvidia-smi gave the correct information. Now they respond with:

NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

Running esxcli software vib list | grep -i nvidia gives:

NVIDIA-VMware_ESXi_6.7_Host_Driver 390.94-1OEM.670.0.0.8169922 NVIDIA VMwareAccepted 2019-01-28

Not sure what’s happening or how to fix it.

Just in case someone sees this when searching. I figured out what went wrong.

On page 42 of this document:

https://www.cisco.com/c/dam/en/us/solutions/collateral/data-center-virtualization/desktop-virtualization-solutions-vmware-horizon-view/whitepaper_c11-739654.pdf

It shows how to enable passthrough, if you’re using vDGA. You don’t want passthrough enabled if you’re using vGPU. Disabling passthrough and rebooting the hosts (again…) fixed the issue.