A100 drivers problem - Linux 22.04.3

Hi! I have been struggling for several weeks trying to install drivers for a100. At first, the problem occurred because of the system since I had Linux 22.04.4. However, after moving to 22.04.3, I am still unable to install the drivers. After running the installation with apt (535.161.07) and rebooting I get the information from nvidia-smi:

“NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.”

Please help me with driver installation (I have vGPU splitted with MIG), system information and nvidia logs:

jimmy@a100test:~$ lspci -nnk
03:00.0 3D controller [0302]: NVIDIA Corporation GA100 [A100 PCIe 80GB] [10de:20b5] (rev a1)
DeviceName: pciPassthru0
Subsystem: NVIDIA Corporation GA100 [A100 PCIe 80GB] [10de:1593]
Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia
jimmy@a100test:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy
jimmy@a100test:~$ uname -r

[ 12.520349] kernel: NVRM: The NVIDIA GPU 0000:03:00.0 (PCI ID: 10de:20b5)
NVRM: installed in this system is not supported by the
NVRM: NVIDIA 535.161.07 driver release.
NVRM: Please see ‘Appendix A - Supported NVIDIA GPU Products’
NVRM: in this release’s README, available on the operating system
NVRM: specific graphics driver download page at www.nvidia.com.

nvidia-bug-report.log (2.2 MB)

since it’s a vGPU setup, you’ll have to use the grid driver.

1 Like

I used actually already a grid driver, but it decreased performance significantly after 20 minutes. I found that reason is that I do not have license for grid. Administrator said that it costs so they decided to use MIG approach to split GPU (because it does not require any license) and that I should rather install classical drivers.

Where may I find proper drivers? I need a paid license to use full GPU potential?

Sorry, I found the difference. I do not have vGPU, but rather GPU is splitted with MIG.

An MIG slice is also just a vGPU, it’s created by the vGPU manager driver on the esxi host. So you would need a proper license to use it in a VM.
MIG is license-free when used bare-metal, e.g. assigining different slices to containers.

1 Like

Is there a fee for this license? Will you please send me a link where I can purchase such a license? Also how should I install the drivers for such a configuration?

Yes, there will be fees. I really can’t help you with that as nvidia recently changed their vgpu licensing. Up until v15, compute workloads like yours were included in “Nvidia Virtual GPU” as vCS. Starting with v16, compute workloads are now shifted over to “Nvidia Enterprise AI” at tenfold prices. Please check with the vGPU forums, there someone should be able to give you relevant info, I don’t know anything about it. https://forums.developer.nvidia.com/c/nvidia-virtual-gpu-forums/general-discussion/337