Switched to lowlatency kernel and now nvidia driver not loading

I have a Lenovo/Thinkpad P52 with a Quadro P1000 Mobile that drives my external displays. They just stopped working.

I wanted to expand my audio capabilities. So, I followed the Ubuntu Studio Installer docs which resulted in my kernel becoming lowlatency and now the nvidia kernel modules can’t be found.

root@computer:~# cat /proc/version
Linux version 5.15.0-69-lowlatency (buildd@lcy02-amd64-026) (gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #76-Ubuntu SMP PREEMPT Mon Mar 20 16:03:40 UTC 2023
root@computer:/home/user# lshw -C display
  *-display UNCLAIMED       
       description: VGA compatible controller
       product: GP107GLM [Quadro P1000 Mobile]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:01:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list
       configuration: latency=0
       resources: memory:a3000000-a3ffffff memory:60000000-6fffffff memory:70000000-71ffffff ioport:2000(size=128) memory:a4080000-a40fffff
       description: VGA compatible controller
       product: CoffeeLake-H GT2 [UHD Graphics 630]
       vendor: Intel Corporation
       physical id: 2
       bus info: pci@0000:00:02.0
       logical name: /dev/fb0
       version: 00
       width: 64 bits
       clock: 33MHz
       capabilities: pciexpress msi pm vga_controller bus_master cap_list rom fb
       configuration: depth=32 driver=i915 latency=0 mode=3840x2160 resolution=3840,2160 visual=truecolor xres=3840 yres=2160
       resources: iomemory:600-5ff iomemory:400-3ff irq:150 memory:604a000000-604affffff memory:4000000000-400fffffff ioport:3000(size=64) memory:c0000-dffff
root@computer:/home/user# dkms status
vhba/20211218, 5.15.0-60-generic, x86_64: installed
vhba/20211218, 5.15.0-69-lowlatency, x86_64: installed
vhba/20211218, 5.19.0-38-generic, x86_64: installed
root@computer:/home/user# nvidia-smi
NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.
root@computer:/home/user# modinfo nvidia
modinfo: ERROR: Module nvidia not found.
root@computer:/home/user# modprobe nvidia
modprobe: FATAL: Module nvidia not found in directory /lib/modules/5.15.0-69-lowlatency
root@computer:/home/user# ubuntu-drivers devices
== /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0 ==
modalias : pci:v000010DEd00001CBBsv000017AAsd0000225Fbc03sc00i00
vendor   : NVIDIA Corporation
model    : GP107GLM [Quadro P1000 Mobile]
driver   : nvidia-driver-525 - distro non-free recommended
driver   : nvidia-driver-390 - distro non-free
driver   : nvidia-driver-470 - distro non-free
driver   : nvidia-driver-418-server - distro non-free
driver   : nvidia-driver-525-server - distro non-free
driver   : nvidia-driver-450-server - distro non-free
driver   : nvidia-driver-510 - distro non-free
driver   : nvidia-driver-515 - distro non-free
driver   : nvidia-driver-470-server - distro non-free
driver   : nvidia-driver-515-server - distro non-free
driver   : xserver-xorg-video-nouveau - distro free builtin
root@computer:/home/user# dpkg -l | grep -i nvidia.*525
ii  libnvidia-cfg1-525:amd64                       525.89.02-0ubuntu0.22.04.1              amd64        NVIDIA binary OpenGL/GLX configuration library
ii  libnvidia-common-525                           525.89.02-0ubuntu0.22.04.1              all          Shared files used by the NVIDIA libraries
ii  libnvidia-compute-525:amd64                    525.89.02-0ubuntu0.22.04.1              amd64        NVIDIA libcompute package
ii  libnvidia-compute-525:i386                     525.89.02-0ubuntu0.22.04.1              i386         NVIDIA libcompute package
ii  libnvidia-decode-525:amd64                     525.89.02-0ubuntu0.22.04.1              amd64        NVIDIA Video Decoding runtime libraries
ii  libnvidia-decode-525:i386                      525.89.02-0ubuntu0.22.04.1              i386         NVIDIA Video Decoding runtime libraries
ii  libnvidia-encode-525:amd64                     525.89.02-0ubuntu0.22.04.1              amd64        NVENC Video Encoding runtime library
ii  libnvidia-encode-525:i386                      525.89.02-0ubuntu0.22.04.1              i386         NVENC Video Encoding runtime library
ii  libnvidia-extra-525:amd64                      525.89.02-0ubuntu0.22.04.1              amd64        Extra libraries for the NVIDIA driver
ii  libnvidia-fbc1-525:amd64                       525.89.02-0ubuntu0.22.04.1              amd64        NVIDIA OpenGL-based Framebuffer Capture runtime library
ii  libnvidia-fbc1-525:i386                        525.89.02-0ubuntu0.22.04.1              i386         NVIDIA OpenGL-based Framebuffer Capture runtime library
ii  libnvidia-gl-525:amd64                         525.89.02-0ubuntu0.22.04.1              amd64        NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD
ii  libnvidia-gl-525:i386                          525.89.02-0ubuntu0.22.04.1              i386         NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD
ii  linux-modules-nvidia-525-5.19.0-38-generic     5.19.0-38.39~22.04.1                    amd64        Linux kernel nvidia modules for version 5.19.0-38
ii  linux-modules-nvidia-525-generic-hwe-22.04     5.19.0-38.39~22.04.1                    amd64        Extra drivers for nvidia-525 for the generic-hwe-22.04 flavour
ii  linux-objects-nvidia-525-5.19.0-38-generic     5.19.0-38.39~22.04.1                    amd64        Linux kernel nvidia modules for version 5.19.0-38 (objects)
ii  nvidia-compute-utils-525                       525.89.02-0ubuntu0.22.04.1              amd64        NVIDIA compute utilities
ii  nvidia-driver-525                              525.89.02-0ubuntu0.22.04.1              amd64        NVIDIA driver metapackage
ii  nvidia-kernel-common-525                       525.89.02-0ubuntu0.22.04.1              amd64        Shared files used with the kernel module
ii  nvidia-kernel-source-525                       525.89.02-0ubuntu0.22.04.1              amd64        NVIDIA kernel source package
ii  nvidia-utils-525                               525.89.02-0ubuntu0.22.04.1              amd64        NVIDIA driver support binaries
ii  xserver-xorg-video-nvidia-525                  525.89.02-0ubuntu0.22.04.1              amd64        NVIDIA binary Xorg driver

I tried an “apt reinstall nvidia-driver-525” but it didn’t help.

Titusville, FL

While a reinstall via “apt reinstall nvidia-driver-525” did not work.

This, however, fixed it:

root@computer:/home/user# apt-get remove --purge '^nvidia-.*'
root@computer:/home/user# apt install nvidia-utils-525
root@computer:/home/user# apt install nvidia-driver-525

I then rebooted and it started working again:

root@computer:/home/user# nvidia-smi
Tue Mar 28 19:56:34 2023       
| NVIDIA-SMI 525.89.02    Driver Version: 525.89.02    CUDA Version: 12.0     |
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|   0  Quadro P1000        Off  | 00000000:01:00.0  On |                  N/A |
| N/A   59C    P0    N/A /  N/A |    153MiB /  4096MiB |      6%      Default |
|                               |                      |                  N/A |
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|    0   N/A  N/A      5942      G   /usr/lib/xorg/Xorg                151MiB |
1 Like

Hi, did you also successfully install NVCC? I successfully installed nvidia-driver-525.116.04, but that is not compatible with any CUDA version. I think your driver version is also different from the required version by cuda?