NVIDIA Driver Compatibility Issue with RHEL 9.7 (Kernel 5.14.0-611.16.1.el9_7) – Tesla P100

I am running Red Hat Enterprise Linux 9.7 with kernel 5.14.0-611.16.1.el9_7 on a system equipped with NVIDIA Tesla P100 (GP100) GPUs. When attempting to install the NVIDIA driver using the official CUDA repository, the available packages only provide kmod-nvidia builds for kernel 5.14.0-611.5.1.el9_7, not for the currently shipped 5.14.0-611.16.1.el9_7 kernel. As a result, the NVIDIA driver cannot be loaded and nvidia-smi fails, even though the system and kernel are otherwise functioning correctly. The driver works when booting into the older kernel, but there appears to be no compatible NVIDIA driver package for the newer RHEL 9.7 kernel. Could you please confirm whether support for kernel 5.14.0-611.16.1.el9_7 is planned for Tesla P100, or advise on the recommended approach going forward?

I’m in a similar situation. Were you able to find a fix or did you just have to use the older kernel?

Also having an issue regarding kernel 5.14.0-611.11.1.el9_7.x86_64 and nvidia drivers…. Asked support if there was any timeline for 9.7 support and haven’t got a response yet….

Same here….issue regarding upgrade from rhel9.6 to rhel9.7. Once we did the 9.7 update, we got the following error “NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDA driver is installed."

Interestingly, we were able to get nvidia-smi and drivers to respond appropriately by doing the following:

dnf install -y
kernel-core-5.14.0-570.58.1.el9_6
kernel-modules-5.14.0-570.58.1.el9_6
kernel-devel-5.14.0-570.58.1.el9_6

dkms autoinstall
dkms status
nvidia-smi
nvidia-smi nvlink -s

This re-installed just the 9.6 kernel, and made that the number 1 priority when you reboot. The fact that this seemed to get nvidia-smi to start working again leads me to believe that this is potentially a rhel9.7 compatibility issue.