(rhel9)NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running

Hello, I’m trying to install a Geforce RTX 5090 into my rhel workstation. I used the nvidia-driver-assistant tool to help with installing the drivers. However, as the title states, I cannot get nvidia-smi to work with the GPU.

Secure boot is disabled.

Here are the nvidia drivers that are installed:

rpm -qa | grep -i nvidia
pcp-pmda-nvidia-gpu-6.3.7-1.el9_6.x86_64
libnvidia-ml-575.57.08-1.el9.x86_64
libnvidia-cfg-575.57.08-1.el9.x86_64
libnvidia-gpucomp-575.57.08-1.el9.x86_64
libnvidia-ml-570.172.08-1.el9.x86_64
libnvidia-cfg-570.172.08-1.el9.x86_64
nvidia-driver-assistant-0.21.57.08-1.noarch
nvidia-driver-cuda-libs-575.57.08-1.el9.x86_64
nvidia-driver-libs-575.57.08-1.el9.x86_64
nvidia-libXNVCtrl-575.57.08-1.el9.x86_64
nvidia-persistenced-575.57.08-1.el9.x86_64
nvidia-modprobe-575.57.08-1.el9.x86_64
nvidia-kmod-common-575.57.08-1.el9.noarch
kmod-nvidia-open-dkms-575.57.08-1.el9.noarch
nvidia-driver-575.57.08-1.el9.x86_64
xorg-x11-nvidia-575.57.08-1.el9.x86_64
nvidia-xconfig-575.57.08-1.el9.x86_64
nvidia-settings-575.57.08-1.el9.x86_64
nvidia-driver-cuda-575.57.08-1.el9.x86_64
nvidia-libXNVCtrl-devel-575.57.08-1.el9.x86_64
libnvidia-fbc-575.57.08-1.el9.x86_64

Any ideas what could be wrong? Or what I need to do differently? Any advice is appreciated.

I’ve attached the bug report:
nvidia-bug-report.log (1.2 MB)

–Please forgive my ignorance on this topic. I am relatively new to RHEL and especially new to installing a nvidia gpu on rhel.