Nvidia graphics drivers not installing on a Redhat 9.4 research workstation

Graphics / Linux Linux
1

Nvidia graphics drivers not installing on a Redhat 9.4 research workstation for an A2000 12GB graphics card.

We ran the following commands:
subscription-manager repos --enable=rhel-9-for-x86_64-appstream-rpms
subscription-manager repos --enable=rhel-9-for-x86_64-baseos-rpms
sudo dnf module install nvidia-driver:560-dkms

The driver appeared to install successfully then we rebooted.

We received the following error after rebooting:
#root> nvidia-smi
NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

Then we removed the drivers:

dnf module remove --all nvidia-driver
dnf module reset nvidia-driver

Reinstalled the graphics drivers to an older version or stream and rebooted:
sudo dnf module install nvidia-driver:515-dkms

And we are still receiving the following error:
#root> nvidia-smi
NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

#root>

2

there’s lack of information here, is the driver loaded, is it being used?
You need to run nvidia-bug-report.sh and attatch the log file here.

3

nvidia-bug-report.log.gz (76.3 KB)

4

The driver is installed. See the install commands we ran from my first text. Is there something I’m not doing that needs to do to enable it? Nvidia bug report is attached.

I followed instructions to install the driver in Nvidia’s installation guide below (section 6):

5

To me, it looks like the driver fails to compile. It also says it isn’t loaded.

6

What needs to be done to fix that?

7

It appears the development tools weren’t installed. sudo yum groupinstall “Development Tools”
Now I’m receiving the following error:

modprobe nvidia

modprobe: ERROR: could not insert ‘nvidia’: Key was rejected by service

I believe this is related to secure boot. Does Nvidia A2000 graphics card drivers support secure boot. Do you have a recommended processes for manually signing your modules?

8

Having secure boot enabled needs some additonal setup probably.
I don’t know how redhat does it but it’s possible you need to set up and sign keys for the nvidia modules.
If you can, I suggest you leave secure boot disabled.

9

Is there any way to get an official answer from Nvidia on if they support secure boot? We don’t want to cause major maintenance issues where we’re needing to constantly sign modules, reboot and fix key chains inside the secure boot database everytime we patch or upgrade the drivers. On the other hand, not using secure boot generates significant security risks because malicious drivers can be loaded by a legitimate user that could be used to break into the system. I don’t understand why Nvidia doesn’t sign their modules for secure boot by default, as they do for their Microsoft windows graphics drivers. What’s Nvidia’s recommendation on using secure boot?

10

Nvidia provides precompiled, signed driver modules for the RHEL 9 standard 5.14 kernel. You just enabled the wrong, dkms modules stream. The nvidia key to import is also in the cuda repo. Please se this on how to do it:
https://www.dell.com/support/kbdoc/en-uk/000225087/how-to-install-nvidia-driver-online-with-secure-boot-enabled
It’s for RHEL8 but it works likewise for RHEL 9.