CentOS 7 cannot install Nvidia driver on new kernel

Graphics / Linux Linux
, , ,
1

Hi,

I’m having an issue installing driver 595 (for some new Blackwell GPUs) on CentOS 7. I’m aware CentOS 7 is post EOL, and that has caused quite a few issues, but I don’t have the time to redo a production setup on a new OS, so I’m trying to make this work.

Basically, I had to install a newer kernel for the driver for the Blackwell GPUs, because they need a kernel 4.15 or higher. Basic CentOS 7 is 3.10. I installed a kernel from Elrepo that I got from a back channel (since their server for CentOS 7 was taken offline). It is version 6.1.166-1.el7.x86_64 compiled with devtoolset-10 (gcc version 10.2.1 20210130 (Red Hat 10.2.1-11) (GCC)).

$ uname -r
6.1.166-1.el7.x86_64

$ cat /boot/config-$(uname -r) | grep gcc
CONFIG_CC_VERSION_TEXT=“gcc (GCC) 10.2.1 20210130 (Red Hat 10.2.1-11)”

When I try to install the driver 595.58.03, I get this error

ERROR: An error occurred while performing the step: “Checking to see whether the nvidia kernel module was successfully built”. See /var/log/nvidia-installer.log for details.
→ The command cd kernel; /opt/rh/devtoolset-10/root/usr/bin/make -k -j32 NV_EXCLUDE_KERNEL_MODULES="" SYSSRC="/lib/modules/6.1.166-1.el7.x86_64/source" SYSOUT="/lib/modules/6.1.166-1.el7.x86_64/build" NV_KERNEL_MODULES="nvidia" failed with the following output:

make[1]: Entering directory ‘/usr/src/kernels/6.1.166-1.el7.x86_64’
/bin/sh: -c: line 0: syntax error near unexpected token (’
/bin/sh: -c: line 0: if [ “gcc (GCC) 10.2.1 20210130 (Red Hat 10.2.1-11)” != ““gcc (GCC) 10.2.1 20210130 (Red Hat 10.2.1-11)”” ]; then ’
make[1]: *** [Makefile:1907: prepare] Error 1
make[1]: Target ‘modules’ not remade because of errors.
make[1]: Leaving directory ‘/usr/src/kernels/6.1.166-1.el7.x86_64’
make: *** [Makefile:138: modules] Error 2
ERROR: The nvidia kernel module was not created.

I’m assuming the issue is that the driver thinks I’m using a different version of gcc than the one I used to compile the kernel. But the strings are the same, with the exception that the right hand side has double “”.

I tried explicitly specifying the CC variable like

sudo CC=“/opt/rh/devtoolset-10/root/usr/bin/gcc” sh ./NVIDIA-Linux-x86_64-595.58.03.run

But I get the same error.

Anyone would have some ideas on how to solve this? Is it possible to override the gcc check? I saw on Google that it’s possible for CUDA, but I didn’t find anything about the driver…

2

For anyone wondering, this was not an issue with the driver, but a bug with the dev kernel I got. I found a solution here:

I managed to install the drivers without issues after that