Tesla K40 drivers fail to install on Ubuntu 16.04.1 LTS

Hi,

I am trying to set up a Tesla K40c on Ubuntu (4.4.0-34-generic), but the driver installer NVIDIA-Linux-x86_64-352.99.run recommended by the nvidia website for this configuration dies as follows. Running ./NVIDIA-Linux-x86_64-352.99.run with sudo produces the log file:

nvidia-installer log file ‘/var/log/nvidia-installer.log’
creation time: Tue Aug 9 11:45:44 2016
installer version: 352.99

PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

nvidia-installer command line:
./nvidia-installer

Unable to load: nvidia-installer ncurses v6 user interface

Using: nvidia-installer ncurses user interface
→ Detected 8 CPUs online; setting concurrency level to 8.
WARNING: The NVIDIA Quadro FX 1700 GPU installed in this system is supported through the NVIDIA 340.xx legacy Linux graphics drivers. Please visit Unix Drivers | NVIDIA for more information. The 352.99 NVIDIA Linux graphics driver will ignore this GPU.
→ License accepted.
→ Installing NVIDIA driver version 352.99.
→ Performing CC sanity check with CC=“gcc-5.3”.
ERROR: Unable to determine the version of the kernel sources located in ‘/lib/modules/4.4.0-34-generic/build’. Please make sure you have installed the kernel source files for your kernel and that they are properly configured; on Red Hat Linux systems, for example, be sure you have the ‘kernel-source’ or ‘kernel-devel’ RPM installed. If you know the correct kernel source files are installed, you may specify the kernel source path with the ‘–kernel-source-path’ command line option.
ERROR: Installation has failed. Please see the file ‘/var/log/nvidia-installer.log’ for details. You may find suggestions on fixing installation problems in the README available on the Linux driver download page at www.nvidia.com.

The kernel header and source files are installed (using apt-get) and the kernel header file exists as:
/lib/modules/4.4.0-34-generic/build/include/linux/kernel.h

Running the installer again with:
./NVIDIA-Linux-x86_64-352.99.run --kernel-source-path=/lib/modules/4.4.0-34-generic/build/include

produces log file:

nvidia-installer log file ‘/var/log/nvidia-installer.log’
creation time: Tue Aug 9 11:19:50 2016
installer version: 352.99

PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

nvidia-installer command line:
./nvidia-installer
–kernel-source-path=/lib/modules/4.4.0-34-generic/build/include

Unable to load: nvidia-installer ncurses v6 user interface

Using: nvidia-installer ncurses user interface
→ Detected 8 CPUs online; setting concurrency level to 8.
WARNING: The NVIDIA Quadro FX 1700 GPU installed in this system is supported through the NVIDIA 340.xx legacy Linux graphics drivers. Please visit Unix Drivers | NVIDIA for more information. The 352.99 NVIDIA Linux graphics driver will ignore this GPU.
→ License accepted.
→ Installing NVIDIA driver version 352.99.
→ Performing CC sanity check with CC=“gcc-5.3”.
→ Using the kernel source path ‘/lib/modules/4.4.0-34-generic/build/include’ as specified by the ‘–kernel-source-path’ commandline option.
ERROR: The kernel header file ‘/lib/modules/4.4.0-34-generic/build/include/include/linux/kernel.h’ does not exist. The most likely reason for this is that the kernel source path ‘/lib/modules/4.4.0-34-generic/build/include’ is incorrect. Please make sure you have installed the kernel source files for your kernel and that they are properly configured; on Red Hat Linux systems, for example, be sure you have the ‘kernel-source’ or ‘kernel-devel’ RPM installed. If you know the correct kernel source files are installed, you may specify the kernel source path with the ‘–kernel-source-path’ command line option.
ERROR: Installation has failed. Please see the file ‘/var/log/nvidia-installer.log’ for details. You may find suggestions on fixing installation problems in the README available on the Linux driver download page at www.nvidia.com.

The driver installer seems to be having trouble with the kernel source files’ path. Even when explicitly running:
./NVIDIA-Linux-x86_64-352.99.run --kernel-source-path=/lib/modules/4.4.0-34-generic/build

for which the installer should, based on the second log output, search for and find /lib/modules/4.4.0-34-generic/build/include/linux/kernel.h, the installer dies with the following (same terminating error as the initial run):

nvidia-installer log file ‘/var/log/nvidia-installer.log’
creation time: Tue Aug 9 11:18:08 2016
installer version: 352.99

PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

nvidia-installer command line:
./nvidia-installer
–kernel-source-path=/lib/modules/4.4.0-34-generic/build

Unable to load: nvidia-installer ncurses v6 user interface

Using: nvidia-installer ncurses user interface
→ Detected 8 CPUs online; setting concurrency level to 8.
WARNING: The NVIDIA Quadro FX 1700 GPU installed in this system is supported through the NVIDIA 340.xx legacy Linux graphics drivers. Please visit Unix Drivers | NVIDIA for more information. The 352.99 NVIDIA Linux graphics driver will ignore this GPU.
→ License accepted.
→ Installing NVIDIA driver version 352.99.
→ Performing CC sanity check with CC=“gcc-5.3”.
→ Using the kernel source path ‘/lib/modules/4.4.0-34-generic/build’ as specified by the ‘–kernel-source-path’ commandline option.
ERROR: Unable to determine the version of the kernel sources located in ‘/lib/modules/4.4.0-34-generic/build’. Please make sure you have installed the kernel source files for your kernel and that they are properly configured; on Red Hat Linux systems, for example, be sure you have the ‘kernel-source’ or ‘kernel-devel’ RPM installed. If you know the correct kernel source files are installed, you may specify the kernel source path with the ‘–kernel-source-path’ command line option.
ERROR: Installation has failed. Please see the file ‘/var/log/nvidia-installer.log’ for details. You may find suggestions on fixing installation problems in the README available on the Linux driver download page at www.nvidia.com.

Any help is greatly appreciated.

Thanks

Have you tried explicitly installing the kernel dev packages for your distro?

Also note that Ubuntu 16.04, gcc 5.3, and your kernel version are not officially supported by CUDA 7.5 (not related to your driver issue per se).

Hi,

Yeah, the kernel packages are installed for this particular distro. Earlier, I had been setting CC to gcc-5.3 to pass the gcc check on the installer (gcc-5.4 is installed but the kernel was compiled with 5.3). However, install finished using --no-cc-version-check.