CUDA driver version is insufficient for CUDA runtime version

I am trying to install the CUDA Toolkit 9.0 on my Ubuntu 16.04 system using package manager installation but I have a number of problems when trying to verify the installation (under “7.2 Recommended Actions” in the Installation Guide https://docs.nvidia.com/cuda/cuda-installation-guide-linux).

In particular, the NVIDIA Persistence Daemon does not seem to exist (section 7.2.1):

jonathan@Phoebe:~$ /usr/bin/nvidia-persistenced --verbose
bash: /usr/bin/nvidia-persistenced: No such file or directory

I can’t verify the driver version (section 7.2.3.1):

jonathan@Phoebe:~$ cat /proc/driver/nvidia/version
cat: /proc/driver/nvidia/version: No such file or directory

And running deviceQuery fails with the following error message:

jonathan@Phoebe:~/NVIDIA_CUDA-9.0_Samples/bin/x86_64/linux/release$ ./deviceQuery
./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 35
-> CUDA driver version is insufficient for CUDA runtime version
Result = FAIL

Here’s the version of CUDA:

jonathan@Phoebe:~$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Sep__1_21:08:03_CDT_2017
Cuda compilation tools, release 9.0, V9.0.176

Here’s information about the graphics card:

jonathan@Phoebe:~$ lspci | grep -i nvidia
01:00.0 3D controller: NVIDIA Corporation GM108M [GeForce 940MX] (rev a2)

jonathan@Phoebe:~$ lspci | grep -i vga
00:02.0 VGA compatible controller: Intel Corporation Device 5917 (rev 07)

Here’s the output of nvidia-smi:

jonathan@Phoebe:~$ nvidia-smi
nvidia-smi: command not found

Here’s the output of lshw:

jonathan@Phoebe:~$ sudo lshw -c video
  *-display               
       description: VGA compatible controller
       product: Intel Corporation
       vendor: Intel Corporation
       physical id: 2
       bus info: pci@0000:00:02.0
       version: 07
       width: 64 bits
       clock: 33MHz
       capabilities: pciexpress msi pm vga_controller bus_master cap_list rom
       configuration: driver=i915 latency=0
       resources: irq:280 memory:d4000000-d4ffffff memory:b0000000-bfffffff ioport:f000(size=64) memory:c0000-dffff
  *-display UNCLAIMED
       description: 3D controller
       product: GM108M [GeForce 940MX]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:01:00.0
       version: a2
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress bus_master cap_list
       configuration: latency=0
       resources: memory:d2000000-d2ffffff memory:c0000000-cfffffff memory:d0000000-d1ffffff ioport:e000(size=128) memory:d3000000-d307ffff

Here’s what NVIDIA packages I have got installed:

jonathan@Phoebe:~$ dpkg -l | grep nvidia
ii  nvidia-387                                 387.26-0ubuntu1                              amd64        NVIDIA binary driver - version 387.26
ii  nvidia-387-dev                             387.26-0ubuntu1                              amd64        NVIDIA binary Xorg driver development files
ii  nvidia-modprobe                            387.26-0ubuntu1                              amd64        Load the NVIDIA kernel driver and create device files
ii  nvidia-opencl-icd-387                      387.26-0ubuntu1                              amd64        NVIDIA OpenCL ICD
ii  nvidia-prime                               0.8.2                                        amd64        Tools to enable NVIDIA's Prime
ii  nvidia-settings                            387.26-0ubuntu1                              amd64        Tool for configuring the NVIDIA graphics driver

And here’s what happens when I try to run nvidia-settings:

jonathan@Phoebe:~$ nvidia-settings

ERROR: Error querying enabled displays on GPU 0 (Missing Extension).

ERROR: Error querying connected displays on GPU 0 (Missing Extension).

** Message: PRIME: No offloading required. Abort
** Message: PRIME: is it supported? no

ERROR: nvidia-settings could not find the registry key file. This file should have been installed along with this driver at
       /usr/share/nvidia/nvidia-application-profiles-key-documentation. The application profiles will continue to work, but values cannot be prepopulated or
       validated, and will not be listed in the help text. Please see the README for possible values and descriptions.

So it looks to be something wrong with the NVIDIA driver. Any idea what I should do to fix it?

The driver is not properly installed. Perhaps you installed the driver from a Ubuntu repo instead of from a NVIDIA repo.