CUDA driver version is insufficient for CUDA runtime version

I’m running Ubuntu 16.04 x86_64 with nvidia driver version 384.111 from nvidia-384, and a GTX 1060. I just installed cuda-9.1 from the cuda repo(.deb installation).According to the installation guide,for verifying cuda installation,deviceQuery results in “CUDA driver version is insufficient for CUDA runtime version” error.

Results of some commands:

$ lspci -v | grep -i nvidia
01:00.0 VGA compatible controller: NVIDIA Corporation Device 1c03 (rev a1) (prog-if 00 [VGA controller])
Kernel driver in use: nvidia
Kernel modules: nvidiafb, nouveau, nvidia_384_drm, nvidia_384
01:00.1 Audio device: NVIDIA Corporation Device 10f1 (rev a1)

$ nvidia-settings -q NvidiaDriverVersion
Attribute ‘NvidiaDriverVersion’ (sachin:0.0): 384.111
Attribute ‘NvidiaDriverVersion’ (sachin:0[gpu:0]): 384.111

$ uname -r
4.13.0-37-generic

$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 384.111 Tue Dec 19 23:51:45 PST 2017
GCC version: gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)

$ lsmod | grep -i nvidia
nvidia_uvm 671744 0
nvidia_drm 45056 1
nvidia_modeset 860160 4 nvidia_drm
nvidia 13139968 180 nvidia_modeset,nvidia_uvm
drm_kms_helper 167936 1 nvidia_drm
drm 360448 4 nvidia_drm,drm_kms_helper

$ nvcc -V
nvcc: NVIDIA ® Cuda compiler driver
Copyright © 2005-2017 NVIDIA Corporation
Built on Fri_Nov__3_21:07:56_CDT_2017
Cuda compilation tools, release 9.1, V9.1.85

$ nvidia-smi
Sun Apr 1 11:41:51 2018
±----------------------------------------------------------------------------+
| NVIDIA-SMI 384.111 Driver Version: 384.111 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 106… Off | 00000000:01:00.0 On | N/A |
| 0% 53C P0 30W / 200W | 355MiB / 6069MiB | 0% Default |
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1111 G /usr/lib/xorg/Xorg 192MiB |
| 0 1910 G compiz 160MiB |
±----------------------------------------------------------------------------+

~/NVIDIA_CUDA-9.1_Samples/bin/x86_64/linux/release ./deviceQuery
./deviceQuery Starting…

CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 35
-> CUDA driver version is insufficient for CUDA runtime version
Result = FAIL

Earlier, I followed the installation guide for cuda-9.1 religiously but then , I got stuck in a login loop something similar to https://ubuntuforums.org/showthread.php?t=2367755 and https://askubuntu.com/questions/762831/ubuntu-16-stuck-in-login-loop-after-installing-nvidia-364-drivers . I followed the steps and removed cuda and nvidia drivers;reached a stable state using Nouveau display driver.

I tried installing cuda again and I saw that it was using driver-version 387.26. I manually switched to driver-version 384.11 and rebooted,guessing that the driver was the problem earlier.Now, I can see that my display driver is working fine but cuda is not(deviceQuery is failing).

I am sure that I am following the installation guide correctly.

Update: I switched back and forth between nvidia-384 and nvidia-387 and the result is:387.26 doesn’t work on my setup. Should I upgrade my driver to nvidia-390.12?