[Solved] Why is my CUDA driver version is insufficient for CUDA runtime version?

So I’ve been trying to compile some of the CUDA examples but nothing was behaving as it should, I put a cudaGetLastError() in front of my code and it turns out that it always returns 35, which I believe means: “CUDA driver version is insufficient for CUDA runtime version”.

Even after a full reinstall of my drivers and cuda packages this has not gone away, could someone tell me what is going here?

System info dump:

$ nvidia-smi
Wed Feb 19 04:42:14 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 430.64       Driver Version: 430.64       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Quadro T1000        Off  | 00000000:01:00.0  On |                  N/A |
| N/A   39C    P8     2W /  N/A |     65MiB /  3911MiB |      4%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      2500      G   /usr/lib/Xorg                                 63MiB |
+-----------------------------------------------------------------------------+
$ uname -a
Linux LinuxIsPower 5.5.2-1-MANJARO #1 SMP PREEMPT Tue Feb 4 19:22:35 UTC 2020 x86_64 GNU/Linux
$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_19:24:38_PDT_2019
Cuda compilation tools, release 10.2, V10.2.89
$ mhwd -li
> Installed PCI configs:
--------------------------------------------------------------------------------
                  NAME               VERSION          FREEDRIVER           TYPE
--------------------------------------------------------------------------------
video-hybrid-intel-nvidia-430xx-bumblebee            2019.10.25               false            PCI

Because CUDA 10.2 requires a 440.33.01 or newer driver, and your driver is 430.xx. Too old.

If you don’t need CUDA 10.2, the simplest solution is probably to install CUDA 10.1 without installing any new driver, and just keep your 430.xx driver. Toolkit runfile and package manager install types both have methods for installing the toolkit without the driver.

Ah, in hindsight it’s so obvious, but I guess that’s what happens after a few hours.

I downgraded to CUDA 10.1 and everything works as expected now. Thanks for the help!