nvdia-smi show CUDA Version, but nvcc not found

Accelerated Computing CUDA CUDA Setup and Installation
1

I have Ubuntu Server 16.04, VGA: Quadro P4000
nvdia-smi can show CUDA Version, but nvcc -V not found anything.
So is it CUDA installed or not ?

Detail here:

+-----------------------------------------------------------------------------+
root@ubuntu:/home/ubuntu# nvidia-smi
Tue Nov 13 15:51:15 2018       
| NVIDIA-SMI 410.66       Driver Version: 410.66       CUDA Version: 10.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Quadro P4000        Off  | 00000000:02:00.0 Off |                  N/A |
| 46%   36C    P0    28W / 105W |     10MiB /  8119MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Quadro P4000        Off  | 00000000:06:00.0 Off |                  N/A |
| 47%   43C    P0    29W / 105W |     10MiB /  8119MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Quadro P4000        Off  | 00000000:82:00.0 Off |                  N/A |
| 46%   36C    P0    27W / 105W |     10MiB /  8119MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                             
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |

root@ubuntu:/home/ubuntu# which nvcc
root@ubuntu:/home/ubuntu# find / -name "nvcc"
root@ubuntu:/home/ubuntu# nvcc --version
The program 'nvcc' is currently not installed. You can install it by typing:
apt install nvidia-cuda-toolkit
root@ubuntu:/home/ubuntu# cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module  410.66  Wed Oct 10 12:01:53 CDT 2018
GCC version:  gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)
root@ubuntu:/home/ubuntu#
+-----------------------------------------------------------------------------+
2

It’s not installed correctly. If you used a NVIDIA installer to install CUDA, then you may have missed the necessary steps in section 7 of the linux install guide:

[url]Installation Guide Linux :: CUDA Toolkit Documentation

2 Likes
3

Thanks you for your quick reply.

Because I’m not the man who installed driver & cuda toolkit on this server (someone else did) so I want to verify what he did.

As I can find nvidia-cuda-mps (detail below), I guess that he just install NVIDIA Driver, not CUDA Toolkit. Is it right ?
My main purpose is running ffmpeg transcoding using HW-Accelerating. Do I need to reinstall CUDA Toolkit ?

root@ubuntu:/opt# ls -l /usr/bin/ | grep "nvidia*"
lrwxrwxrwx 1 root   root          52 Oct 26 13:46 nvidia-bug-report.sh -> /etc/alternatives/x86_64-linux-gnu_nvidia_bug_report
lrwxrwxrwx 1 root   root          58 Oct 26 13:46 nvidia-cuda-mps-control -> /etc/alternatives/x86_64-linux-gnu_nvidia-cuda-mps-control
lrwxrwxrwx 1 root   root          57 Oct 26 13:46 nvidia-cuda-mps-server -> /etc/alternatives/x86_64-linux-gnu_nvidia-cuda-mps-server
lrwxrwxrwx 1 root   root          51 Oct 26 13:46 nvidia-debugdump -> /etc/alternatives/x86_64-linux-gnu_nvidia-debugdump
lrwxrwxrwx 1 root   root          54 Oct 26 13:46 nvidia-persistenced -> /etc/alternatives/x86_64-linux-gnu_nvidia_persistenced
-rwxr-xr-x 1 root   root      224736 Oct 17 04:00 nvidia-settings
lrwxrwxrwx 1 root   root          45 Oct 26 13:46 nvidia-smi -> /etc/alternatives/x86_64-linux-gnu_nvidia_smi
lrwxrwxrwx 1 root   root          49 Oct 26 13:46 nvidia-xconfig -> /etc/alternatives/x86_64-linux-gnu_nvidia_xconfig
root@ubuntu:/opt# find / -name "nvcc"
root@ubuntu:/opt#
4

Finally found out that someone installed Nvidia Driver 410.66 & CUDA Driver (or Runtime ???) Version 10 (nvidia-smi shows in the first post) through graphics-drivers-ubuntu-ppa-xenial.
You have to remove that nvidia driver version, disable that ppa before follow the cuda toolkit official guide or you run into version mismatch.

5

It means you miss any installation step or it didn’t add to your PATH.

  • check if it is in your PATH by “whereis nvcc”, if it returns “nvcc:” then you need to add below two lines in “.bashrc”
  • usually “.bashrc” file path is like “/home/username/.bashrc” then add below two lines (change cuda version with installed version)
    export PATH=“/usr/local/cuda-11.4/bin:$PATH”
    export LD_LIBRARY_PATH=“/usr/local/cuda-11.4/lib64:$LD_LIBRARY_PATH”
    then save and close the file
  • check “nvcc --version”
3 Likes
6

Thank you for your solution!

7

Great solution, thanks so much!