Why do two commands show my cuda version not consistent?

Graphics / Linux Linux
1 
# nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:01_CDT_2018
Cuda compilation tools, release 10.0, V10.0.130

Does this mean the version is 10.0?

However, nvidia-smi shows it is 10.1:

# nvidia-smi
Sat Nov 28 16:53:05 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.87.01    Driver Version: 418.87.01    CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla P100-PCIE...  On   | 00000000:00:0D.0 Off |                    0 |
| N/A   34C    P0    25W / 250W |      0MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla P100-PCIE...  On   | 00000000:00:0E.0 Off |                    0 |
| N/A   67C    P0   175W / 250W |  15601MiB / 16280MiB |     96%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Tesla P100-PCIE...  On   | 00000000:00:0F.0 Off |                    0 |
| N/A   30C    P0    27W / 250W |      0MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  Tesla P100-PCIE...  On   | 00000000:00:10.0 Off |                    0 |
| N/A   32C    P0    27W / 250W |      0MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

Then, I did this:
# pip install cupy-cuda100

And test it:
    In [1]: import cupy
       ...: a = cupy.zeros((5, 5))
    ---------------------------------------------------------------------------
    ImportError                               Traceback (most recent call last)
    ~/.venv/wbnlu/lib/python3.7/site-packages/cupy/__init__.py in <module>
         19             message='can\'t resolve package from __spec__')
    ---> 20         from cupy import core  # NOQA
         21 except ImportError as e:

    ~/.venv/wbnlu/lib/python3.7/site-packages/cupy/core/__init__.py in <module>
    ----> 1 from cupy.core import core  # NOQA
          2 from cupy.core import internal  # NOQA
          3 

    ImportError: libnvrtc.so.10.0: cannot open shared object file: No such file or directory

    The above exception was the direct cause of the following exception:

    ImportError                               Traceback (most recent call last)
    <ipython-input-1-13f62b8eee5e> in <module>
    ----> 1 import cupy
          2 a = cupy.zeros((5, 5))

    ~/.venv/wbnlu/lib/python3.7/site-packages/cupy/__init__.py in <module>
         39 original error: {}'''.format(_exc_info[1]))  # NOQA
         40 
    ---> 41     raise ImportError(_msg) from e
         42 
         43 

    ImportError: CuPy is not correctly installed.

    If you are using wheel distribution (cupy-cudaXX), make sure that the version of CuPy you installed matches with the version of CUDA on your host.
    Also, confirm that only one CuPy package is installed:
      $ pip freeze

    If you are building CuPy from source, please check your environment, uninstall CuPy and reinstall it with:
      $ pip install cupy --no-cache-dir -vvvv

    Check the Installation Guide for details:
      https://docs.cupy.dev/en/latest/install.html

So what’t the problem of installing cupy here?

2

A similar topic: Libnvrtc.so.10.2 not found - #9 by AastaLLL

Please try reinstalling all your NVIDIA drivers.