Nvcc is not installed in jetson orin nano although nvidia-toolkit is installed

Autonomous Machines Jetson & Embedded Systems Jetson Nano
, ,
1

Hi! I am new to jetson platforms.

I want to infer a VLM on a Jetson Orin Nano.

I got this for nvcc --version.

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Sun_Oct_23_22:16:07_PDT_2022
Cuda compilation tools, release 11.4, V11.4.315
Build cuda_11.4.r11.4/compiler.31964100_0

today morning, I started uninstalling and installing packages to solve the issue that I had back then.

However now I get this error saying that cuda is not installed when running the inference python script. It is saying that nvcc is not installed. What’s the reason for this?

(moon_env) sasika@sasika-desktop:~/moondream/notebooks$ free -h
              total        used        free      shared  buff/cache   available
Mem:          7,2Gi       934Mi       6,1Gi       3,0Mi       149Mi       6,1Gi
Swap:         3,6Gi       812Mi       2,8Gi

(moon_env) sasika@sasika-desktop:~/moondream/notebooks$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p1   79G   51G   24G  68% /
none            3,5G     0  3,5G   0% /dev
tmpfs           3,6G   36K  3,6G   1% /dev/shm
tmpfs           734M   19M  715M   3% /run
tmpfs           5,0M  4,0K  5,0M   1% /run/lock
tmpfs           3,6G     0  3,6G   0% /sys/fs/cgroup
tmpfs           734M   20K  734M   1% /run/user/124
tmpfs           734M   48K  734M   1% /run/user/1000 

(moon_env) sasika@sasika-desktop:~/moondream/notebooks$ nvcc --version
bash: nvcc: command not found
(moon_env) sasika@sasika-desktop:~/moondream/notebooks$ python 
Python 3.9.21 (main, Dec  4 2024, 08:53:34) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch; torch.cuda.is_available()
False
>>> 
(moon_env) sasika@sasika-desktop:~/moondream/notebooks$ head -n 1 /etc/nv_tegra_release
# R35 (release), REVISION: 3.1, GCID: 32827747, BOARD: t186ref, EABI: aarch64, DATE: Sun Mar 19 15:19:21 UTC 2023
(moon_env) sasika@sasika-desktop:~/moondream/notebooks$ python inference.py 
/home/sasika/moondream/notebooks

Killed
(moon_env) sasika@sasika-desktop:~/moondream/notebooks$ python train.py 
/home/sasika/moondream/moon_env/lib/python3.9/site-packages/bitsandbytes/cextension.py:34: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
  warn("The installed version of bitsandbytes was compiled without GPU support. "
'NoneType' object has no attribute 'cadam32bit_grad_fp32'
Traceback (most recent call last):
  File "/home/sasika/moondream/notebooks/train.py", line 217, in <module>
    main()
  File "/home/sasika/moondream/notebooks/train.py", line 211, in main
    trainer = MoondreamTrainer()
  File "/home/sasika/moondream/notebooks/train.py", line 54, in __init__
    self._setup_environment()
  File "/home/sasika/moondream/notebooks/train.py", line 64, in _setup_environment
    torch.cuda.reset_peak_memory_stats()
  File "/home/sasika/moondream/moon_env/lib/python3.9/site-packages/torch/cuda/memory.py", line 344, in reset_peak_memory_stats
    return torch._C._cuda_resetPeakMemoryStats(device)
AttributeError: module 'torch._C' has no attribute '_cuda_resetPeakMemoryStats'
(moon_env) sasika@sasika-desktop:~/moondream/notebooks$ pip show torch torchvision

Name: torch
Version: 2.5.1
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: packages@pytorch.org
License: BSD-3-Clause
Location: /home/sasika/moondream/moon_env/lib/python3.9/site-packages
Requires: filelock, fsspec, jinja2, networkx, sympy, typing-extensions
Required-by: accelerate, torchvision
---
Name: torchvision
Version: 0.20.1
Summary: image and video datasets and models for torch deep learning
Home-page: https://github.com/pytorch/vision
Author: PyTorch Core Team
Author-email: soumith@pytorch.org
License: BSD
Location: /home/sasika/moondream/moon_env/lib/python3.9/site-packages
Requires: numpy, pillow, torch
Required-by:

I think the above output will be helpful for understanding the configuration I am using.

What i don’t understand is whether this error due to jetson or torch library. Deeply grateful if you can assist me identifying the issue here.

Thank you in advance! (Please consider the fact that I am new to jetson devices and working with libraries involved with jetson)

3

Hi,

It looks like you upgraded the Python version into 3.9.
For JetPack 5, we provide prebuilt for the default Python 3.8 environment.
So the PyTorch in your environment may not compiled with GPU support.

To solve this issue, the easier approach is to reset your device with our latest JetPack 6.1 with SDK manager:

And you can find the PyTorch prebuilt (with GPU support) in the link below:
https://pypi.jetson-ai-lab.dev/jp6/cu126

Thanks.

4

I am also experiencing this same problem as exemplified here:

Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.zeros(1).cuda()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.local/lib/python3.10/site-packages/torch/cuda/__init__.py", line 310, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

I’ve seen suggestions to check the output of nvcc. Ok, I’ll try that:

bash: nvcc: command not found

And I have already “flashed” my device to the latest with SDKManager (the latest as of 10 days ago).

Can anyone help?

5

Is /usr/local/cuda-12.6/bin in the $PATH?

6

It is.

7

Can you check what the PyTorch version you are using
pip3 show torch

I installed a PyTorch version compatible with my jetpack version, python version, and CUDA version. and it worked.

The CUDA version that you can use is constrained by the Jetpack version you are using.

I used this torch version "torch @ https://developer.download.nvidia.com/compu te/redist/jp/v512/pytorch/torch-2.1.0a0+41361538.nv23.06-cp38-cp38-linux_aarch64.whl#sha256=5112e2ef5051f1003ae2ffb545ae596377df66979d62a954d774f18009064dbe
"
and this is my configuration

CUDA_VERSION  = 11.4 
NVIDIA-JETPACK = 5.1.1-b56
OS = Ubuntu 20.04 focal
Python Virtual Environment version = Python 3.8.10
GCC Compiler Version : [GCC 9.4.0]

I followed this tutorial. https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html

Just a heads-up!If you install torchvision (pip install torchvision) it will uninstall the correct torchversion and install a PyTorch version which is not compatible with CUDA. So I had to install Torchvision library by building it from source.