How to specify which CUDA to use

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
1

Dear Community,

I have a Jetson AGX Orin with Jetpack 5.1, and therefore came with it CUDA 11.4 and cuDNN 8.6

In order to have a working onnxruntime-gpu, I needed to update it to CUDA 11.8
I first mistakenly installed the CUDA 12.2 version, and then was able to install CUDA 11.8.

Now, although I set this variable in my .zshrc:
export LD_LIBRARY_PATH=/usr/lib/llvm-8/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/cuda-11/lib64
and
export PATH=$PATH:/usr/local/cuda-11/bin

I still see the following when running the famous jetsonInfo.py script:
$ python3 jetsonInfo.py
NVIDIA Jetson AGX Orin
L4T 35.2.1 [ JetPack 5.1 ]
Ubuntu 20.04.6 LTS
Kernel Version: 5.10.104-tegra
CUDA 12.2.140
CUDA Architecture: 8.7
OpenCV version: 4.5.4
OpenCV Cuda: YES
CUDNN: 8.6.0.166
TensorRT: 8.5.2.2
Vision Works: NOT_INSTALLED
VPI: 2.2.4
Vulcan: 1.3.204

Could someone help me set my jetson so that it uses CUDA 11.8 please ?
Without it, onnxruntime_gpu still uses CPU as device instead of the GPU.

Thank you very much for your help in advance

Cheers

3

Hi,

Please check the settings of $LD_LIBRARY_PATH and $PATH.
If there are some links pointing to CUDA 12, please remove them and try it again.

Thanks.

4

Hello AastaLLL,

My $PATH and $LD_LIBRARY_PATH have no pointers to either CUDA 12 or even just CUDA without version specified.

$ env
LC_TERMINAL=iTerm2
LC_CTYPE=UTF-8
LC_TERMINAL_VERSION=3.4.19
USER=jetson
LOGNAME=jetson
HOME=/home/jetson
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/cuda-11/bin:/home/jetson/.local/bin
SHELL=/usr/bin/zsh
TERM=xterm-256color
DISPLAY=:0.0
XDG_SESSION_ID=3
XDG_RUNTIME_DIR=/run/user/1000
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
XDG_SESSION_TYPE=tty
XDG_SESSION_CLASS=user
MOTD_SHOWN=pam
LANG=en_US.UTF-8
SSH_CLIENT=192.168.1.29 65079 22
SSH_CONNECTION=192.168.1.29 65079 192.168.1.189 22
SSH_TTY=/dev/pts/0
SHLVL=1
PWD=/home/jetson
OLDPWD=/home/jetson/Downloads
ZSH=/home/jetson/.oh-my-zsh
PAGER=less
LESS=-R
LSCOLORS=Gxfxcxdxbxegedabagacad
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.zst=01;31:.tzst=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.wim=01;31:.swm=01;31:.dwm=01;31:.esd=01;31:.jpg=01;35:.jpeg=01;35:.mjpg=01;35:.mjpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.m4a=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.oga=00;36:.opus=00;36:.spx=00;36:*.xspf=00;36:
LD_LIBRARY_PATH=/usr/lib/llvm-8/lib:/usr/local/cuda-11/lib64
LD_PRELOAD=/lib/aarch64-linux-gnu/libgomp.so.1
VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
WORKON_HOME=/home/jetson/.virtualenvs
VIRTUALENVWRAPPER_VIRTUALENV=/home/jetson/.local/bin/virtualenv
VIRTUALENVWRAPPER_PROJECT_FILENAME=.project
VIRTUALENVWRAPPER_WORKON_CD=1
VIRTUALENVWRAPPER_SCRIPT=/home/jetson/.local/bin/virtualenvwrapper.sh
VIRTUALENVWRAPPER_HOOK_DIR=/home/jetson/.virtualenvs
_=/usr/bin/env

(/usr/loca/cuda-11 points to /usr/local/cuda-11.8)

However, the link /usr/local/cuda was pointing on /etc/alternatives/cuda, itself pointing back at /usr/local/cuda-12.2

Even though there are no cuda without versions specified in my PATH and LD_LIBRARY_PATH, changing the link /etc/alternatives/cuda back to /usr/local/cuda-11.8 seems to have fixed my problem.

I don’t know where else there could be something pointing to /usr/local/cuda, but it appears to be somewhere.
Anyway, my issue is currently fixed.

Thank you very much for your help.
Cheers

6

Hi,

Good to know it works now.
Thanks a lot for the feedback!

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.