Installation of TensorRT in VEnv on Jetson Nano B01

I would be grateful for assistance installing TensorRT in to a virtual environment on a Jetson Nano B01.

I’m an amateur home user and have been working with a couple B01s since September 2021.

If I can just be pointed in the right direction, I should be able to work out the details myself.

I’d like to use a yolo3 tiny model so it seems that I need to install onnx as well so need help with that as well.

My starting point is the nvidia installation instructions found at:

If I read correctly, pip can’t be used on the Jetson Nano for installation of TensorRT. I did try it and, as expected, it did not work.

There are a few other methods listed which I have little or no experience with.

Here’s my setup:

CUDA 10.2.300
cuDNN: 8.2.1.32
Cuda ARCH: 5.3

in .bashrc:

export OPENBLAS_CORETYPE=ARMV8

PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/home/james/Public/Projects/myPython36VEnv/bin:/usr/local/cuda-10.2/bin:$PATH"

export CPATH=$CPATH:/usr/local/cuda-10.0/targets/aarch64-linux/include

export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/cuda-10.0/targets/aarch64-linux/lib

export LD_LIBRARY_PATH="/usr/local/cuda-10.2/lib64:$LD_LIBRARY_PATH"

Also, I have entered these commands:

$ sudo sh -c “echo ‘/usr/local/cuda/lib64’ >> /etc/ld.so.conf.d/nvidia-tegra.conf”

$ sudo ldconfig

My virtual environment includes:
Python 3.7
numpy version 1.19.5
wheel version 0.37.1
setuptools version 49.6.0
and more that I have not listed…

I’ve also successfully installed pycuda:

pip3 install -U pycuda

Successfully installed appdirs-1.4.4 mako-1.2.0 platformdirs-2.5.2 pycuda-2021.1 pytools-2022.1.9

Any direction would be greatly appreciated.

James

TensorRT is already installed.
Search for the “tensorrt” file/directory in your system.
Unless there is a specific reason that cannot be avoided, I recommend that you use the standard 3.6.9 version of python as is.

sudo apt-get update
sudo apt-get install locate
sudo updatedb
locate tensorrt

Python virtualenv options include --system-site-packages.
This allows you to use TensorRT with virtualenv.

python3 -m virtualenv -p python3 /PATH_TO_VENV --system-site-packages

You will see that “tensorrt” is under /PATH_TO_VENV.

sudo updatedb
locate tensorrt

However, if you are familiar with how apt/pip/python virtualenv manages packages, you understand that using --system-site-packages will cause conflicts between apt packages and pip packages.

If you intentionally remove --system-site-packages to avoid conflicts between apt and pip packages, you can use tensorrt inside virtualenv by creating a symbolic link.

python3 -m virtualenv -p python3 /PATH_TO_VENV

sudo ln -s /usr/lib/python3.6/dist-packages/cv2 /PATH_TO_VENV/lib/python3.6/site-packages/cv2
sudo ln -s /usr/lib/python3.6/dist-packages/graphsurgeon /PATH_TO_VENV/lib/python3.6/site-packages/graphsurgeon
sudo ln -s /usr/lib/python3.6/dist-packages/tensorrt /PATH_TO_VENV/lib/python3.6/site-packages/tensorrt
sudo ln -s /usr/lib/python3.6/dist-packages/uff /PATH_TO_VENV/lib/python3.6/site-packages/uff

Thank you for your prompt reply. Is this the only possibility?

I’m trying to avoid opening up the virtual environment to site-packages.

Also I couldn’t get pycuda to install in a virtual environment with Python 3.6. but was OK with Python 3.7 using pip3.

Also, I believe that Python 3.6 is near its end of life so I’d prefer to use 3.7.

Any thoughts on all of this?

I know of two problems encountered with pycuda installations.
The same problem can be installed as follows

# numpy/core/src/multiarray/numpyos.c:18:10: fatal error: xlocale.h: No such file or directory
sudo ln -s /usr/include/locale.h /usr/include/xlocale.h

# pycuda: src/cpp/cuda.hpp:14:10: fatal error: cuda.h: No such file or directory
# pycuda: /usr/bin/ld: cannot find "-lcurand"
export CPATH=/usr/local/cuda-10.2/targets/aarch64-linux/include
export LDFLAGS="-L/usr/local/cuda-10.2/targets/aarch64-linux/lib"

pip install -U pycuda

It is natural to see the termination of python 3.6 as a problem and to want to try to update.
However, when a single critical package fails to build, it is not an easy problem to solve.
It can take many days and result in having to rewrite the package.
I would not recommend that.

Thanks for the clarification. I’ll go with your suggestion. Thanks for your help.

Could you provide more information on the command?:
python3 -m virtualenv -p python3 /PATH_TO_VENV

The reason that I am asking is that I am using VEnv:
python3.6-venv is already the newest version (3.6.9-1~18.04ubuntu1.7)

So, need to to translate the virtualenv command to the equivalent VEnv.

The specific problem is the meaning of “-p”. I can’t find info on the Internet.

Btw, I created my VEnv with this command:
python3.6 -m venv myPython36VEnv

Is this the equivalent to?:
python3 -m virtualenv -p python3 /PATH_TO_VENV

virtualenv is an extension of venv.
“-p python3” means python3 used with virtualenv.
“-p” option does not exist because you cannot select python2 with venv.

Reference:

I successfully installed pycuda in my VEnv for Python3.6 after using the suggested export commands.

I can now access TensorRT from the same VEnv having used the suggested symbolic links.

I was able to import both in to a jupyter-notebook in the same VEnv.

Great! Thanks!