Tensorrt on Jetson with python 3.9

[appv]

Hi,

im following up on Can TensorRT work on python 3.9 on Jetson AGX Xavier? and try to get tensorrt to run with python 3.9 on nvidia jetson NX.

As far as i understand i need to build TensorRT OSS (GitHub - NVIDIA/TensorRT: TensorRT is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators.) on the jetson in order to run the build script as described in TensorRT/python at main · NVIDIA/TensorRT · GitHub to create python wheel file.
Is that the correct approach to get it running?

The first problem i encounter is actually building TensorRT OSS on the jetson, as i need to specify " TRT_LIB_DIR : Path to the TensorRT installation directory containing libraries." (see GitHub - NVIDIA/TensorRT: TensorRT is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators.), which i cannot find. I used the sd-card method to get my jetson running and can only find the preinstalled tensorrt under /usr/src/tensorrt which does not seem to incluce libraries. Am i missing something here?

[AastaLLL]

Hi,

Please install the TensorRT C++ library first and build a python binding for python3.9 from OSS GitHub.
The TensorRT C++ library can be found in SDK manager or installed via the following command:

$ sudo apt update
$ sudo apt install nvidia-tensorrt

After that, the header and library can be found in the below folder:

$ ll /usr/include/aarch64-linux-gnu/NvInfer*
-rw-r--r-- 1 root root 268726 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInfer.h
-rw-r--r-- 1 root root  42497 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferImpl.h
-rw-r--r-- 1 root root   6176 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferLegacyDims.h
-rw-r--r-- 1 root root  11327 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferPlugin.h
-rw-r--r-- 1 root root  11819 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferPluginUtils.h
-rw-r--r-- 1 root root  60163 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferRuntimeCommon.h
-rw-r--r-- 1 root root  94791 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferRuntime.h
-rw-r--r-- 1 root root   3148 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferVersion.h

$ ll /usr/lib/aarch64-linux-gnu/libnvinfer*
lrwxrwxrwx 1 root root        26 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so -> libnvinfer_plugin.so.8.0.1
lrwxrwxrwx 1 root root        26 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.8 -> libnvinfer_plugin.so.8.0.1
-rw-r--r-- 1 root root  18280576 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.8.0.1
-rw-r--r-- 1 root root  21018654 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin_static.a
lrwxrwxrwx 1 root root        19 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer.so -> libnvinfer.so.8.0.1
lrwxrwxrwx 1 root root        19 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer.so.8 -> libnvinfer.so.8.0.1
-rw-r--r-- 1 root root 188463744 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer.so.8.0.1
-rw-r--r-- 1 root root 333591900 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer_static.a

Thanks.

[appv]

Was able to build TensorRT OSS this way.

However there is a problem gernerating or using the python bindings. Following everything under TensorRT/python at main · NVIDIA/TensorRT · GitHub i’m able to create “tensorrt-8.0.1.6-cp39-none-linux_aarch64.whl” and installing it in a venv with python3 -m pip install build/dist/tensorrt-8.0.1.6-cp39-none-linux_aarch64.whl succeeds.
But import tensorrt fails with the following error.

import tensorrt
Traceback (most recent call last):
File “”, line 1, in
File “path/to/my/venv/lib/python3.9/site-packages/tensorrt/init.py”, line 36, in
from .tensorrt import *
ModuleNotFoundError: No module named ‘tensorrt.tensorrt’

the .whl file is only 3.8kb small and running the ./build.sh that generates the .whl file in TensorRT/python also has some suspicious logs:

– Generating done
– Build files have been written to:/path/to/TensorRT/python
make: *** No targets specified and no makefile found. Stop.
Generating python 3.9 bindings for TensorRT 8.0.1.6

Any suggestions?

[AastaLLL]

Hi,

Thanks for updating the status with us.

We are going to check this internally.
Will share more information with you later.

[AastaLLL]

Hi,

We have confirmed that bindings for python3.9 can be compiled and installed on JetPack4.6.
Please check the following for the detailed instructions:

1. Building python3.9

$ sudo apt install zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev libsqlite3-dev libbz2-dev
$ wget https://www.python.org/ftp/python/3.9.1/Python-3.9.1.tar.xz
$ tar xvf Python-3.9.1.tar.xz Python-3.9.1/

$ mkdir build-python-3.9.1
$ cd build-python-3.9.1/
$ ../Python-3.9.1/configure --enable-optimizations
$ make -j $(nproc)
$ sudo -H make altinstall
$ cd ../

2. Build cmake 3.13.5

$ sudo apt-get install -y protobuf-compiler libprotobuf-dev openssl libssl-dev libcurl4-openssl-dev
$ wget https://github.com/Kitware/CMake/releases/download/v3.13.5/cmake-3.13.5.tar.gz
$ tar xvf cmake-3.13.5.tar.gz
$ rm cmake-3.13.5.tar.gz

$ cd cmake-3.13.5/
$ ./bootstrap --system-curl
$ make -j$(nproc)

$ echo 'export PATH='${PWD}'/bin/:$PATH' >> ~/.bashrc
$ source ~/.bashrc
$ cd ../

3. Prepare header

$ mkdir python3.9
$ mkdir python3.9/include
$ wget http://ftp.us.debian.org/debian/pool/main/p/python3.9/libpython3.9-dev_3.9.9-2_arm64.deb
$ ar x libpython3.9-dev_3.9.9-2_arm64.deb
$ tar -xvf data.tar.xz
$ cp ./usr/include/aarch64-linux-gnu/python3.9/pyconfig.h python3.9/include/
$ cp -r Python-3.9.1/Include/* python3.9/include/

4. Build TensorRT pybinding

$ git clone https://github.com/pybind/pybind11.git
$ git clone -b release/8.0 https://github.com/NVIDIA/TensorRT.git
$ cd TensorRT
$ git submodule update --init --recursive

Please replay {TensorRT}/python/build.sh with this file: build.sh (2.5 KB)

$ cd python/
$ TRT_OSSPATH=${PWD}/.. EXT_PATH=${PWD}/../.. ./build.sh
$ python3.9 -m pip install build/dist/tensorrt-8.0.1.6-cp39-none-linux_aarch64.whl

Reference:
https://arcanesciencelab.wordpress.com/2021/02/14/building-python-3-9-1-on-jetpack-4-5-and-the-jetson-xavier-nx/
https://github.com/NVIDIA/TensorRT/tree/main/python

Thanks.

[appv]

It works. I dont know what went wrong before but following your guide with a fresh start solved it.
Thank you very much for putting the instructions together like that. That’s really awesome support.

Just as a note for others (and probably future me):

For the ./build.sh call (last step) i needed to additionally set the environment variables TARGET=aarch64 and PYTHON_MINOR_VERSION=9 in order to get the desired version of the .whl.

Also in 3. Prepare header the “$” in

tar -xvf data.tar.xz$ cp ./usr/include/aarch64-linux-gnu/python3.9/pyconfig.h python3.9/include/

should be a “&&” i guess?

[AastaLLL]

Hi,

Thanks for the feedback.

It should be a newline but connected to the previous line accidentally.
We have updated the commands and also attach the script for completeness.

Thanks.