Tensorrt on Jetson with python 3.9

Hi,

im following up on Can TensorRT work on python 3.9 on Jetson AGX Xavier? and try to get tensorrt to run with python 3.9 on nvidia jetson NX.

As far as i understand i need to build TensorRT OSS (GitHub - NVIDIA/TensorRT: TensorRT is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators.) on the jetson in order to run the build script as described in TensorRT/python at main · NVIDIA/TensorRT · GitHub to create python wheel file.
Is that the correct approach to get it running?

The first problem i encounter is actually building TensorRT OSS on the jetson, as i need to specify " TRT_LIB_DIR : Path to the TensorRT installation directory containing libraries." (see GitHub - NVIDIA/TensorRT: TensorRT is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators.), which i cannot find. I used the sd-card method to get my jetson running and can only find the preinstalled tensorrt under /usr/src/tensorrt which does not seem to incluce libraries. Am i missing something here?

1 Like

Hi,

Please install the TensorRT C++ library first and build a python binding for python3.9 from OSS GitHub.
The TensorRT C++ library can be found in SDK manager or installed via the following command:

$ sudo apt update
$ sudo apt install nvidia-tensorrt

After that, the header and library can be found in the below folder:

$ ll /usr/include/aarch64-linux-gnu/NvInfer*
-rw-r--r-- 1 root root 268726 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInfer.h
-rw-r--r-- 1 root root  42497 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferImpl.h
-rw-r--r-- 1 root root   6176 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferLegacyDims.h
-rw-r--r-- 1 root root  11327 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferPlugin.h
-rw-r--r-- 1 root root  11819 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferPluginUtils.h
-rw-r--r-- 1 root root  60163 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferRuntimeCommon.h
-rw-r--r-- 1 root root  94791 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferRuntime.h
-rw-r--r-- 1 root root   3148 Jun 25 20:17 /usr/include/aarch64-linux-gnu/NvInferVersion.h
$ ll /usr/lib/aarch64-linux-gnu/libnvinfer*
lrwxrwxrwx 1 root root        26 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so -> libnvinfer_plugin.so.8.0.1
lrwxrwxrwx 1 root root        26 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.8 -> libnvinfer_plugin.so.8.0.1
-rw-r--r-- 1 root root  18280576 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.8.0.1
-rw-r--r-- 1 root root  21018654 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin_static.a
lrwxrwxrwx 1 root root        19 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer.so -> libnvinfer.so.8.0.1
lrwxrwxrwx 1 root root        19 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer.so.8 -> libnvinfer.so.8.0.1
-rw-r--r-- 1 root root 188463744 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer.so.8.0.1
-rw-r--r-- 1 root root 333591900 Jun 25 20:17 /usr/lib/aarch64-linux-gnu/libnvinfer_static.a

Thanks.

Was able to build TensorRT OSS this way.

However there is a problem gernerating or using the python bindings. Following everything under TensorRT/python at main · NVIDIA/TensorRT · GitHub i’m able to create “tensorrt-8.0.1.6-cp39-none-linux_aarch64.whl” and installing it in a venv with python3 -m pip install build/dist/tensorrt-8.0.1.6-cp39-none-linux_aarch64.whl succeeds.
But import tensorrt fails with the following error.

import tensorrt
Traceback (most recent call last):
File “”, line 1, in
File “path/to/my/venv/lib/python3.9/site-packages/tensorrt/init.py”, line 36, in
from .tensorrt import *
ModuleNotFoundError: No module named ‘tensorrt.tensorrt’

the .whl file is only 3.8kb small and running the ./build.sh that generates the .whl file in TensorRT/python also has some suspicious logs:

– Generating done
– Build files have been written to:/path/to/TensorRT/python
make: *** No targets specified and no makefile found. Stop.
Generating python 3.9 bindings for TensorRT 8.0.1.6

Any suggestions?

Hi,

Thanks for updating the status with us.

We are going to check this internally.
Will share more information with you later.

Hi,

We have confirmed that bindings for python3.9 can be compiled and installed on JetPack4.6.
Please check the following for the detailed instructions:

1. Building python3.9

$ sudo apt install zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev libsqlite3-dev libbz2-dev
$ wget https://www.python.org/ftp/python/3.9.1/Python-3.9.1.tar.xz
$ tar xvf Python-3.9.1.tar.xz Python-3.9.1/
$ mkdir build-python-3.9.1
$ cd build-python-3.9.1/
$ ../Python-3.9.1/configure --enable-optimizations
$ make -j $(nproc)
$ sudo -H make altinstall
$ cd ../

2. Build cmake 3.13.5

$ sudo apt-get install -y protobuf-compiler libprotobuf-dev openssl libssl-dev libcurl4-openssl-dev
$ wget https://github.com/Kitware/CMake/releases/download/v3.13.5/cmake-3.13.5.tar.gz
$ tar xvf cmake-3.13.5.tar.gz
$ rm cmake-3.13.5.tar.gz
$ cd cmake-3.13.5/
$ ./bootstrap --system-curl
$ make -j$(nproc)
$ echo 'export PATH='${PWD}'/bin/:$PATH' >> ~/.bashrc
$ source ~/.bashrc
$ cd ../

3. Prepare header

$ mkdir python3.9
$ mkdir python3.9/include
$ wget http://ftp.us.debian.org/debian/pool/main/p/python3.9/libpython3.9-dev_3.9.9-2_arm64.deb
$ ar x libpython3.9-dev_3.9.9-2_arm64.deb
$ tar -xvf data.tar.xz
$ cp ./usr/include/aarch64-linux-gnu/python3.9/pyconfig.h python3.9/include/
$ cp -r Python-3.9.1/Include/* python3.9/include/

4. Build TensorRT pybinding

$ git clone https://github.com/pybind/pybind11.git
$ git clone -b release/8.0 https://github.com/NVIDIA/TensorRT.git
$ cd TensorRT
$ git submodule update --init --recursive

Please replay {TensorRT}/python/build.sh with this file: build.sh (2.5 KB)

$ cd python/
$ TRT_OSSPATH=${PWD}/.. EXT_PATH=${PWD}/../.. ./build.sh
$ python3.9 -m pip install build/dist/tensorrt-8.0.1.6-cp39-none-linux_aarch64.whl

Reference:
https://arcanesciencelab.wordpress.com/2021/02/14/building-python-3-9-1-on-jetpack-4-5-and-the-jetson-xavier-nx/
https://github.com/NVIDIA/TensorRT/tree/main/python

Thanks.

2 Likes

It works. I dont know what went wrong before but following your guide with a fresh start solved it.
Thank you very much for putting the instructions together like that. That’s really awesome support.

Just as a note for others (and probably future me):

For the ./build.sh call (last step) i needed to additionally set the environment variables TARGET=aarch64 and PYTHON_MINOR_VERSION=9 in order to get the desired version of the .whl.

Also in 3. Prepare header the “$” in

tar -xvf data.tar.xz$ cp ./usr/include/aarch64-linux-gnu/python3.9/pyconfig.h python3.9/include/

should be a “&&” i guess?

Hi,

Thanks for the feedback.

It should be a newline but connected to the previous line accidentally.
We have updated the commands and also attach the script for completeness.

Thanks.

2 Likes

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.