TRTorch on jetson xavier agx

Robotics & Edge Computing Jetson Systems Jetson Xavier NX
,
1

How do I install TRTorch on jetson xavier agx with a fresh install of jetpack 4.5?
It says it supports jetpack 4.4 but need Cuda 11.0 and TensorRT v 7.2.16 wich is not part of that jetpack.

I’m trying to follow the instructions on how to install TRTorch(GitHub - pytorch/TensorRT: PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT · GitHub) for jetson but seems to be failing massivly

First I need to update to cuda 11.0 by following this guide:
https://www.seeedstudio.com/blog/2020/07/29/install-cuda-11-on-jetson-nano-and-xavier-nx/
and are downloading the file cuda-repo-ubuntu1804-11-0-local_11.0.2-450.51.05-1_arm64.deb

When doing the check with “dpkg -l | grep cuda” it then seems like both cuda 10.2 and 11.0 is listed.
I guessed that cuda 10.2 should be removed or can they be used simultaniously?

Anyway, when trying to continue by installing bazel

export BAZEL_VERSION=3.7.0
curl -fSsL -O https://github.com/bazelbuild/bazel/releases/download/$BAZEL_VERSION/bazel-$BAZEL_VERSION-dist.zip

but it needs curl which is not installed.
When trying to install curl “sudo apt-install curl -y” I get the message:
“Unmet dependencies. Try ‘apt --fix-broken install’ with no packages (or specify a solution).”
When trying to fix the problem I get :
“Sub-process /usr/bin/dpkg returned an error code (1)”

When trying to install Bazel it needs cdnn version 8 and tensorrt 7.2.16
Where can I download these packages for agx?

this lists only TensorRT 7.2.1 and is only available for cuda 11.1 for the ARM architecture.

I cant find the cudnn library either (cuDNN 9.20.0 Downloads | NVIDIA Developer)
The one I find is for aarch64sbsa and for ubuntu 20.04

summary:
errors when trying to upgrade cuda
not possible to find compatible cudnn and TensorRT versions.

3

Hi,

Thank you to report this.

You will need to use the CUDA package from JetPack for the Jetson system.
Based on the tutorial, it is tested on the nvidia:l4t-pytorch container.

Let us check it further and share the detailed procedure later.
Thanks.

5

Hi,

Please try the following to compile Trtorch with JetPack 4.5.

1. Pull docker

$ sudo docker run -it --rm --runtime nvidia --network host -v /home/nvidia/workspace:/home/nvidia/workspace nvcr.io/nvidia/l4t-pytorch:r32.5.0-pth1.7-py3

2. Build bazel

$ apt-get update
$ apt-get install -y pkg-config zip g++ zlib1g-dev unzip openjdk-8-jdk
$ wget https://github.com/bazelbuild/bazel/releases/download/4.0.0/bazel-4.0.0-dist.zip
$ unzip bazel-4.0.0-dist.zip -d bazel-4.0.0-dist
$ cd bazel-4.0.0-dist
$ EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" ./compile.sh
$ cp output/bazel /usr/bin/

3. Clone TRTorch source

$ cd /home/nvidia/workspace
$ git clone https://github.com/NVIDIA/TRTorch.git

4. Apply changes

Update WORKSPACE with WORKSPACE (5.1 KB)
Apply following change

diff --git a/py/setup.py b/py/setup.py
index 1e6ca6e..218c39d 100644
--- a/py/setup.py
+++ b/py/setup.py
@@ -204,7 +204,7 @@ setup(name='trtorch',
       long_description=long_description,
       ext_modules=ext_modules,
       install_requires=[
-          'torch==1.7.1',
+          'torch==1.7.0',
       ],
       setup_requires=[],
       cmdclass={

5. Compile and install

$ cd TRTorch/py
$ python3 setup.py bdist_wheel --use-cxx11-abi
$ pip3 install dist/trtorch-0.2.0a0-cp36-cp36m-linux_aarch64.whl

Thanks.

6

Here is a wheel file built on Xavier with JetPack4.5. trtorch-0.2.0a0-cp36-cp36m-linux_aarch64.whl (5.6 MB)
For the same software version and GPU architecutre(sm_72), you can also install it with pip3 directly.

$ sudo docker run -it --rm --runtime nvidia --network host -v /home/nvidia/workspace:/home/nvidia/workspace nvcr.io/nvidia/l4t-pytorch:r32.5.0-pth1.7-py3
$ pip3 install dist/trtorch-0.2.0a0-cp36-cp36m-linux_aarch64.whl
8

@AastaLLL
Hello, bazel build //:libtrtorch (compilation of C++ library) give me much of erros.
Is there a built file on Xavier with this same software ?

9

Hi ghassenrebai.enetcom.us,

Please help to open a new topic if it’s still an issue. Thanks