TRTorch on jetson xavier agx

How do I install TRTorch on jetson xavier agx with a fresh install of jetpack 4.5?
It says it supports jetpack 4.4 but need Cuda 11.0 and TensorRT v 7.2.16 wich is not part of that jetpack.

I’m trying to follow the instructions on how to install TRTorch(GitHub - pytorch/TensorRT: PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT) for jetson but seems to be failing massivly

First I need to update to cuda 11.0 by following this guide:

and are downloading the file cuda-repo-ubuntu1804-11-0-local_11.0.2-450.51.05-1_arm64.deb

When doing the check with “dpkg -l | grep cuda” it then seems like both cuda 10.2 and 11.0 is listed.
I guessed that cuda 10.2 should be removed or can they be used simultaniously?

Anyway, when trying to continue by installing bazel

export BAZEL_VERSION=3.7.0
curl -fSsL -O https://github.com/bazelbuild/bazel/releases/download/$BAZEL_VERSION/bazel-$BAZEL_VERSION-dist.zip

but it needs curl which is not installed.
When trying to install curl “sudo apt-install curl -y” I get the message:
“Unmet dependencies. Try ‘apt --fix-broken install’ with no packages (or specify a solution).”
When trying to fix the problem I get :
“Sub-process /usr/bin/dpkg returned an error code (1)”

When trying to install Bazel it needs cdnn version 8 and tensorrt 7.2.16
Where can I download these packages for agx?

this lists only TensorRT 7.2.1 and is only available for cuda 11.1 for the ARM architecture.

I cant find the cudnn library either (Log in | NVIDIA Developer)
The one I find is for aarch64sbsa and for ubuntu 20.04

summary:
errors when trying to upgrade cuda
not possible to find compatible cudnn and TensorRT versions.

Hi,

Thank you to report this.

You will need to use the CUDA package from JetPack for the Jetson system.
Based on the tutorial, it is tested on the nvidia:l4t-pytorch container.

Let us check it further and share the detailed procedure later.
Thanks.

Hi,

Please try the following to compile Trtorch with JetPack 4.5.

1. Pull docker

$ sudo docker run -it --rm --runtime nvidia --network host -v /home/nvidia/workspace:/home/nvidia/workspace nvcr.io/nvidia/l4t-pytorch:r32.5.0-pth1.7-py3

2. Build bazel

$ apt-get update
$ apt-get install -y pkg-config zip g++ zlib1g-dev unzip openjdk-8-jdk
$ wget https://github.com/bazelbuild/bazel/releases/download/4.0.0/bazel-4.0.0-dist.zip
$ unzip bazel-4.0.0-dist.zip -d bazel-4.0.0-dist
$ cd bazel-4.0.0-dist
$ EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" ./compile.sh
$ cp output/bazel /usr/bin/

3. Clone TRTorch source

$ cd /home/nvidia/workspace
$ git clone https://github.com/NVIDIA/TRTorch.git

4. Apply changes

Update WORKSPACE with WORKSPACE (5.1 KB)
Apply following change

diff --git a/py/setup.py b/py/setup.py
index 1e6ca6e..218c39d 100644
--- a/py/setup.py
+++ b/py/setup.py
@@ -204,7 +204,7 @@ setup(name='trtorch',
       long_description=long_description,
       ext_modules=ext_modules,
       install_requires=[
-          'torch==1.7.1',
+          'torch==1.7.0',
       ],
       setup_requires=[],
       cmdclass={

5. Compile and install

$ cd TRTorch/py
$ python3 setup.py bdist_wheel --use-cxx11-abi
$ pip3 install dist/trtorch-0.2.0a0-cp36-cp36m-linux_aarch64.whl 

Thanks.

1 Like

Here is a wheel file built on Xavier with JetPack4.5. trtorch-0.2.0a0-cp36-cp36m-linux_aarch64.whl (5.6 MB)
For the same software version and GPU architecutre(sm_72), you can also install it with pip3 directly.

$ sudo docker run -it --rm --runtime nvidia --network host -v /home/nvidia/workspace:/home/nvidia/workspace nvcr.io/nvidia/l4t-pytorch:r32.5.0-pth1.7-py3
$ pip3 install dist/trtorch-0.2.0a0-cp36-cp36m-linux_aarch64.whl 
1 Like

@AastaLLL
Hello, bazel build //:libtrtorch (compilation of C++ library) give me much of erros.
Is there a built file on Xavier with this same software ?

Hi ghassenrebai.enetcom.us,

Please help to open a new topic if it’s still an issue. Thanks