Installing/ Building Torch-Tensorrt for JetPack 5.0.1 DP/ L4t-ml r34.1.1-py3

[mohapatra.sambit8467]

Hi,

Is there a stable way to build Torch-Tensorrt for Xavier AGX from source?
I am aware of a post here:

TRTorch on jetson xavier agx

Hi, Please try the following to compile Trtorch with JetPack 4.5. 1. Pull docker $ sudo docker run -it --rm --runtime nvidia --network host -v /home/nvidia/workspace:/home/nvidia/workspace nvcr.io/nvidia/l4t-pytorch:r32.5.0-pth1.7-py3 2. Build bazel $ apt-get update $ apt-get install -y pkg-config zip g++ zlib1g-dev unzip openjdk-8-jdk $ wget https://github.com/bazelbuild/bazel/releases/download/4.0.0/bazel-4.0.0-dist.zip $ unzip bazel-4.0.0-dist.zip -d bazel-4.0.0-dist $ cd bazel-4.0.0-dis…

However, I need to build it for JetPack 5.0+ since I want to use Ubuntu 20.04 base.
I tried using the instructions:
https://pytorch.org/TensorRT/

But I get several errors when trying to build.

Further, I used L4t-ML docker image r34.1.1-py3

Please help me out.

Best Regards
Sambit

[AastaLLL]

Hi,

Do you compile it with the instructions below?
If yes, would you mind sharing the error you met with us?
https://pytorch.org/TensorRT/tutorials/installation.html

Thanks.

[mohapatra.sambit8467]

Sure, I will share it shortly.
However, as you can see in the section for native build on jetson, the platform option only shows for jetpack 4.6

This is the first point of confusion, can I just change it to 5.0 and issue the build command?

I also had to change lines in workspace since Jetpack 5.0 has Python3.8

Shown lines are for Python3.6

Best Regards
Sambit

[AastaLLL]

Hi,

It should work by updating the version into JetPack 5.0 DP and python into 3.8.
It will be good if you can share the commands/steps as well.

Thanks.

[mohapatra.sambit8467]

Here is the error message I get when I change to jetpack_5.0

Best Regards

[mohapatra.sambit8467]

Update - I was able to build it.
Change - toolchain/BUILD
platform(
name = “jetpack_5.0”,
constraint_values = [
“@platforms//os:linux”,
“@platforms//cpu:aarch64”,
“@//toolchains/jetpack:5.0”
]
)

Change - toolchain/Jetpack/BUILD
constraint_value(
name = “5.0”,
constraint_setting = “:jetpack”
)

Last question -
After building, do I need to copy the generated files from bazel-bin (include files and .so) to some other system directory?
Also, how can I use it with base Python interpreter?

Best Regards

[AastaLLL]

Hi,

There should be a package like torch-xxx-linux_aarch64.whl.
Then you can copy it to another platform and install it with pip3.

Thanks.