TensorRT 5.1 Cross Compile for Jetson AGX Xavier

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
#1

Hello,

Is “Tensor RT Cross Compile for Linux” for TensorRT 5.1 avialable for Jetson Xavier?

With the latest (0.9.14) SDK Manager I can only download “TensorRT on Target” for version 5.1 in Jetpack 4.2.1.
It would be nice to have cross compile support, like in the Drive AGX Developer Kit, where “Tensor RT Cross Compile for Linux” package is available in the SDK Manager among the host components (but only for version 5.0)

#2

Hi,

Do you select the “Host Machine” option in the step 01.

You can find the host component in the step 02 once you select the option.
The cross-compiled tool is included in the CUDA toolkit section.

Thanks.

#3

Hello,
Thanks for the response.

When I choose “DRIVE” Product category and “Host Machine” configuration in step 01, there is a “Host Components -> AI -> TensorRT Cross Compile for LINUX” version 5.0 package in step 02:
https://imgur.com/2xnOi8q

However, when I choose “Jetson” Product category and Host Machine configuration in step 01, there is no “Host Components -> AI -> TensorRT Cross Compile for LINUX” package included in the list in step 02:
https://imgur.com/4BmYd2v

Under “Host Components -> CUDA” there is no TensorRT related package, so I my guess is that it is not included in this case. Am I overlooking something?

Regards,
Gábor

#4

Hi,

May I know the Ubuntu version of your host?
Please noticed that you will need Ubuntu16.04 or above to install the packages.

Thanks.

#5

Hello,

I use Ubuntu 16.04, but I also tried it on Ubuntu 18.04 with the same result: there was no TensorRT cross compile package under step 02 for Jetson.

#6

Hi,

Sorry for the missing.

For the Jetson system, the cross-compiling toolkit is intergraded in the CUDA toolkit.
So the “CUDA cross compile package” should be enough.

Thanks.

#7

Can you please elaborate on that statement?

After installing all selectable host packages / toolchains using SDK Manager for the Jetson TX2 (but skipping the flashing process of the TX2), I still don’t see TensorRT specific tools like graphsurgeon-tf anywhere on my system.

Moreover, I should expect to see static libs like libnvinfer_static.a somewhere to link against but doing either a find / -iname "nvinfer or dpkg -l | grep nvinfer shows up nothing.

#8

Hi,

Sorry for my unclear statement.

The cross-compiler is integrated into the CUDA toolkit.
So you can cross-compile a Jetson application once the “CUDA cross compile package” installed.

However, we don’t install the required libraries automatically.
This requires you to copy the files on your own.

Since all the Jetson based package is released by the Jetpack installer, you will need to manually copy the required library back to the host.
Here is example for your reference: https://devblogs.nvidia.com/cuda-jetson-nvidia-nsight-eclipse-edition/

$ sudo scp ubuntu@your.ip.address:/usr/lib/aarch64-linux-gnu/libglut.so.3  /usr/aarch64-linux-gnu/lib
$ sudo ln -s /usr/aarch64-linux-gnu/lib/libglut.so.3 /usr/aarch64-linux-gnu/lib/libglut.so
...

Thanks.

#9

Thanks for the reply, but I think it’s still rather vague - exactly which src/header/library files do we need for both TensorRT and DeepStream? Moreover, these could change from version to version and it’s more reassuring if the cross compilation env could be a bit less brittle.

At the moment, I’ve downloaded and installed the x86 TensorRT and DeepStream packages manually just so I have the required tools like graphsurgeon and uff-convert to run on my host, and I’m serving the entire /usr folder from the TX2 with NFS just so I can compile TensorRT apps for TX2. Quite adhoc and hardly an optimal process.

Is it possible for you guys to make a proper release of an aarch64 cross-compile .deb for TensorRT and DeepStream (and for other relevant NVIDIA products) on the host? Otherwise, cross-compilation seems to be a bit of an after-thought.

#10

Can I get official thoughts on the above? It’s some feedback for NVIDIA to improve the toolchain ecosystem.

Cross-compilation is not only cleaner from an architectural POV, but pretty much mandatory when you’re working with huge projects.

#11

Hi,

There are some alternative. Would you mind to check if these are suitable for you?

1. You can use the remote compiling option in the Nsight.
This will compile the application directly on the target and you don’t need to handle the libraries issue.

2. Use the docker in NGC.
https://ngc.nvidia.com/catalog/containers/nvidia:deepstream-l4t

This allow you to implement the target application without tegra access.
But you may not able to use Nsight due to the docker environment.

Thanks.

#12

Thanks for the reply.

  1. Can you clarify your comment about NSight? I’m not sure how NSight is relevant to cross compiling TensorRT and DeepStream applications for Xavier / TX2.

  2. You mean running the deepstream-l4t container on the Xavier / TX2 target and compiling on target instead of host? Not quite sure what you mean here as well.

Just so we’re both on the same page, this thread was about cross-compiling on host for Xavier / TX2 targets. You have the cross compile headers and binaries for gcc and cuda toolchain packaged in SDKManager, but no aarch64 libs for TensorRT and DeepStream to actually build apps on host for target.

#13

If anyone is actually following up, even the CUDA cross-compile toolkit that the SDK Manager installs is incomplete.

There is also no aarch64 cross-compile libs for CUDNN and CUBLAS anywhere after installing the host cross compile toolchain using SDK Manager. But the worst thing is that NVIDIA does not seem to host those aarch64-linux-gnu packages in any of the public URLs that I can find, so I can’t even manually download and install these…

#14

Hi,

Do you meet any problem when installing CUDA toolkit for host with JetPack installer.
If yes, please let us know.

As mentioned above, to cross compiling a deepstream app on the host requires you to prepare all the package back to the host.
We don’t have a pre-collected package for Jetson user currently.
This request is updated to our internal team and will be prioritized internally.

More, to compile an application directly on the target with Nsight.
You can choose the “remote” option when pressing the compiling button.
Nsight will automatically sync up your project to the target and compile it directly the Jetson.

Thanks.

#15

I also have the requirement of the tool package for cross compiling and debugging for jetson nano development by using X86-64 platform. When Nvidia plans to release the tools?

#16

Hi cuter490703,

Please open a new topic at Jetson Nano forum: https://devtalk.nvidia.com/default/board/371/jetson-nano/