TensorRT on Jetson Nano (Jetpack4.6.1) for Python3.8 Cuda10.2

AI & Data Science Deep Learning (Training & Inference) TensorRT
, , ,
1

I have a Jetson Nano (Jetpack4.6.1) and I want to run Yolov8 for object detection in images.

In order to use Yolo through the ultralytics library, I had to install Python3.8, and then I performed some black magic to finally install pytorch and torchvision. Those parts all work now.

But now, I’m trying to install TensorRT.

Because the architecture is arm64, the deb files I found on this download page don’t work. Doing pip install directly doesn’t seem to work either (it returns an error: “RuntimeError: TensorRT does not currently build wheels for Tegra systems”). I tried to follow those instructions, but the cuda version here is 10.2.
Then I also tried this guide for the python binding, and beyond the fact that the version of Python I have is 3.8 not 3.9 (so I tried to change the commands here to 3.8 instead), I almost seemed to run smoothly, but I end up with an install of tensorrt that only include a init.py, so when I import tensort (in Python), it says “no module named tensorrt.tensorrt” or something like that.

Is there any way to make tensorrt work with Jetson Nano, ultralytics (requiring Python3.8) and Cuda10.2? Or am I approaching the issue from the wrong angle?

5

I get the same error when trying to set up torch-tensorrt following this installation manual: Installation — Torch-TensorRT v2.7.0.dev0+d6be4ba documentation

Is there any possibility to run torch-tensorrt on Jetson [Orin] Nano? I am using CUDA 12.6.

6

The only way I managed to make it work in the end is inside a docker container as recommended in this guide:
https://docs.ultralytics.com/guides/nvidia-jetson/#__tabbed_1_1

7

Ah great, thanks! Did you also try to use torch_tensorrt or did you try mainly tensorrt within the docker container?

8

I do it inside the container. Everything is preinstalled and works right away inside the container, so it’s much easier.

9

For native TensorRT installation, you need to use the SDK Manager to install it. Docker is easier, but it may have performance penalty.