I was wondering if there were any .deb files or ways to install TensorRT 8.4.2 onto my Jetson TX2 device. Or if it is even possible. I am preferably looking for a .deb file because that’s what I used to get the latest version of TensorRT onto Google Colab, but tar files would work fine too, it’s just the hardware specifications that are tricky right now.
Environment
TensorRT Version: Currently 8.2 but looking to get it to 8.4.2 GPU Type: Jetson TX2 Nvidia Driver Version: unknown CUDA Version: 10.2 CUDNN Version: 8.2 Operating System + Version: Ubuntu 18.04 Python Version (if applicable): Python 3.6.9 TensorFlow Version (if applicable): NA PyTorch Version (if applicable): NA Baremetal or Container (if container which image + tag): Flash (Jetpack 4.6.2)
The TensorRT libraries themselves aren’t open-source so you couldn’t just build them from source. Further, they have dependencies on specific versions of cuDNN, CUDA, and the GPU drivers in the L4T that come with JetPack. Hence the version of TensorRT is bundled with JetPack.
The GitHub repo (https://github.com/nvidia/tensorrt) contains open-source samples, plugins, and parsers, but not the actual TensorRT runtime libraries themselves:
This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. Included are the sources for TensorRT plugins and parsers (Caffe and ONNX), as well as sample applications demonstrating usage and capabilities of the TensorRT platform. These open source software components are a subset of the TensorRT General Availability (GA) release with some extensions and bug-fixes.
This gave me TensorRT 8.4.2, is there anything like that for arm architectures?
edit: That command and a couple more but thats the thing I need most right now
There aren’t public TensorRT debs for Jetson (aside from what is in the JetPack apt servers, which are the same debs that already come installed with JetPack). I believe there are ones for arm64 SBSA, but those are for ARM servers with discrete GPU cards and not the integrated GPU that Jetson uses.
TensorRT engines aren’t portable across different GPU devices or across different versions of TensorRT. For example on TX2, the TensorRT engines need to be built on another TX2 running the same version of TensorRT.
Sorry, Jetson TX2 will remain on JetPack 4.6.x and won’t be receiving newer versions of TensorRT. For more information, please refer to the Jetson Software Roadmap.
Hmm, because then there is really a lack of understanding. I was told on the github that I needed to go onto a different PC, make the engine there and test it and whatnot then deploy it on the Jetson. Ofc I didn’t have another PC with GPU capabilities so I did it on colab successfully, but still. Really weird. Also as for the Jetson tx2 tensorrt versions thanks for letting me know
Why don’t you verify with detectron2 sample on your PC and check what evaluation and inference results are following Detectron2 readme? Inference in the sample is purely for checking purposes not real world inference. This python inference will significantly slow you down.
The main reason you don’t get proper results is your TRT version is old 8.2.1. In detectron2 readme is says that TRT must be >= 8.4.1, I also mentioned it to you in your initial issue.
Getting back to proper inference. Here is what you should do: convert detectron2 model to TensorRT on your PC, evaluate on your PC (using detectron2 code and TRT 8.4.1) and check if you are getting 40.2 mAP, you can also infer a couple images after that if you want. After that you should figure out how to install TRT 8.4.1 on Jetson device replacing the default one in JetPack. This was originally mentioned. After you have done that you should look into DeepStream SDK and build efficient inference pipeline (using converted on your PC ONNX and built from it TRT engine on a Jetson). Please don’t ask me how to use DeepStream, I have not done that yet, so I simply do not know. With DeepStream you will get best inference perf possible which is critical if you want to run on Jetson. In addition, I recommend using DeepStream C++ SDK to get best perf, you will loose some perf due to python interpreter if using python SDK.
from azhurkevich in issue -Inference accuracy · Issue #2233 · NVIDIA/TensorRT · GitHub
Unfortunately I don’t think the other person understood that it’s not officially supported for individuals to independently upgrade the version of TensorRT in JetPack, outside of upgrading JetPack itself.
Unofficially, I haven’t tried it and doubt that it would work installing the ARM64 SBSA TensorRT debs that you can download from the TensorRT page, because they probably aren’t built with the Jetson integrated GPU architectures in mind and may have dependencies on newer versions of other libraries (like CUDA and the GPU drivers) that aren’t available in JetPack 4.x.
Alright - I tried some things that I found online on something NVIDIA but my browser doesnt save history (RIP) but it worked. I basically had to upgrade the libnvinfer plugin or smt (i totally forget what it was i think that) and it worked for creating the engine and what not. It also worked for inference (not that it was super accurate but mby a 35 MAP (detectron2). Thanks anyways for the help.