The NVIDIA L4T TensorRT containers only come with runtime variants. We compile TensorRT plugins in those containers and are currently unable to do so because include headers are missing.
Is there a plan to support a l4t-tensorrt version which not only ships the runtime but the full install? Similar to the non tegra tensorrt base image? Bonus: having the same versioning (e.g. 22.04) as the ngc tensorrt container for servers would be a very good addition as well.
We need this to get started with porting our stack to Jetson Orin and Jetpack 5.0. The l4t-base image no longer mounts the host TensorRT install, but the l4t-tensorrt does not have the necessary headers, so this is kinda messed up now…
From recent post maybe @AastaLLL can you help out here? It would be very much appreciated.
@NVES issue moved to Jetson AGX Orin and tagged with TensorRT.
The two provided links are not relevant for the discussion.
The key issue here is how TensorRT is no longer mounted from the docker host (jetpack install) in the most recent versions of the L4T base container images.
While this is a very welcomed change (makes everything a little more portable) we are lacking a l4t-tensorrt image with a full install so we can run certain build commands for TensorRT plugins in our multi-stage docker build.
@NVES if you know who could help here feel free to ping the colleagues. Thanks.
Tried installing libnvinfer-dev inside the container but it points to internal repository http://cuda-internal.nvidia.com/release-candidates/kitpicks/tensorrt-rel-8-4-tegra/8.4.0/001/repos/l4t/arm64/InRelease which does not help.
Thanks @AastaLLL
We can use the provided workaround for a multi-stage docker build with l4t-base for building the engine and l4t-tensorrt runtime image for deployment. For now we are set.
A develop l4t-tensorrt version still would be useful to have, happy to hear news on that internal ticket.
Got some feedback from our internal team.
We have an ML container that has all the dev components installed.
Is it enough for you or a devel tensorrt container is better?
Hello @AastaLLL
Which container is this? l4t-ml I suppose?
According to the description of that container it wasn’t clear to me that it ships with TensorRT.
We can use l4t-ml as the build stage and then ship with l4t-tensorrt:runtime, so we are good.
Would probably be helpful for others to have a reference to l4t-ml in the NGC documentation though.
Thanks for the fast responses the last few days. Very much appreciated.
Yes, and starting on R34.1 and newer, the l4t-ml container is based on that new jetpack dockerfile I pointed you to, so both of them have the dev packages for CUDA/cuDNN/TensorRT inside (and the same goes for l4t-pytorch and l4t-tensorflow)