TensorRT Version Mismatch Between DeepStream L4T 7.1 Container and Jetson Orin Nano

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
1

• Hardware Platform: Jetson orin nano
• DeepStream Version: 7.1
• JetPack Version 6.2.1
• TensorRT Version: 10.3.0.30
• NVIDIA GPU Driver Version: 540.4.0

Hi

I’m running a DeepStream application inside the official deepstream-l4t:7.1-triton-multiarch container on a Jetson
Orin device. The issue is that the TensorRT version inside the container (10.3.0.26) differs from the host system
(10.3.0.30).

This causes my YOLOv8 models to regenerate .engine files every time the container starts, because TensorRT engine files are version-specific and not compatible between 10.3.0.26 and 10.3.0.30.

Docker Image: nvcr.io/nvidia/deepstream-l4t:7.1-triton-multiarch

And also i am getting this when i run container:

=============================
== Triton Inference Server ==
=============================

NVIDIA Release 24.08 (build 107631426)
Triton Server Version 2.49.0

Copyright (c) 2018-2024, NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

ERROR: This container was built for NVIDIA Driver Release 560.28 or later, but
version 540.4.0 was detected and compatibility mode is UNAVAILABLE.

   [[]]

/bin/bash: line 1: lsmod: command not found
/bin/bash: line 1: modprobe: command not found

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:16.835: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstmpeg2enc.so': libmjpegutils-2.1.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:16.845: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstvoaacenc.so': libvo-aacenc.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:16.916: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstfluidsynthmidi.so': libFLAC.so.8: cannot open shared object file: No such file or directory

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:16.971: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstlibav.so': libmp3lame.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:17.031: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstlame.so': libmp3lame.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:17.047: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstsndfile.so': libFLAC.so.8: cannot open shared object file: No such file or directory

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:17.048: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstopenh264.so': libopenh264.so.6: cannot open shared object file: No such file or directory

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:17.127: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstchromaprint.so': libmp3lame.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:17.146: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstmplex.so': libmjpegutils-2.1.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:17.157: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstpulseaudio.so': libFLAC.so.8: cannot open shared object file: No such file or directory

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:17.163: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstfaad.so': libfaad.so.2: cannot open shared object file: No such file or directory

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:17.167: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstflac.so': libFLAC.so.8: cannot open shared object file: No such file or directory

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:17.504: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:137): GStreamer-WARNING **: 03:10:17.523: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory

Questions

  1. Is there an updated deepstream-l4t:7.1 image with TensorRT 10.3.0.30?
  2. Can I safely upgrade TensorRT inside the container to match the host version, or will this break DeepStream compatibility?
  3. How can i fix driver version mismatch and plugin load issue
3

There is no a deepstream-l4t:7.1 image with TensorRT 10.3.0.30. The host and container TRT are fully compatible; versions 26 and 30 only have some bug fixes.

This is a known issue and can be ignored for now; it does not affect the container’s operation.

Please map the engine file created in the host to the container when create a container, and ensure that the creation parameters such as batch_size, network-mode, and gpu_id are consistent with nvinfer configuration file.

-v /path/to/host/engine:/path/to/container
4

Thank you for the response. I’ve mapped the host-generated engine files into the container as you suggested, but i am still getting same issue.

this log is from container

ERROR: [TRT]: IRuntime::deserializeCudaEngine: Error Code 6: API Usage Error (The engine plan file is not compatible with this version of TensorRT, expecting library version 10.3.0.26 got 
..)
ERROR: Deserialize engine failed from file: /workspace/models/ds/detector.pt_fp16.engine
0:00:21.883955206     1 0xffff00605350 WARN                 nvinfer gstnvinfer.cpp:681:gst_nvinfer_logger:<pgie_hoop_ball> NvDsInferContext[UID 2]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:2080> [UID = 2]: deserialize engine from file :/workspace/models/ds/detector.pt_fp16.engine failed
0:00:21.884011432     1 0xffff00605350 WARN                 nvinfer gstnvinfer.cpp:681:gst_nvinfer_logger:<pgie_hoop_ball> NvDsInferContext[UID 2]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2185> [UID = 2]: deserialize backend context from engine from file :/workspace/models/ds/detector.pt_fp16.engine failed, try rebuild
0:00:21.884022632     1 0xffff00605350 INFO                 nvinfer gstnvinfer.cpp:684:gst_nvinfer_logger:<pgie_hoop_ball> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2106> [UID = 2]: Trying to create engine from model files
NvMapMemAllocInternalTagged: 1075072515 error 12
NvMapMemHandleAlloc: error 0
NvMapMemAllocInternalTagged: 1075072515 error 12
NvMapMemHandleAlloc: error 0

The engine files exist and are accessible

and this is yml file

services:
  deepstream-app:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: deepstream
    runtime: nvidia
    network_mode: host
    volumes:
      - /tmp/.X11-unix:/tmp/.X11-unix
      - ./models:/workspace/models
    env_file:
      - .env
5

The DeepStream 7.1 GA container was published before the JetPack 6.2.1 was published.

The DeepStream docker files are open source: NVIDIA-AI-IOT/deepstream_dockers at dev/ds/ds7.1

You can build the docker with your own customization based on the platform compatibilities. Installation — DeepStream documentation