Segmentation Fault in libnvds_nvmultiobjecttracker.so with Re-ID enabled on DeepStream 8.0 + Custom TensorRT 10.13

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Issue Type: Bug / ABI Compatibility Issue

Hardware Platform: RTX 2060 GPU (x86_64)
DeepStream Version: 8.0 (8.0.0)
TensorRT Version: 10.13.2 (Custom build, System-wide installation)
NVIDIA GPU Driver Version: 580.95.05
CUDA Version: 12.8 (System)

Description

I am encountering a Segmentation Fault when enabling Re-ID (reidType: 2) in the NvDCF tracker on DeepStream 8.0. My environment uses a custom installation of TensorRT 10.13.2, whereas DeepStream 8.0 typically expects an older version (e.g., TRT 10.3).

Although I have successfully rebuilt the OSS gst-nvtracker plugin against TensorRT 10.13 headers to handle the wrapper compatibility, the application crashes inside the closed-source low-level library libnvds_nvmultiobjecttracker.so when it attempts to load the Re-ID engine.

Steps to Reproduce

  1. Environment Setup:

    • Install DeepStream 8.0.

    • Running deepstream-app with config tracker block

      [tracker]
ll-config-file=/home/nvidia/deepstream-app/tracker/config_tracker_NvDCF_accuracy.yml
display-tracking-id=1
enable=1
gpu-id=0
ll-lib-file=/opt/nvidia/deepstream/deepstream-8.0/lib/libnvds_nvmultiobjecttracker.so
tracker-height=256
tracker-width=128

      Install custom TensorRT 10.13.2 at system level ([/usr/lib/x86_64-linux-gnu/libnvinfer.so.10.13.2](file:///usr/lib/x86_64-linux-gnu/libnvinfer.so.10.13.2)).

    • Rebuild gst-nvtracker plugin from sources/gst-plugins/gst-nvtracker linking against TRT 10.13. Ive converted the onnx to engine using the trtexec from the Tensorrt 10.13.2 bin directory
      trtexec
      –onnx=resnet50_market1501_aicity156.onnx
      –saveEngine=resnet50_market1501_aicity156.enfine
      –fp16
      –verbose
      –minShapes=input:1x3x256x128
      –optShapes=input:100x3x256x128
      –maxShapes=input:100x3x256x128
      –memPoolSize=workspace:2048M

  2. Configuration:

    • Use [config_tracker_NvDCF_accuracy.yml]

      config_tracker_NvDCF_accuracy.txt (8.7 KB)

      )

    • Set reidType: 2 (Reid based reassoc).

    • Point to a valid Re-ID engine built with trtexec (TRT 10.13).

  3. Execution:

    • Run the DeepStream application.

Observed Behavior

The application crashes with a segmentation fault immediately upon loading the tracker.

GDB Backtrace:

#0  0x00007fff96b66da8 in ??? () at /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#1  0x00007fffd04c9aa1 in ??? () at /usr/lib/x86_64-linux-gnu/libnvinfer.so.10
...
#7  0x00007ffed43412c1 in TrtModule::loadTRTEngine() ()
    at /opt/nvidia/deepstream/deepstream-8.0/lib/libnvds_nvmultiobjecttracker.so
#8  0x00007ffed4343fd0 in TrtModule::initializeModel(...) ()
    at /opt/nvidia/deepstream/deepstream-8.0/lib/libnvds_nvmultiobjecttracker.so
...
#12 0x00007ffed42a2d3a in NvMultiObjectTrackerBase::initialize(NvMotDataTypes::Size, unsigned long) ()
    at /opt/nvidia/deepstream/deepstream-8.0/lib/libnvds_nvmultiobjecttracker.so

LDD Check:
Running ldd on the proprietary library:

ldd /opt/nvidia/deepstream/deepstream-8.0/lib/libnvds_nvmultiobjecttracker.so | grep nvinfer
        libnvinfer.so.10 => /usr/lib/x86_64-linux-gnu/libnvinfer.so.10 (0x000076e1d0600000)  <-- Points to TRT 10.13

Analysis

The crash appears to be a binary incompatibility (ABI mismatch) between libnvds_nvmultiobjecttracker.so and the newer TensorRT 10.13 runtime.

  • Without Re-ID (reidType: 0): The application works fine (NvDCF visual tracking works), likely because the specific TensorRT paths in the library are not exercised.
  • With Re-ID: The library attempts to deserialize the engine using TrtModule::loadTRTEngine. Since libnvds_nvmultiobjecttracker.so is closed-source and was likely compiled against an older TRT version (10.3?), it crashes when interacting with the TRT 10.13 objects/memory layout.

Request

Is there a way to obtain a version of libnvds_nvmultiobjecttracker.so compatible with newer TensorRT versions (10.13+), or is downgrading the system TensorRT the only solution? Respective rebuilding of the OSS plugin wrapper does not seem to solve the issue within the core proprietary library.

3

Seems like I’ve loaded other trt models alongside that conflicted with the trt runtime of the tracker. I’ve isolated the other models and the tracker engine is loading now correctly.