DeepStream nvtracker: Config Parameters Not Applied & ReID Status Unclear in Headless Docker

[agodin]

DeepStream nvtracker: Config Parameters Not Applied & ReID Status Unclear in Headless Docker

System Configuration

• Platform: Jetson Orin, L4T R36.4 (JetPack 6.x)
• DeepStream: 7.0 (nvcr.io/nvidia/deepstream:7.0-samples-multiarch)
• Docker: 25.0.3 with --runtime=nvidia
• Environment: Headless Docker container

Problem 1: Tracker Config Parameters Not Being Applied

Issue: nvtracker loads the config file successfully, but parameter changes have no effect on tracking output.

Evidence from Logs:

✅ ReID model file found: /app/deepstream_app/models/resnet50_market1501.etlt_b1_gpu0_fp16.engine
📁 Config file exists: True
📄 Config file size: 8767 bytes
🔍 Key parameters in config file:
   minDetectorConfidence: 0.1894
   minTrackerConfidence: 0.1513
   maxTargetsPerStream: 150
   filterLr: 0.0767
   gaussianSigma: 0.5687
✅ Tracker ll-config-file property confirmed: /app/tracker_config.yml

What I’ve Tried:

• Changed minDetectorConfidence: 0.8 → 0.1894 (no effect on results)
• Changed minTrackerConfidence: 0.5 → 0.1513 (no effect)
• Changed maxTargetsPerStream: 150 → 250 (no effect)
• Changed filterLr: 0.075 → 0.0767 (no effect)
• Changed gaussianSigma: 0.75 → 0.5687 (no effect)
• Modified 10+ parameters simultaneously (tracking output identical)
• Verified config file loads and property is set correctly

Result: Config file loads, parameters are displayed in logs, but tracking results are byte-for-byte identical regardless of values.

Problem 2: ReID Status Unclear - ConvBufManager Warning

Logs Show:

nvbufsurftransform: Could not get EGL display connection
[NvMultiObjectTracker] Loading TRT Engine for tracker ReID...
[NvMultiObjectTracker] Loading Complete!  ✅
[NvMultiObjectTracker] Initialized  ✅
⚠️  ConvBufManager may have failed - continuing with limited functionality
[TRACKER] Frame 1: 18 tracked objects
...
📊 Processed 749 frames successfully

How This Warning is Triggered:
The warning message is generated by my tracker output probe when the first frame is processed:

def tracker_output_probe(self, pad, info, u_data):
    """Probe function to capture tracker output."""
    gst_buffer = info.get_buffer()
    if not gst_buffer:
        return Gst.PadProbeReturn.OK
    
    # Skip frames if ConvBufManager failed to initialize
    if self.frame_count == 0:
        print("⚠️  ConvBufManager may have failed - continuing with limited functionality")

However, the actual EGL error (nvbufsurftransform: Could not get EGL display connection) comes from the DeepStream library itself, indicating a real issue with buffer transformations in the headless environment.

What Works:

• ReID TensorRT engine loads successfully
• Tracker initializes without errors
• Pipeline processes all 749 frames and produces output

What’s Unclear:

• Config has reidType: 2 (ReID-based re-association) but I can’t verify if it’s actually being used
• The nvbufsurftransform EGL error suggests buffer transformations (needed for ReID) may be failing
• Question: Is appearance-based ReID active, or am I only getting motion-based tracking?
• No clear indication in logs whether ReID features are working or disabled

Need Help:

• How to verify if ReID is actually functioning despite the EGL/ConvBufManager errors?
• Does the nvbufsurftransform: Could not get EGL display connection error mean ReID feature extraction is silently failing?
• Are there diagnostic logs or metrics to confirm ReID is active?

Use Case

• Multi-object tracking with pre-computed detections
• Need to optimize tracker params for crowded scenes
• Need ReID for tracking across occlusions
• Must run in headless Docker for edge deployment

Pipeline

filesrc → qtdemux → h264parse → nvv4l2decoder → nvstreammux → nvtracker → fakesink
                                                      ↑
                                            (detection injection probe)

Docker Run Command

docker run --rm -it \
    --runtime=nvidia \
    -v /mnt/storage/UbiUniversalTracker/jetson-tracker/deepstream_app:/app/deepstream_app \
    -v /mnt/storage/UbiUniversalTracker/jetson-tracker/MOT17-13-FRCNN_h264.mp4:/app/MOT17-13-FRCNN_h264.mp4:ro \
    -v /opt/pva-libs:/opt/pva-libs:ro \
    -e DISPLAY=:0 \
    -e EGL_PLATFORM=surfaceless \
    -e CUDA_VISIBLE_DEVICES=0 \
    -e LD_LIBRARY_PATH="/opt/pva-libs/aarch64-linux-gnu/tegra:/opt/pva-libs/usr/lib:/opt/nvidia/deepstream/deepstream/lib:/usr/lib/aarch64-linux-gnu" \
    nvcr.io/nvidia/deepstream:7.0-samples-multiarch \
    python3 /app/deepstream_app/detection_tracker_final.py \
        /app/MOT17-13-FRCNN_h264.mp4 \
        /app/deepstream_app/output/det_sorted.txt \
        /app/deepstream_app/output/nvdeepsort_results.txt \
        NvDeepSORT \
        /app/deepstream_app/tracker_config.yml

Tracker Configuration (tracker_config.yml)

Click to expand full configuration

BaseConfig:
  minDetectorConfidence: 0.1894

TargetManagement:
  enableBboxUnClipping: 1
  maxTargetsPerStream: 150
  minIouDiff4NewTarget: 0.3686
  minTrackerConfidence: 0.1513
  probationAge: 2
  maxShadowTrackingAge: 42

TrajectoryManagement:
  enableReAssoc: 1
  minMatchingScore4Overall: 0.6622
  minTrackletMatchingScore: 0.2940
  minMatchingScore4ReidSimilarity: 0.0771

VisualTracker:
  visualTrackerType: 1  # NvDCF
  filterLr: 0.0767
  gaussianSigma: 0.5687

ReID:
  reidType: 2  # Reid based reassoc
  batchSize: 1
  reidFeatureSize: 256
  networkMode: 1  # FP16
  inferDims: [3, 256, 128]
  tltEncodedModel: "/opt/nvidia/deepstream/deepstream/samples/models/Tracker/resnet50_market1501.etlt"
  modelEngineFile: "/app/deepstream_app/models/resnet50_market1501.etlt_b1_gpu0_fp16.engine"

Sample Output Logs

Click to expand relevant log excerpts

🔧 Creating DeepStream pipeline...
✅ All elements created
✅ Set streammux to use system memory (headless compatible)
🔧 Using CUSTOM tracker configuration
   Config file: /app/deepstream_app/tracker_config.yml
✅ ReID model file found
📁 Config file exists: True
📄 Config file size: 8767 bytes
🔍 Key parameters in config file:
   minDetectorConfidence: 0.1894
   minTrackerConfidence: 0.1513
   filterLr: 0.0767
✅ Tracker ll-config-file property confirmed: /app/tracker_config.yml

▶️  Starting playback...
[NvMultiObjectTracker] Loading TRT Engine for tracker ReID...
[NvMultiObjectTracker] Loading Complete!
[NvMultiObjectTracker] Initialized
⚠️  ConvBufManager may have failed - continuing with limited functionality

[TRACKER] Frame 1: 18 tracked objects
[TRACKER] Frame 2: 19 tracked objects
...
🏁 End of stream
📊 Processed 749 frames
📊 Total tracked objects: 7177

Questions

1. Why doesn’t changing config parameters affect tracking output?

   • Config loads successfully and parameters are displayed in logs
   • But tracking results are byte-for-byte identical regardless of parameter values
   • Is there a way to verify parameters are actually being applied at runtime?
   • Could this be related to the tracker using default values instead?

2. How can I verify if ReID is actually working?

   • ReID engine loads successfully but ConvBufManager shows warning
   • reidType: 2 is configured for ReID-based re-association
   • Pipeline runs but no way to confirm if appearance features are being used
   • Are there specific log messages or metrics that indicate ReID is active?

3. Is there a debug/verbose mode for nvtracker?

   • Need to see which config values are being used at runtime
   • Need to see if ReID feature extraction is happening
   • How to enable more detailed tracker logging?

Related Issues

• Forum: Headless mode fails with EGL error
• Forum: libnvds_nvmultiobjecttracker.so failed to load ReID

---

Tags: deepstream docker #nvtracker #reid jetson-orin #config-ignored headless

[kesong]

For headless setup, can you “unset DISPLAY” environment parameter?

Can you have a try with default deepstream test2 to check if change nvtracker config works?

[agodin]

Both recommendations worked:

1. unset DISPLAY instead of DISPLAY="" - Fixed headless operation
2. Tested with deepstream-app - Confirmed tracker loads correctly:
   • ✅ libnvds_nvmultiobjecttracker.so loads
   • ✅ ReID engine loads: resnet50_market1501.etlt_b1_gpu0_fp16.engine
   • ✅ [NvMultiObjectTracker] Initialized successfully
   • ✅ Pipeline completes without errors

The headless setup is working perfectly.

---

Issue: Tracker Config Parameters Seem to Be Ignored

While the tracker runs successfully and processes all frames, our custom config parameters don’t appear to be applied.

Config File Loads But Parameters Not Applied

Our tracker_config.yml contains:

minDetectorConfidence: 0.1894
minTrackerConfidence: 0.1513
maxTargetsPerStream: 150
minIouDiff4NewTarget: 0.3686
enableReAssoc: 1
# ... plus ReID configuration

Verification shows config is read:

tracker.set_property('ll-config-file', '/app/tracker_config.yml')
actual_config = tracker.get_property('ll-config-file')
# Returns: '/app/tracker_config.yml' ✅

# File exists and contains our parameters ✅
# Tracker initializes successfully ✅

But behavior suggests default config is used instead:

• All tracked objects get track_id = 0 (suggests tracking isn’t actually happening)
• No evidence of ReID re-association despite enableReAssoc: 1
• Appears to be passing through detections unchanged
• Our tuned confidence thresholds don’t seem to filter anything

Test Results

Test	Detector	Config	Result
deepstream-app (no detector)	❌ None	Custom tracker_config.yml	✅ Runs, no objects (expected)
Python script (manual injection)	✅ Manual	Custom tracker_config.yml	✅ Runs, but all ID=0 (not tracking)

---

Our Implementation

Goal: Track pre-computed detections from MOT17 dataset (no inference needed)

Pipeline: filesrc → qtdemux → h264parse → nvv4l2decoder → nvstreammux → nvtracker → fakesink

Approach: Inject detections as NvDsObjectMeta on streammux.src pad before tracker

Detection injection code (Python/PyDS)

def detection_injection_probe(self, pad, info, u_data):
    gst_buffer = info.get_buffer()
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    
    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
        frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        frame_number = frame_meta.frame_num
        
        if frame_number in self.detections:
            for det in self.detections[frame_number]:
                obj_meta = pyds.nvds_acquire_obj_meta_from_pool(batch_meta)
                
                obj_meta.rect_params.left = det['x']
                obj_meta.rect_params.top = det['y']
                obj_meta.rect_params.width = det['width']
                obj_meta.rect_params.height = det['height']
                obj_meta.confidence = det['confidence']
                obj_meta.class_id = 0  # Person
                obj_meta.unique_component_id = 1
                
                pyds.nvds_add_obj_meta_to_frame(frame_meta, obj_meta, None)
        
        l_frame = l_frame.next
    return Gst.PadProbeReturn.OK

# Probe attached to streammux src pad (before tracker)
streammux_src_pad.add_probe(Gst.PadProbeType.BUFFER, detection_injection_probe, 0)

Tracker configuration

tracker = Gst.ElementFactory.make("nvtracker", "tracker")
tracker.set_property('tracker-width', 640)
tracker.set_property('tracker-height', 384)
tracker.set_property('gpu-id', 0)
tracker.set_property('ll-lib-file', '/opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so')
tracker.set_property('ll-config-file', '/app/deepstream_app/tracker_config.yml')

# Link: streammux → tracker → fakesink

---

Questions

1. Does nvtracker require a GIE (detector) element in the pipeline to load config parameters, even if we’re injecting detections manually?

2. What’s the correct way to inject pre-computed detections so the tracker recognizes them as “new detections” requiring tracking?

   • Are we missing required metadata fields?
   • Should we set obj_meta.object_id to a specific value?
   • Is unique_component_id = 1 correct?

3. How can we verify that tracker config parameters are actually being applied (beyond just checking the property returns the right path)?

4. Is there a reference implementation or sample for using nvtracker with manual detection injection in Python?

---

Environment: Jetson (AGX Orin/Xavier), DeepStream 7.0, NvMultiObjectTracker (NvDeepSORT + ReID), Headless Docker, Python 3 + PyDS

Full script (~600 lines) available if helpful for diagnosis. Thank you!

[kesong]

For new question, please submit new topic. if deepstream-app works fine, please refer deepstream-app for your application.

[agodin]

No it doesn’t work fine. I want to use the params from the NvDeepSort but they wont apply to the runtime. When I start the app it always runs the same config no matter what params I try to modify. I know they are not being used because when I get the outputs from the tracker I compare them to MOT ground truth data I have and the scores are always the same. I have the engine running without issue and everything but I don’t have any different results which is my issue. Do you need file or snippet of code to better understand what I’m doing? Just ask and I’ll provide them.

[kesong]

Can you have a try the issue with deepstream_test_2.py: deepstream_python_apps/apps/deepstream-test2/deepstream_test_2.py at master · NVIDIA-AI-IOT/deepstream_python_apps · GitHub?