Bounding Box Flickering in YOLOv11 + NvDCF Tracking (DeepStream 6.3 on Jetson Xavier AGX)

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi everyone,

I’m working on real-time traffic analysis using live RTSP streams, where vehicles and persons are detected and tracked. I’m using YOLOv11 as the detection model and NvDCF as the tracker. However, I’m facing an issue where the bounding boxes around objects continuously flicker and blink during tracking.

Here are the steps I’ve taken so far:

  • Set inference interval = 2 (default) and increased tracking confidence as per NVIDIA’s recommendations.
  • Also tried inference interval = 0, but the flickering persisted.
  • Despite these optimizations, the issue remains.

Pipeline Structure:

📌 RTSP → Streammux → PGIE → Tracker → NVVIDCONV → CapsFilter → NVOSD → NVVIDCONV → CapsFilter → Encoder → H264Parser → Mux → FileSink

Attaching a sample video and tuned config_tracker_NvDCF_accuracy here


tracker_config.txt (2.8 KB)

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) - Jetson Xavier AGX
• DeepStream Version - 6.3
• JetPack Version (valid for Jetson only) - 5.1.4
• Issue Type( questions, new requirements, bugs) - Question

2

Does it only occur during tracking?

3

We tested the detection model separately, and the flickering issue did not occur. However, as soon as it was integrated into the pipeline, the flickering began.

4

In your tracking configuration make the probationAge 0, I ran into this issue awhile back. The probationAge is the number of frames that the object detector must detect the object in a row.

probationAge: 0
6

Please also share the original video and the sample video with only detector. Thanks!

7

I Have tried with probationAge: 0, but the issue still persists. I will attach the sample video with probation age as 0.

8

Original video and sample video with Yolov11 detector is attached below.



Note: The video shown here has not yet been integrated into the DeepStream pipeline. I assume you need to observe how YOLOv11 detection works.

9

Please use DeepStream pipeline to run inference and tracking. Please share the same video with/without nvtracker. So we can compare if the nvtracker cause the BBox flicking.

10

We have implemented the DeepStream pipeline without the tracker and concluded that the flickering issue still occurs. Previously, we had not run the pipeline without the tracker to debug this issue. I will provide the video and the corresponding code. (And also detection interval is taken as 0).

Can you explain why this flickering occurs when integrating the detection model into the pipeline, even though the model performed fine when tested separately? I will also attach the inference configuration file.


detect.py (10.7 KB)
infer.txt (719 Bytes)

11

@kesong I wanted to follow up on my query regarding the flickering issue in the DeepStream pipeline. I haven’t received a response yet, and I would really appreciate any insights you could share.

12

If you see the flicking when detection. Can you compare those settings with pytorch?

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
onnx-file=models/best_model3.onnx
model-engine-file=model_b1_gpu0_fp16_v3.engine
labelfile-path=labels.txt
batch-size=1
network-mode=2 # Enable FP16
num-detected-classes=12 # Update this to match number of classes in labels.txt
interval=0
gie-unique-id=1
process-mode=1
network-type=0
cluster-mode=2
maintain-aspect-ratio=0
symmetric-padding=0
parse-bbox-func-name=NvDsInferParseYolo
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

workspace-size=20000

[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=100

13

@kesong I have tried tuning the following parameters: network-mode=0 (enabling FP32), maintain-aspect-ratio=1, symmetric-padding=1, and adjusting nms-iou-threshold and pre-cluster-threshold. However, the flickering still persists. I also reviewed the YouTube video NVIDIA DeepStream Technical Deep Dive: Multi-Object Tracker, where I noticed slight flickering of the bounding boxes as well. Could this be an inherent issue with the DeepStream pipeline?

14

You need check all those settings to ensure those settings same with your Pytorch code. You need ensure the detector’s accuracy firstly.