Debug tracker state - ShadowTracks

[tracker.debugging]

Hardware Platform (GPU): NVIDIA GeForce RTX 5070
DeepStream Version DeepStreamSDK 8.0.0
NVIDIA GPU Driver Version: 580.95.05
CUDA Version: 13.0 NVIDIA-SMI 580.95.05

I am having issues with tracks flashing, they appear and dissapear even when the track is maintained and confidence is high enough.

maxShadowTrackingAge: 60

outputShadowTracks: 1

earlyTerminationAge: 30

enableReAssoc: 1

When debugging the tracker there is no way to find the state of each tracklet. How can I show them in python?

I am checking pyds.NvDsInferTensorMeta, pyds.NvDsUserMeta, pyds.get_nvds_LayerInfo, pyds.NvDsObjectMeta, pyds.NvDsBatchMeta, pyds.NvDsMetaType.NVDS_TRACKER_PAST_FRAME_META

In this objects there is info about detector and tracker confidence but age but not the tracklet state.

How can I access the state to know if it is ACTIVE, TENTATIVE, SHADOW?

Thanks

[kesong]

To access the state of each tracklet in the DeepStream tracker, you can use the pyds.NvDsPastFrameObj and pyds.NvDsPastFrameObjList objects. These objects contain information about the past frames of a track, including the state of each tracklet.

Here’s an example of how you can access the state of each tracklet in Python:

import pyds

# ...

# Get the past frame object list
past_frame_obj_list = pyds.NvDsPastFrameObjList.cast(data)

# Iterate over the past frame objects
for past_frame_obj in past_frame_obj_list:
    # Get the tracklet state
    tracklet_state = past_frame_obj.tracklet_state

    # Check the tracklet state
    if tracklet_state == pyds.NvDsTrackletState.ACTIVE:
        print("Tracklet is active")
    elif tracklet_state == pyds.NvDsTrackletState.TENTATIVE:
        print("Tracklet is tentative")
    elif tracklet_state == pyds.NvDsTrackletState.SHADOW:
        print("Tracklet is shadow")

In this example, we first get the past frame object list using pyds.NvDsPastFrameObjList.cast(data). We then iterate over the past frame objects and get the tracklet state using past_frame_obj.tracklet_state. Finally, we check the tracklet state and print a message accordingly.

Note that the pyds.NvDsTrackletState enum has the following values:

pyds.NvDsTrackletState.ACTIVE = 0
pyds.NvDsTrackletState.TENTATIVE = 1
pyds.NvDsTrackletState.SHADOW = 2

You can use these values to check the tracklet state in your code.

Also, make sure to check the DeepStream documentation for more information on the pyds.NvDsPastFrameObj and pyds.NvDsPastFrameObjList objects, as well as the pyds.NvDsTrackletState enum.

Here is the guide to dump misc data of nvtracker:Gst-nvtracker — DeepStream documentation

[tracker.debugging]

Hi Kesong,

I am using this pyds version and not finding the attributes:

pyds @ file:///opt/nvidia/deepstream/deepstream-8.0/sources/deepstream_python_apps/bindings/dist/pyds-1.2.2-cp312-cp312-linux_x86_64.whl#sha256=68b1a09d37debf1aa382029260027c6fa5f307efbab226c540df8085f2cac274

Traceback (most recent call last):
File “/workspace/scripts/detector2.py”, line 40, in appsink_callback
review_tracking_data(batch_meta)
File “/workspace/scripts/detector2.py”, line 279, in review_tracking_data
past_frame_obj_list = pyds.NvDsPastFrameObjList.cast(user_meta.user_meta_data)
^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module ‘pyds’ has no attribute ‘NvDsPastFrameObjList’

Where can I find that on the documentation?

NvDsTrackerMeta — Deepstream Deepstream Version: 8.0 documentation

Thanks,

[kesong]

You can check the past frame refer here: deepstream_python_apps/apps/deepstream-test2/deepstream_test_2.py at master · NVIDIA-AI-IOT/deepstream_python_apps · GitHub

Past frame means the target is in shadow tracking state. Can you share your tracker config file? Can you share the how to reproduce the issue?

[tracker.debugging]

Hello Kesong,

In the deepstream-test2 there is no output about the state of the tracklets to know if they are ACTIVE, TENTATIVE, SHADOW,…

This is how it is tested:

Getting an error

Traceback (most recent call last):
  File "/workspace/scripts/detector2.py", line 40, in appsink_callback
    review_tracking_data(batch_meta)
  File "/workspace/scripts/detector2.py", line 279, in review_tracking_data
    past_frame_obj_list = pyds.NvDsPastFrameObjList.cast(user_meta.user_meta_data)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'pyds' has no attribute 'NvDsPastFrameObjList'

Working when removing the indicated code above:

past_frame_obj_list = pyds.NvDsPastFrameObjList.cast(user_meta.user_meta_data)
past_frame_obj_list.print_list()
if past_frame_obj_list:
    print("past_frame_obj_list")
for past_frame_obj in past_frame_obj_list:
    tracklet_state = past_frame_obj.tracklet_state
    if tracklet_state == pyds.NvDsTrackletState.ACTIVE:
        print("Tracklet State: ACTIVE")
    elif tracklet_state == pyds.NvDsTrackletState.TENTATIVE:
        print("Tracklet State: TENTATIVE")
    elif tracklet_state == pyds.NvDsTrackletState.SHADOW:
        print("Tracklet State: SHADOW")

#!/usr/bin/env python3
"""
DeepStream detector with bbox matching probe - Python GStreamer version of detector.py

This replicates detector.py but uses Python GStreamer to allow attaching a probe
for bbox matching and soft label extraction.
"""

import argparse
import sys
import signal
import ctypes
import gi
from colorama import Fore, Style

gi.require_version('Gst', '1.0')
from gi.repository import Gst, GLib

import pyds
import numpy as np


def appsink_callback(appsink):
    """
    Appsink callback to match tracked objects with raw boxes and extract soft labels.
    """
    sample = appsink.emit("pull-sample")
    if not sample:
        return Gst.FlowReturn.OK

    gst_buffer = sample.get_buffer()
    if not gst_buffer:
        return Gst.FlowReturn.OK

    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))

    l_frame = batch_meta.frame_meta_list

    print("\n--- Review Data ---")
    review_tracking_data(batch_meta)
    print("\n--- NVDS Frame Meta ---")
    review_nvdsframemeta(batch_meta)

    while l_frame is not None:
        try:
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break

        # Find tensor metadata
        l_user = frame_meta.frame_user_meta_list
        tensor_meta = None

        while l_user is not None:
            try:
                user_meta = pyds.NvDsUserMeta.cast(l_user.data)
                if user_meta.base_meta.meta_type == pyds.NvDsMetaType.NVDSINFER_TENSOR_OUTPUT_META:
                    tensor_meta = pyds.NvDsInferTensorMeta.cast(user_meta.user_meta_data)
                    break
            except StopIteration:
                break
            try:
                l_user = l_user.next
            except StopIteration:
                break

        if tensor_meta is None:
            try:
                l_frame = l_frame.next
            except StopIteration:
                break
            continue

        # Extract raw tensors from model outputs
        raw_boxes = None
        raw_scores = None
        raw_labels = None

        for i in range(tensor_meta.num_output_layers):
            layer = pyds.get_nvds_LayerInfo(tensor_meta, i)
            layer_name = layer.layerName

            if layer_name == "boxes":
                # Shape: [1, 300, 4] = 1200 elements
                ptr = pyds.get_ptr(layer.buffer)
                raw_boxes = np.ctypeslib.as_array(
                    (ctypes.c_float * 1200).from_address(ptr)
                ).reshape(1, 300, 4)
            elif layer_name == "scores":
                # Shape: [1, 300] = 300 elements
                ptr = pyds.get_ptr(layer.buffer)
                raw_scores = np.ctypeslib.as_array(
                    (ctypes.c_float * 300).from_address(ptr)
                ).reshape(1, 300)
            elif layer_name == "labels":
                # Shape: [1, 300, 12] = 3600 elements
                ptr = pyds.get_ptr(layer.buffer)
                raw_labels = np.ctypeslib.as_array(
                    (ctypes.c_float * 3600).from_address(ptr)
                ).reshape(1, 300, 12)

        if raw_boxes is None or raw_labels is None:
            try:
                l_frame = l_frame.next
            except StopIteration:
                break
            continue

        # Print raw detector outputs (pre-tracker)
        print(f"{Fore.RED} frame {frame_meta.frame_num} {Style.RESET_ALL}:")
        for idx in range(300):
            score = raw_scores[0, idx] if raw_scores is not None else 0.0
            if score > 0.15:  # Only print detections above threshold
                box = raw_boxes[0, idx]
                label = int(np.argmax(raw_labels[0, idx])) if raw_labels is not None else -1
                print(f"{Fore.GREEN}  Detector: idx={idx}, Class={label}, Score={score:.2f}, BBox: x1={box[0]:.0f}, y1={box[1]:.0f}, x2={box[2]:.0f}, y2={box[3]:.0f} {Style.RESET_ALL}")

        # Match tracked objects with raw boxes using exact coordinate matching
        l_obj = frame_meta.obj_meta_list
        matched_detections = []

        while l_obj is not None:
            try:
                obj_meta = pyds.NvDsObjectMeta.cast(l_obj.data)
            except StopIteration:
                break

            print(f"{Fore.BLUE}  Tracker: Track ID={obj_meta.object_id}, Class ID={obj_meta.class_id}, Confidence={obj_meta.confidence:.2f}, BBox: left={obj_meta.rect_params.left:.0f}, top={obj_meta.rect_params.top:.0f}, width={obj_meta.rect_params.width:.0f}, height={obj_meta.rect_params.height:.0f}{Style.RESET_ALL}")
            
            # Object bbox in x1y1x2y2 format
            obj_x1 = obj_meta.rect_params.left
            obj_y1 = obj_meta.rect_params.top
            obj_x2 = obj_meta.rect_params.left + obj_meta.rect_params.width
            obj_y2 = obj_meta.rect_params.top + obj_meta.rect_params.height

            print("## bbbboxx", (obj_x1, obj_y1, obj_x2, obj_y2))


            # Find exact matching raw box
            matched_idx = -1

            for idx in range(300):
                raw_box = raw_boxes[0, idx]
                # Check if coordinates match exactly (or very close due to float precision)
                if (abs(raw_box[0] - obj_x1) < 0.01 and
                    abs(raw_box[1] - obj_y1) < 0.01 and
                    abs(raw_box[2] - obj_x2) < 0.01 and
                    abs(raw_box[3] - obj_y2) < 0.01):
                    matched_idx = idx
                    break

            print(f" ###   Matched raw idx: {matched_idx}")
            # Only add if we found exact match
            if matched_idx != -1:
                soft_labels = raw_labels[0, matched_idx]

                matched_detections.append({
                    'track_id': obj_meta.object_id,
                    'class_id': obj_meta.class_id,
                    'confidence': obj_meta.confidence,
                    'bbox': [obj_x1, obj_y1, obj_x2, obj_y2],
                    'soft_labels': soft_labels.copy(),
                    'raw_idx': matched_idx
                })
                

            try:
                l_obj = l_obj.next
            except StopIteration:
                break

        # For now, just print summary every 30 frames
        if frame_meta.frame_num % 30 == 0 and len(matched_detections) > 0:
            print(f"\n=== Frame {frame_meta.frame_num} ===")
            print(f"Matched {len(matched_detections)} detections")
            for i, det in enumerate(matched_detections[:3]):  # Print first 3
                print(f"  Detection {i}:")
                print(f"    Track ID: {det['track_id']}")
                print(f"    Class ID: {det['class_id']}")
                print(f"    Confidence: {det['confidence']:.3f}")
                print(f"    Raw idx: {det['raw_idx']}")
                print(f"    Soft labels (top 3): {np.sort(det['soft_labels'])[-3:]}")

        try:
            l_frame = l_frame.next
        except StopIteration:
            break

    return Gst.FlowReturn.OK



def review_nvdsframemeta(batch_meta: pyds.NvDsBatchMeta) -> None:
    """
    Review NvDsFrameMeta data in the batch meta for debugging.
    """
    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
        try:
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break

        print(f"Frame: pad_index={frame_meta.pad_index} batch_id={frame_meta.batch_id} "
              f"frame_num={frame_meta.frame_num} buf_pts={frame_meta.buf_pts} "
              f"ntp_timestamp={frame_meta.ntp_timestamp} source_id={frame_meta.source_id} "
              f"num_surfaces_per_frame={frame_meta.num_surfaces_per_frame}")
        print(f"  source_frame_width={frame_meta.source_frame_width} "
              f"source_frame_height={frame_meta.source_frame_height} "
              f"surface_type={frame_meta.surface_type} surface_index={frame_meta.surface_index}")
        print(f"  num_obj_meta={frame_meta.num_obj_meta} bInferDone={frame_meta.bInferDone}")

        # Iterate through object meta list
        l_obj = frame_meta.obj_meta_list
        while l_obj is not None:
            try:
                obj_meta = pyds.NvDsObjectMeta.cast(l_obj.data)
            except StopIteration:
                break

            color_map = {
                0: Fore.LIGHTBLUE_EX,
                1: Fore.GREEN,
                2: Fore.LIGHTCYAN_EX,
                3: Fore.MAGENTA,
                4: Fore.CYAN,
            }
            color = color_map.get(obj_meta.class_id, Fore.YELLOW)
            print(f"{color}   Object: class_id={obj_meta.class_id} object_id={obj_meta.object_id} "
                  f"confidence={obj_meta.confidence:.2f} tracker_confidence={obj_meta.tracker_confidence:.2f}{Style.RESET_ALL}")
            print(f"      rect: left={obj_meta.rect_params.left:.1f} top={obj_meta.rect_params.top:.1f} "
                  f"width={obj_meta.rect_params.width:.1f} height={obj_meta.rect_params.height:.1f}")
            print(f"      detector: left={obj_meta.detector_bbox_info.org_bbox_coords.left:.1f} "
                  f"top={obj_meta.detector_bbox_info.org_bbox_coords.top:.1f} "
                  f"width={obj_meta.detector_bbox_info.org_bbox_coords.width:.1f} "
                  f"height={obj_meta.detector_bbox_info.org_bbox_coords.height:.1f}")
            print(f"      tracker: left={obj_meta.tracker_bbox_info.org_bbox_coords.left:.1f} "
                  f"top={obj_meta.tracker_bbox_info.org_bbox_coords.top:.1f} "
                  f"width={obj_meta.tracker_bbox_info.org_bbox_coords.width:.1f} "
                  f"height={obj_meta.tracker_bbox_info.org_bbox_coords.height:.1f}")

            try:
                l_obj = l_obj.next
            except StopIteration:
                break

        try:
            l_frame = l_frame.next
        except StopIteration:
            break


def review_tracking_data(batch_meta : pyds.NvDsBatchMeta) -> None:
    """
    Review tracking data in the batch meta for debugging.
    """
    l_user=batch_meta.batch_user_meta_list #Retrieve glist of NvDsUserMeta objects from given NvDsBatchMeta object
    while l_user is not None:
        try:
            user_meta=pyds.NvDsUserMeta.cast(l_user.data)
        except StopIteration:
            break
        if(user_meta and user_meta.base_meta.meta_type==pyds.NvDsMetaType.NVDS_TRACKER_PAST_FRAME_META): #Make sure metatype is correct
            try:
                pPastFrameObjBatch = pyds.NvDsTargetMiscDataBatch.cast(user_meta.user_meta_data) #See NvDsTargetMiscDataBatch for details
            except StopIteration:
                break
            for trackobj in pyds.NvDsTargetMiscDataBatch.list(pPastFrameObjBatch): # NvDsTargetMiscDataStream objects
                # NvDsTargetMiscDataStream attributes
                print(f"streamId={trackobj.streamID} surfaceStreamID={trackobj.surfaceStreamID} numAllocated={trackobj.numAllocated} numFilled={trackobj.numFilled}")
                for pastframeobj in pyds.NvDsTargetMiscDataStream.list(trackobj): # NvDsFrameObjList objects
                    # NvDsTargetMiscDataObject attributes
                    print(f"numobj={pastframeobj.numObj} uniqueId= {pastframeobj.uniqueId} classId={pastframeobj.classId} objLabel={pastframeobj.objLabel}")
                    for objlist in pyds.NvDsTargetMiscDataObject.list(pastframeobj): # NvDsFrameObj objects
                        # NvDsTargetMiscDataFrame attributes
                        print(f"frameNum: {objlist.frameNum} confidence:{objlist.confidence:.2f} age:{objlist.age} [left:{objlist.tBbox.left:.1f} width:{objlist.tBbox.width:.1f} top:{objlist.tBbox.top:.1f} height:{objlist.tBbox.height:.1f}]")
                        
            
            past_frame_obj_list = pyds.NvDsPastFrameObjList.cast(user_meta.user_meta_data)
            past_frame_obj_list.print_list()
            if past_frame_obj_list:
                print("past_frame_obj_list")
            for past_frame_obj in past_frame_obj_list:
                tracklet_state = past_frame_obj.tracklet_state
                if tracklet_state == pyds.NvDsTrackletState.ACTIVE:
                    print("Tracklet State: ACTIVE")
                elif tracklet_state == pyds.NvDsTrackletState.TENTATIVE:
                    print("Tracklet State: TENTATIVE")
                elif tracklet_state == pyds.NvDsTrackletState.SHADOW:
                    print("Tracklet State: SHADOW")



        try:
            l_user=l_user.next
        except StopIteration:
            break


def osd_sink_pad_buffer_probe(pad, info, u_data):
    """
    Probe to modify OSD colors for shadow tracks (lost but still projected).
    Shadow tracks are shown in yellow/orange, active tracks in green.
    """
    gst_buffer = info.get_buffer()
    if not gst_buffer:
        return Gst.PadProbeReturn.OK

    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    if not batch_meta:
        return Gst.PadProbeReturn.OK

    l_frame = batch_meta.frame_meta_list

    while l_frame is not None:
        try:
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break

        l_obj = frame_meta.obj_meta_list
        while l_obj is not None:
            try:
                obj_meta = pyds.NvDsObjectMeta.cast(l_obj.data)
            except StopIteration:
                break


            obj_meta.rect_params.border_color.set(0.0, 1.0, 1.0, 1.0)
            obj_meta.rect_params.border_width = 1
            # Make text orange too
            obj_meta.text_params.font_params.font_color.set(1.0, 0.6, 0.0, 1.0)


            try:
                l_obj = l_obj.next
            except StopIteration:
                break

        try:
            l_frame = l_frame.next
        except StopIteration:
            break

    return Gst.PadProbeReturn.OK


class DeepStreamDetector:
    def __init__(self, socket_path: str):
        self.socket_path = socket_path
        self.pipeline = None
        self.loop = None

        Gst.init(None)

    def create_pipeline(self):
        """Create GStreamer pipeline matching detector.py structure"""

        self.pipeline = Gst.Pipeline()

        # Source
        source = Gst.ElementFactory.make("nvunixfdsrc", "source")
        source.set_property("socket-path", self.socket_path)

        # Caps filter for source
        caps_source = Gst.ElementFactory.make("capsfilter", "caps_source")
        caps_source.set_property("caps",
            Gst.Caps.from_string("video/x-raw(memory:NVMM),format=NV12"))

        # Video convert
        nvvidconv = Gst.ElementFactory.make("nvvideoconvert", "nvvidconv")

        # Caps filter after convert
        caps_convert = Gst.ElementFactory.make("capsfilter", "caps_convert")
        caps_convert.set_property("caps",
            Gst.Caps.from_string("video/x-raw(memory:NVMM)"))

        # Stream mux
        streammux = Gst.ElementFactory.make("nvstreammux", "mux")
        streammux.set_property("batch-size", 1)
        streammux.set_property("width", 1280)
        streammux.set_property("height", 1280)
        streammux.set_property("nvbuf-memory-type", 0)

        # Inference
        nvinfer = Gst.ElementFactory.make("nvinfer", "nvinfer")
        nvinfer.set_property("config-file-path", "/workspace/configs/rtdetr_config.txt")

        # Queue after inference
        queue_infer = Gst.ElementFactory.make("queue", "queue_infer")
        queue_infer.set_property("max-size-buffers", 30)
        queue_infer.set_property("max-size-time", 0)
        queue_infer.set_property("max-size-bytes", 0)
        queue_infer.set_property("leaky", 2)  # downstream

        # Tracker
        nvtracker = Gst.ElementFactory.make("nvtracker", "nvtracker")
        nvtracker.set_property("tracker-width", 320)
        nvtracker.set_property("tracker-height", 320)
        nvtracker.set_property("ll-lib-file",
            "/opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so")
        
        nvtracker.set_property("ll-config-file","/workspace/configs/config_tracker_NvDCF_accuracy.yml")
        

        # Tee after tracker for display + appsink branches
        tee = Gst.ElementFactory.make("tee", "tee")

        # Branch 1: Display path
        queue_display_branch = Gst.ElementFactory.make("queue", "queue_display_branch")
        queue_display_branch.set_property("max-size-buffers", 10)
        queue_display_branch.set_property("max-size-time", 0)
        queue_display_branch.set_property("max-size-bytes", 0)

        # OSD
        nvosd = Gst.ElementFactory.make("nvdsosd", "nvosd")

        # Branch 2: Appsink path
        queue_appsink = Gst.ElementFactory.make("queue", "queue_appsink")
        queue_appsink.set_property("max-size-buffers", 10)
        queue_appsink.set_property("max-size-time", 0)
        queue_appsink.set_property("max-size-bytes", 0)

        appsink = Gst.ElementFactory.make("appsink", "appsink")
        appsink.set_property("emit-signals", True)
        appsink.set_property("sync", False)
        appsink.connect("new-sample", appsink_callback)

        # Video convert for display
        nvvidconv2 = Gst.ElementFactory.make("nvvideoconvert", "nvvidconv2")

        # Caps for display
        caps_display = Gst.ElementFactory.make("capsfilter", "caps_display")
        caps_display.set_property("caps",
            Gst.Caps.from_string("video/x-raw(memory:NVMM),width=1920,height=1080,pixel-aspect-ratio=1/1"))

        # Queue for display
        queue_display = Gst.ElementFactory.make("queue", "queue_display")
        queue_display.set_property("max-size-buffers", 5)
        queue_display.set_property("max-size-time", 0)
        queue_display.set_property("max-size-bytes", 0)
        queue_display.set_property("leaky", 2)

        # Sink
        sink = Gst.ElementFactory.make("nveglglessink", "sink")
        sink.set_property("sync", False)

        # Add all elements to pipeline
        elements = [
            source, caps_source, nvvidconv, caps_convert,
            streammux, nvinfer, queue_infer, nvtracker, tee,
            # Branch 1: Display
            queue_display_branch, nvosd, nvvidconv2, caps_display, queue_display, sink,
            # Branch 2: Appsink
            queue_appsink, appsink
        ]

        for elem in elements:
            if not elem:
                print(f"ERROR: Failed to create element")
                return False
            self.pipeline.add(elem)

        # Link elements
        if not source.link(caps_source):
            print("ERROR: Failed to link source -> caps_source")
            return False
        if not caps_source.link(nvvidconv):
            print("ERROR: Failed to link caps_source -> nvvidconv")
            return False
        if not nvvidconv.link(caps_convert):
            print("ERROR: Failed to link nvvidconv -> caps_convert")
            return False

        # Get streammux sink pad and link
        sinkpad = streammux.request_pad_simple("sink_0")
        srcpad = caps_convert.get_static_pad("src")
        if srcpad.link(sinkpad) != Gst.PadLinkReturn.OK:
            print("ERROR: Failed to link to streammux")
            return False

        # Link rest of pipeline up to tracker
        if not streammux.link(nvinfer):
            print("ERROR: Failed to link streammux -> nvinfer")
            return False
        if not nvinfer.link(queue_infer):
            print("ERROR: Failed to link nvinfer -> queue_infer")
            return False
        if not queue_infer.link(nvtracker):
            print("ERROR: Failed to link queue_infer -> nvtracker")
            return False
        if not nvtracker.link(tee):
            print("ERROR: Failed to link nvtracker -> tee")
            return False

        # Branch 1: Display path
        tee_display_pad = tee.request_pad_simple("src_%u")
        queue_display_sink = queue_display_branch.get_static_pad("sink")
        if tee_display_pad.link(queue_display_sink) != Gst.PadLinkReturn.OK:
            print("ERROR: Failed to link tee -> queue_display_branch")
            return False

        if not queue_display_branch.link(nvosd):
            print("ERROR: Failed to link queue_display_branch -> nvosd")
            return False

        # Add probe to color shadow tracks differently
        osd_sink_pad = nvosd.get_static_pad("sink")
        if osd_sink_pad:
            osd_sink_pad.add_probe(Gst.PadProbeType.BUFFER, osd_sink_pad_buffer_probe, 0)
            print("Added OSD probe for shadow track coloring")

        if not nvosd.link(nvvidconv2):
            print("ERROR: Failed to link nvosd -> nvvidconv2")
            return False
        if not nvvidconv2.link(caps_display):
            print("ERROR: Failed to link nvvidconv2 -> caps_display")
            return False
        if not caps_display.link(queue_display):
            print("ERROR: Failed to link caps_display -> queue_display")
            return False
        if not queue_display.link(sink):
            print("ERROR: Failed to link queue_display -> sink")
            return False

        # Branch 2: Appsink path for ROS2
        tee_appsink_pad = tee.request_pad_simple("src_%u")
        queue_appsink_sink = queue_appsink.get_static_pad("sink")
        if tee_appsink_pad.link(queue_appsink_sink) != Gst.PadLinkReturn.OK:
            print("ERROR: Failed to link tee -> queue_appsink")
            return False

        if not queue_appsink.link(appsink):
            print("ERROR: Failed to link queue_appsink -> appsink")
            return False

        print("Pipeline created with tee: display branch + appsink branch for ROS2")

        return True

    def bus_call(self, bus, message, loop):
        """Handle bus messages"""
        t = message.type

        if t == Gst.MessageType.EOS:
            print("End-of-stream")
            loop.quit()
        elif t == Gst.MessageType.ERROR:
            err, debug = message.parse_error()
            print(f"ERROR: {err.message}")
            print(f"Debug info: {debug}")
            loop.quit()
        elif t == Gst.MessageType.WARNING:
            warn, debug = message.parse_warning()
            print(f"WARNING: {warn.message}")

        return True

    def run(self):
        """Run the pipeline"""
        if not self.create_pipeline():
            print("Failed to create pipeline")
            return False

        self.loop = GLib.MainLoop()

        bus = self.pipeline.get_bus()
        bus.add_signal_watch()
        bus.connect("message", self.bus_call, self.loop)

        print("Starting detector...")
        ret = self.pipeline.set_state(Gst.State.PLAYING)
        if ret == Gst.StateChangeReturn.FAILURE:
            print("ERROR: Unable to set pipeline to PLAYING state")
            return False

        print("Pipeline running...")
        print("Press Ctrl+C to stop")

        try:
            self.loop.run()
        except KeyboardInterrupt:
            print("\nStopping pipeline...")

        self.pipeline.set_state(Gst.State.NULL)
        return True


def main():
    parser = argparse.ArgumentParser(
        description="Run DeepStream Detection with Bbox Matching Probe"
    )
    parser.add_argument(
        "--socket-path",
        default="/tmp/deepstream-video.sock",
        help="Unix socket path for video input",
    )
    args = parser.parse_args()

    detector = DeepStreamDetector(socket_path=args.socket_path)
    success = detector.run()

    sys.exit(0 if success else 1)


if __name__ == "__main__":
    main()

%YAML:1.0
---
####################################################################################################
# SPDX-FileCopyrightText: Copyright (c) 2021-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: LicenseRef-NvidiaProprietary
#
# NVIDIA CORPORATION, its affiliates and licensors retain all intellectual
# property and proprietary rights in and to this material, related
# documentation and any modifications thereto. Any use, reproduction,
# disclosure or distribution of this material and related documentation
# without an express license agreement from NVIDIA CORPORATION or
# its affiliates is strictly prohibited.
####################################################################################################

BaseConfig:
  minDetectorConfidence: 0.10    # If the confidence of a detector bbox is lower than this, then it won't be considered for tracking

TargetManagement:
  enableBboxUnClipping: 1    # In case the bbox is likely to be clipped by image border, unclip bbox
  preserveStreamUpdateOrder: 0    # When assigning new target ids, preserve input streams' order to keep target ids in a deterministic order over multuple runs
  maxTargetsPerStream: 10    # Max number of targets to track per stream. Recommended to set >10. Note: this value should account for the targets being tracked in shadow mode as well. Max value depends on the GPU memory capacity

  # [Creation & Termination Policy]
  minIouDiff4NewTarget: 0.3    # If the IOU between the newly detected object and any of the existing targets is higher than this threshold, this newly detected object will be discarded.
  minTrackerConfidence: 0.15    # If the confidence of an object tracker is lower than this on the fly, then it will be tracked in shadow mode. Valid Range: [0.0, 1.0]
  probationAge: 5    # If the target's age exceeds this, the target will be considered to be valid.
  maxShadowTrackingAge: 60    # Max length of shadow tracking. If the shadowTrackingAge exceeds this limit, the tracker will be terminated.
  earlyTerminationAge: 100    # If the shadowTrackingAge reaches this threshold while in TENTATIVE period, the target will be terminated prematurely.
  outputShadowTracks: 1    # Output shadow tracks to downstream elements (required to display them)


TrajectoryManagement:
  useUniqueID: 0    # Use 64-bit long Unique ID when assignining tracker ID. Default is [true]
  enableReAssoc: 1    # Enable Re-Assoc

  # [Re-Assoc Metric: Thresholds for valid candidates]
  minMatchingScore4Overall: 0.65    # min matching score for overall
  minTrackletMatchingScore: 0.30    # min tracklet similarity score for re-assoc
  minMatchingScore4ReidSimilarity: 0.05    # min reid similarity score for re-assoc

  # [Re-Assoc Metric: Weights]
  matchingScoreWeight4TrackletSimilarity: 0.80    # weight for tracklet similarity score
  matchingScoreWeight4ReidSimilarity: 0.40    # weight for reid similarity score

  # [Re-Assoc: Motion-based]
  minTrajectoryLength4Projection: 34    # min trajectory length required to make projected trajectory
  prepLength4TrajectoryProjection: 58    # the length of the trajectory during which the state estimator is updated to make projections
  trajectoryProjectionLength: 33    # the length of the projected trajectory
  maxAngle4TrackletMatching: 67    # max angle difference for tracklet matching [degree]
  minSpeedSimilarity4TrackletMatching: 0.0574    # min speed similarity for tracklet matching
  minBboxSizeSimilarity4TrackletMatching: 0.1013    # min bbox size similarity for tracklet matching
  maxTrackletMatchingTimeSearchRange: 27    # the search space in time for max tracklet similarity
  trajectoryProjectionProcessNoiseScale: 0.0100    # trajectory projector's process noise scale w.r.t. state estimator
  trajectoryProjectionMeasurementNoiseScale: 100    # trajectory projector's measurement noise scale w.r.t. state estimator
  trackletSpacialSearchRegionScale: 0.0100    # the search region scale for peer tracklet

  # [Re-Assoc: Reid based. Reid model params are set in ReID section]
  reidExtractionInterval: 8    # frame interval to extract reid features per target

DataAssociator:
  dataAssociatorType: 0    # the type of data associator among { DEFAULT= 0 }
  associationMatcherType: 1    # the type of matching algorithm among { GREEDY=0, CASCADED=1 }
  checkClassMatch: 1    # If checked, only the same-class objects are associated with each other. Default: true

  # [Association Metric: Thresholds for valid candidates]
  minMatchingScore4Overall: 0.0222    # Min total score
  minMatchingScore4SizeSimilarity: 0.3552    # Min bbox size similarity score
  minMatchingScore4Iou: 0.0548   # Min IOU score
  minMatchingScore4VisualSimilarity: 0.5043    # Min visual similarity score

  # [Association Metric: Weights]
  matchingScoreWeight4VisualSimilarity: 0.3951    # Weight for the visual similarity (in terms of correlation response ratio)
  matchingScoreWeight4SizeSimilarity: 0.6003    # Weight for the Size-similarity score
  matchingScoreWeight4Iou: 0.4033    # Weight for the IOU score

  # [Association Metric: Tentative detections] only uses iou similarity for tentative detections
  tentativeDetectorConfidence: 0.1024    # If a detection's confidence is lower than this but higher than minDetectorConfidence, then it's considered as a tentative detection
  minMatchingScore4TentativeIou: 0.2852    # Min iou threshold to match targets and tentative detection

StateEstimator:
  stateEstimatorType: 1    # the type of state estimator among { DUMMY=0, SIMPLE=1, REGULAR=2 }

  # [Dynamics Modeling]
  processNoiseVar4Loc: 6810.8668    # Process noise variance for bbox center
  processNoiseVar4Size: 1541.8647   # Process noise variance for bbox size
  processNoiseVar4Vel: 1348.4874    # Process noise variance for velocity
  measurementNoiseVar4Detector: 100.0000   # Measurement noise variance for detector's detection
  measurementNoiseVar4Tracker: 293.3238    # Measurement noise variance for tracker's localization

VisualTracker:
  visualTrackerType: 2    # the type of visual tracker among { DUMMY=0, NvDCF_legacy=1, NvDCF_VPI=2 }

  # [NvDCF: Feature Extraction]
  useColorNames: 1    # Use ColorNames feature
  useHog: 1    # Use Histogram-of-Oriented-Gradient (HOG) feature
  featureImgSizeLevel: 3    # Size of a feature image. Valid range: {1, 2, 3, 4, 5}, from the smallest to the largest
  featureFocusOffsetFactor_y: -0.1054    # The offset for the center of hanning window relative to the feature height. The center of hanning window would move by (featureFocusOffsetFactor_y*featureMatSize.height) in vertical direction

  # [NvDCF: Correlation Filter]
  filterLr: 0.0767    # learning rate for DCF filter in exponential moving average. Valid Range: [0.0, 1.0]
  filterChannelWeightsLr: 0.0339    # learning rate for the channel weights among feature channels. Valid Range: [0.0, 1.0]
  gaussianSigma: 0.5687    # Standard deviation for Gaussian for desired response when creating DCF filter [pixels]

ReID:
  reidType: 2    # The type of reid among { DUMMY=0, NvDEEPSORT=1, Reid based reassoc=2, both NvDEEPSORT and reid based reassoc=3}

  # [Reid Network Info]
  batchSize: 100    # Batch size of reid network
  workspaceSize: 1000    # Workspace size to be used by reid engine, in MB
  reidFeatureSize: 256    # Size of reid feature
  reidHistorySize: 100    # Max number of reid features kept for one object
  inferDims: [3, 256, 128]    # Reid network input dimension CHW or HWC based on inputOrder
  networkMode: 1    # Reid network inference precision mode among {fp32=0, fp16=1, int8=2 }

  # [Input Preprocessing]
  inputOrder: 0    # Reid network input order among { NCHW=0, NHWC=1 }. Batch will be converted to the specified order before reid input.
  colorFormat: 0    # Reid network input color format among {RGB=0, BGR=1 }. Batch will be converted to the specified color before reid input.
  offsets: [123.6750, 116.2800, 103.5300]    # Array of values to be subtracted from each input channel, with length equal to number of channels
  netScaleFactor: 0.01735207    # Scaling factor for reid network input after substracting offsets
  keepAspc: 1    # Whether to keep aspc ratio when resizing input objects for reid
  useVPICropScaler: 1 # Use VPI for image cropping and rescaling

  # [Output Postprocessing]
  addFeatureNormalization: 1       # If reid feature is not normalized in network, adding normalization on output so each reid feature has l2 norm equal to 1
  minVisibility4GalleryUpdate: 0.6 # Add ReID embedding to the gallery only if the visibility is not lower than this

  # [Paths and Names]
  # tltEncodedModel: "/opt/nvidia/deepstream/deepstream/samples/models/Tracker/resnet50_market1501.etlt" # NVIDIA TAO model path
  # tltModelKey: "nvidia_tao" # NVIDIA TAO model key
  # modelEngineFile: "/opt/nvidia/deepstream/deepstream/samples/models/Tracker/resnet50_market1501.etlt_b100_gpu0_fp16.engine" # Engine file path
  modelEngineFile: "../model/new_model.onnx_b1_gpu0_fp16.engine"


  useBufferedOutput: 1  # Enable for smoothing

Thanks,

[kesong]

Do you see the issue with deepstream-test2? Can you share the video to reproduce the issue? You can share with private message if don’t want to public it.

[tracker.debugging]

For instance in deepstream_test2.py some of the tracks are not included even if the confidence of the detector is high.

The `minTrackerConfidence: 0.02` so It shoud be included right?

The inclusion of detections inside the tracker seens to not follow parameters

#!/usr/bin/env python3

################################################################################
# SPDX-FileCopyrightText: Copyright (c) 2019-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################

import sys
sys.path.append('../')
import platform
import configparser
import ctypes
import logging
import numpy as np
from colorama import Fore, Style

import gi
gi.require_version('Gst', '1.0')
from gi.repository import GLib, Gst
from common.platform_info import PlatformInfo
from common.bus_call import bus_call

import pyds

# Set up logger to stdout
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
handler = logging.StreamHandler(sys.stdout)
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)

PGIE_CLASS_ID_VEHICLE = 0
PGIE_CLASS_ID_BICYCLE = 1
PGIE_CLASS_ID_PERSON = 2
PGIE_CLASS_ID_ROADSIGN = 3
MUXER_BATCH_TIMEOUT_USEC = 33000





def appsink_callback(appsink):
    """
    Appsink callback to match tracked objects with raw boxes and extract soft labels.
    """
    sample = appsink.emit("pull-sample")
    if not sample:
        return Gst.FlowReturn.OK

    gst_buffer = sample.get_buffer()
    if not gst_buffer:
        return Gst.FlowReturn.OK

    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))

    l_frame = batch_meta.frame_meta_list

    logger.debug("\n--- Review Data ---")
    review_tracking_data(batch_meta)
    logger.debug("\n--- NVDS Frame Meta ---")
    review_nvdsframemeta(batch_meta)

    while l_frame is not None:
        try:
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break

        # Find tensor metadata
        l_user = frame_meta.frame_user_meta_list
        tensor_meta = None

        while l_user is not None:
            try:
                user_meta = pyds.NvDsUserMeta.cast(l_user.data)
                if user_meta.base_meta.meta_type == pyds.NvDsMetaType.NVDSINFER_TENSOR_OUTPUT_META:
                    tensor_meta = pyds.NvDsInferTensorMeta.cast(user_meta.user_meta_data)
                    break
            except StopIteration:
                break
            try:
                l_user = l_user.next
            except StopIteration:
                break

        if tensor_meta is None:
            try:
                l_frame = l_frame.next
            except StopIteration:
                break
            continue

        # Extract raw tensors from model outputs
        raw_boxes = None
        raw_scores = None
        raw_labels = None

        for i in range(tensor_meta.num_output_layers):
            layer = pyds.get_nvds_LayerInfo(tensor_meta, i)
            layer_name = layer.layerName

            if layer_name == "boxes":
                # Shape: [1, 300, 4] = 1200 elements
                ptr = pyds.get_ptr(layer.buffer)
                raw_boxes = np.ctypeslib.as_array(
                    (ctypes.c_float * 1200).from_address(ptr)
                ).reshape(1, 300, 4)
            elif layer_name == "scores":
                # Shape: [1, 300] = 300 elements
                ptr = pyds.get_ptr(layer.buffer)
                raw_scores = np.ctypeslib.as_array(
                    (ctypes.c_float * 300).from_address(ptr)
                ).reshape(1, 300)
            elif layer_name == "labels":
                # Shape: [1, 300, 12] = 3600 elements
                ptr = pyds.get_ptr(layer.buffer)
                raw_labels = np.ctypeslib.as_array(
                    (ctypes.c_float * 3600).from_address(ptr)
                ).reshape(1, 300, 12)

        if raw_boxes is None or raw_labels is None:
            try:
                l_frame = l_frame.next
            except StopIteration:
                break
            continue

        # Log raw detector outputs (pre-tracker)
        logger.debug(f"{Fore.RED} frame {frame_meta.frame_num} {Style.RESET_ALL}:")
        for idx in range(300):
            score = raw_scores[0, idx] if raw_scores is not None else 0.0
            if score > 0.15:  # Only log detections above threshold
                box = raw_boxes[0, idx]
                label = int(np.argmax(raw_labels[0, idx])) if raw_labels is not None else -1
                logger.debug(f"{Fore.GREEN}  Detector: idx={idx}, Class={label}, Score={score:.2f}, BBox: x1={box[0]:.0f}, y1={box[1]:.0f}, x2={box[2]:.0f}, y2={box[3]:.0f} {Style.RESET_ALL}")

        # Match tracked objects with raw boxes using exact coordinate matching
        l_obj = frame_meta.obj_meta_list
        matched_detections = []

        while l_obj is not None:
            try:
                obj_meta = pyds.NvDsObjectMeta.cast(l_obj.data)
            except StopIteration:
                break

            logger.debug(f"{Fore.BLUE}  Tracker: Track ID={obj_meta.object_id}, Class ID={obj_meta.class_id}, Confidence={obj_meta.confidence:.2f}, BBox: left={obj_meta.rect_params.left:.0f}, top={obj_meta.rect_params.top:.0f}, width={obj_meta.rect_params.width:.0f}, height={obj_meta.rect_params.height:.0f}{Style.RESET_ALL}")

            # Object bbox in x1y1x2y2 format
            obj_x1 = obj_meta.rect_params.left
            obj_y1 = obj_meta.rect_params.top
            obj_x2 = obj_meta.rect_params.left + obj_meta.rect_params.width
            obj_y2 = obj_meta.rect_params.top + obj_meta.rect_params.height

            logger.debug(f"## bbox: {(obj_x1, obj_y1, obj_x2, obj_y2)}")


            # Find exact matching raw box
            matched_idx = -1

            for idx in range(300):
                raw_box = raw_boxes[0, idx]
                # Check if coordinates match exactly (or very close due to float precision)
                if (abs(raw_box[0] - obj_x1) < 0.01 and
                    abs(raw_box[1] - obj_y1) < 0.01 and
                    abs(raw_box[2] - obj_x2) < 0.01 and
                    abs(raw_box[3] - obj_y2) < 0.01):
                    matched_idx = idx
                    break

            logger.debug(f"###   Matched raw idx: {matched_idx}")
            # Only add if we found exact match
            if matched_idx != -1:
                soft_labels = raw_labels[0, matched_idx]

                matched_detections.append({
                    'track_id': obj_meta.object_id,
                    'class_id': obj_meta.class_id,
                    'confidence': obj_meta.confidence,
                    'bbox': [obj_x1, obj_y1, obj_x2, obj_y2],
                    'soft_labels': soft_labels.copy(),
                    'raw_idx': matched_idx
                })
                

            try:
                l_obj = l_obj.next
            except StopIteration:
                break

        # Log summary every 30 frames
        if frame_meta.frame_num % 30 == 0 and len(matched_detections) > 0:
            logger.info(f"\n=== Frame {frame_meta.frame_num} ===")
            logger.info(f"Matched {len(matched_detections)} detections")
            for i, det in enumerate(matched_detections[:3]):  # Log first 3
                logger.info(f"  Detection {i}:")
                logger.info(f"    Track ID: {det['track_id']}")
                logger.info(f"    Class ID: {det['class_id']}")
                logger.info(f"    Confidence: {det['confidence']:.3f}")
                logger.info(f"    Raw idx: {det['raw_idx']}")
                logger.info(f"    Soft labels (top 3): {np.sort(det['soft_labels'])[-3:]}")

        try:
            l_frame = l_frame.next
        except StopIteration:
            break

    return Gst.FlowReturn.OK



def review_nvdsframemeta(batch_meta: pyds.NvDsBatchMeta) -> None:
    """
    Review NvDsFrameMeta data in the batch meta for debugging.
    """
    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
        try:
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break

        logger.debug(f"Frame: pad_index={frame_meta.pad_index} batch_id={frame_meta.batch_id} "
              f"frame_num={frame_meta.frame_num} buf_pts={frame_meta.buf_pts} "
              f"ntp_timestamp={frame_meta.ntp_timestamp} source_id={frame_meta.source_id} "
              f"num_surfaces_per_frame={frame_meta.num_surfaces_per_frame}")
        logger.debug(f"  source_frame_width={frame_meta.source_frame_width} "
              f"source_frame_height={frame_meta.source_frame_height} "
              f"surface_type={frame_meta.surface_type} surface_index={frame_meta.surface_index}")
        logger.debug(f"  num_obj_meta={frame_meta.num_obj_meta} bInferDone={frame_meta.bInferDone}")

        # Iterate through object meta list
        l_obj = frame_meta.obj_meta_list
        while l_obj is not None:
            try:
                obj_meta = pyds.NvDsObjectMeta.cast(l_obj.data)
            except StopIteration:
                break

            color_map = {
                0: Fore.LIGHTBLUE_EX,
                1: Fore.GREEN,
                2: Fore.LIGHTCYAN_EX,
                3: Fore.MAGENTA,
                4: Fore.CYAN,
            }
            color = color_map.get(obj_meta.class_id, Fore.YELLOW)
            logger.debug(f"{color}   Object: class_id={obj_meta.class_id} object_id={obj_meta.object_id} "
                  f"confidence={obj_meta.confidence:.2f} tracker_confidence={obj_meta.tracker_confidence:.2f}{Style.RESET_ALL}")
            logger.debug(f"      rect: left={obj_meta.rect_params.left:.1f} top={obj_meta.rect_params.top:.1f} "
                  f"width={obj_meta.rect_params.width:.1f} height={obj_meta.rect_params.height:.1f}")
            logger.debug(f"      detector: left={obj_meta.detector_bbox_info.org_bbox_coords.left:.1f} "
                  f"top={obj_meta.detector_bbox_info.org_bbox_coords.top:.1f} "
                  f"width={obj_meta.detector_bbox_info.org_bbox_coords.width:.1f} "
                  f"height={obj_meta.detector_bbox_info.org_bbox_coords.height:.1f}")
            logger.debug(f"      tracker: left={obj_meta.tracker_bbox_info.org_bbox_coords.left:.1f} "
                  f"top={obj_meta.tracker_bbox_info.org_bbox_coords.top:.1f} "
                  f"width={obj_meta.tracker_bbox_info.org_bbox_coords.width:.1f} "
                  f"height={obj_meta.tracker_bbox_info.org_bbox_coords.height:.1f}")

            try:
                l_obj = l_obj.next
            except StopIteration:
                break

        try:
            l_frame = l_frame.next
        except StopIteration:
            break


def review_tracking_data(batch_meta : pyds.NvDsBatchMeta) -> None:
    """
    Review tracking data in the batch meta for debugging.
    """
    l_user=batch_meta.batch_user_meta_list #Retrieve glist of NvDsUserMeta objects from given NvDsBatchMeta object
    while l_user is not None:
        try:
            user_meta=pyds.NvDsUserMeta.cast(l_user.data)
        except StopIteration:
            break
        if(user_meta and user_meta.base_meta.meta_type==pyds.NvDsMetaType.NVDS_TRACKER_PAST_FRAME_META): #Make sure metatype is correct
            try:
                pPastFrameObjBatch = pyds.NvDsTargetMiscDataBatch.cast(user_meta.user_meta_data) #See NvDsTargetMiscDataBatch for details
            except StopIteration:
                break
            for trackobj in pyds.NvDsTargetMiscDataBatch.list(pPastFrameObjBatch): # NvDsTargetMiscDataStream objects
                # NvDsTargetMiscDataStream attributes
                logger.debug(f"streamId={trackobj.streamID} surfaceStreamID={trackobj.surfaceStreamID} numAllocated={trackobj.numAllocated} numFilled={trackobj.numFilled}")
                for pastframeobj in pyds.NvDsTargetMiscDataStream.list(trackobj): # NvDsFrameObjList objects
                    # NvDsTargetMiscDataObject attributes
                    logger.debug(f"numobj={pastframeobj.numObj} uniqueId= {pastframeobj.uniqueId} classId={pastframeobj.classId} objLabel={pastframeobj.objLabel}")
                    for objlist in pyds.NvDsTargetMiscDataObject.list(pastframeobj): # NvDsFrameObj objects
                        # NvDsTargetMiscDataFrame attributes
                        state = objlist.trackerState
                        logger.debug(f"{Fore.RED}trackletState:{state} {Style.RESET_ALL} frameNum: {objlist.frameNum} confidence:{objlist.confidence:.2f} age:{objlist.age} [left:{objlist.tBbox.left:.1f} width:{objlist.tBbox.width:.1f} top:{objlist.tBbox.top:.1f} height:{objlist.tBbox.height:.1f}]")


        try:
            l_user=l_user.next
        except StopIteration:
            break





def osd_sink_pad_buffer_probe(pad,info,u_data):
    frame_number=0
    #Intiallizing object counter with 0.
    obj_counter = {
        PGIE_CLASS_ID_VEHICLE:0,
        PGIE_CLASS_ID_PERSON:0,
        PGIE_CLASS_ID_BICYCLE:0,
        PGIE_CLASS_ID_ROADSIGN:0
    }
    num_rects=0
    gst_buffer = info.get_buffer()
    if not gst_buffer:
        logger.error("Unable to get GstBuffer ")
        return

    # Retrieve batch metadata from the gst_buffer
    # Note that pyds.gst_buffer_get_nvds_batch_meta() expects the
    # C address of gst_buffer as input, which is obtained with hash(gst_buffer)
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
        try:
            # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
            # The casting is done by pyds.NvDsFrameMeta.cast()
            # The casting also keeps ownership of the underlying memory
            # in the C code, so the Python garbage collector will leave
            # it alone.
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break

        frame_number=frame_meta.frame_num
        num_rects = frame_meta.num_obj_meta
        l_obj=frame_meta.obj_meta_list
        while l_obj is not None:
            try:
                # Casting l_obj.data to pyds.NvDsObjectMeta
                obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
            except StopIteration:
                break
            obj_counter[obj_meta.class_id] += 1
            try: 
                l_obj=l_obj.next
            except StopIteration:
                break

        # Acquiring a display meta object. The memory ownership remains in
        # the C code so downstream plugins can still access it. Otherwise
        # the garbage collector will claim it when this probe function exits.
        display_meta=pyds.nvds_acquire_display_meta_from_pool(batch_meta)
        display_meta.num_labels = 1
        py_nvosd_text_params = display_meta.text_params[0]

        # Setting display text to be shown on screen
        # Note that the pyds module allocates a buffer for the string, and the
        # memory will not be claimed by the garbage collector.
        # Reading the display_text field here will return the C address of the
        # allocated string. Use pyds.get_string() to get the string content.
        py_nvosd_text_params.display_text = f"{Fore.GREEN}Frame Number={frame_number} Number of Objects={num_rects} Vehicle_count={obj_counter[PGIE_CLASS_ID_VEHICLE]} Person_count={obj_counter[PGIE_CLASS_ID_PERSON]} {Style.RESET_ALL}"

        # Now set the offsets where the string should appear
        py_nvosd_text_params.x_offset = 10
        py_nvosd_text_params.y_offset = 12

        # Font , font-color and font-size
        py_nvosd_text_params.font_params.font_name = "Serif"
        py_nvosd_text_params.font_params.font_size = 10
        # set(red, green, blue, alpha); set to White
        py_nvosd_text_params.font_params.font_color.set(1.0, 1.0, 1.0, 1.0)

        # Text background color
        py_nvosd_text_params.set_bg_clr = 1
        # set(red, green, blue, alpha); set to Black
        py_nvosd_text_params.text_bg_clr.set(0.0, 0.0, 0.0, 1.0)
        # Using pyds.get_string() to get display_text as string
        logger.info(f"------ {pyds.get_string(py_nvosd_text_params.display_text)}")
        pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)
        try:
            l_frame=l_frame.next
        except StopIteration:
            break
    #past tracking meta data
    l_user=batch_meta.batch_user_meta_list
    while l_user is not None:
        try:
            # Note that l_user.data needs a cast to pyds.NvDsUserMeta
            # The casting is done by pyds.NvDsUserMeta.cast()
            # The casting also keeps ownership of the underlying memory
            # in the C code, so the Python garbage collector will leave
            # it alone
            user_meta=pyds.NvDsUserMeta.cast(l_user.data)
        except StopIteration:
            break
        if(user_meta and user_meta.base_meta.meta_type==pyds.NvDsMetaType.NVDS_TRACKER_PAST_FRAME_META):
            try:
                # Note that user_meta.user_meta_data needs a cast to pyds.NvDsTargetMiscDataBatch
                # The casting is done by pyds.NvDsTargetMiscDataBatch.cast()
                # The casting also keeps ownership of the underlying memory
                # in the C code, so the Python garbage collector will leave
                # it alone
                pPastDataBatch = pyds.NvDsTargetMiscDataBatch.cast(user_meta.user_meta_data)
            except StopIteration:
                break
            # for miscDataStream in pyds.NvDsTargetMiscDataBatch.list(pPastDataBatch):
                # logger.debug(f"streamId={miscDataStream.streamID}")
                # logger.debug(f"surfaceStreamID={miscDataStream.surfaceStreamID}")
                # for miscDataObj in pyds.NvDsTargetMiscDataStream.list(miscDataStream):
                #     logger.debug(f"numobj={miscDataObj.numObj}")
                #     logger.debug(f"uniqueId={miscDataObj.uniqueId}")
                #     logger.debug(f"classId={miscDataObj.classId}")
                #     logger.debug(f"objLabel={miscDataObj.objLabel}")
                #     for miscDataFrame in pyds.NvDsTargetMiscDataObject.list(miscDataObj):
                #         logger.debug(f"frameNum: {miscDataFrame.frameNum}")
                #         logger.debug(f"tBbox.left: {miscDataFrame.tBbox.left}")
                #         logger.debug(f"tBbox.width: {miscDataFrame.tBbox.width}")
                #         logger.debug(f"tBbox.top: {miscDataFrame.tBbox.top}")
                #         logger.debug(f"tBbox.right: {miscDataFrame.tBbox.height}")
                #         logger.debug(f"confidence: {miscDataFrame.confidence}")
                #         logger.debug(f"age: {miscDataFrame.age}")
        try:
            l_user=l_user.next
        except StopIteration:
            break
    return Gst.PadProbeReturn.OK	

def main(args):
    # Check input arguments
    if(len(args)<2):
        sys.stderr.write("usage: %s <h264_elementary_stream>\n" % args[0])
        sys.exit(1)

    platform_info = PlatformInfo()
    # Standard GStreamer initialization

    Gst.init(None)

    # Create gstreamer elements
    # Create Pipeline element that will form a connection of other elements
    logger.info("Creating Pipeline")
    pipeline = Gst.Pipeline()

    if not pipeline:
        logger.error("Unable to create Pipeline")

    # Source element for reading from the file
    logger.info("Creating Source")
    source = Gst.ElementFactory.make("filesrc", "file-source")
    if not source:
        logger.error("Unable to create Source")

    # Since the data format in the input file is elementary h264 stream,
    # we need a h264parser
    logger.info("Creating H264Parser")
    h264parser = Gst.ElementFactory.make("h264parse", "h264-parser")
    if not h264parser:
        logger.error("Unable to create h264 parser")

    # Use nvdec_h264 for hardware accelerated decode on GPU
    logger.info("Creating Decoder")
    decoder = Gst.ElementFactory.make("nvv4l2decoder", "nvv4l2-decoder")
    if not decoder:
        logger.error("Unable to create Nvv4l2 Decoder")

    # Create nvstreammux instance to form batches from one or more sources.
    streammux = Gst.ElementFactory.make("nvstreammux", "Stream-muxer")
    if not streammux:
        logger.error("Unable to create NvStreamMux")

    # Use nvinfer to run inferencing on decoder's output,
    # behaviour of inferencing is set through config file
    pgie = Gst.ElementFactory.make("nvinfer", "primary-inference")
    if not pgie:
        logger.error("Unable to create pgie")

    tracker = Gst.ElementFactory.make("nvtracker", "tracker")
    if not tracker:
        logger.error("Unable to create tracker")

    sgie1 = Gst.ElementFactory.make("nvinfer", "secondary1-nvinference-engine")
    if not sgie1:
        logger.error("Unable to make sgie1")


    sgie2 = Gst.ElementFactory.make("nvinfer", "secondary2-nvinference-engine")
    if not sgie2:
        logger.error("Unable to make sgie2")

    nvvidconv = Gst.ElementFactory.make("nvvideoconvert", "convertor")
    if not nvvidconv:
        logger.error("Unable to create nvvidconv")

    # Create OSD to draw on the converted RGBA buffer
    nvosd = Gst.ElementFactory.make("nvdsosd", "onscreendisplay")

    if not nvosd:
        logger.error("Unable to create nvosd")

    # Create tee to split pipeline for display and appsink
    logger.info("Creating Tee")
    tee = Gst.ElementFactory.make("tee", "tee")
    if not tee:
        logger.error("Unable to create tee")

    # Create queues for each tee branch
    logger.info("Creating Queues")
    queue_display = Gst.ElementFactory.make("queue", "queue-display")
    if not queue_display:
        logger.error("Unable to create queue for display")

    queue_appsink = Gst.ElementFactory.make("queue", "queue-appsink")
    if not queue_appsink:
        logger.error("Unable to create queue for appsink")

    # Finally render the osd output
    if platform_info.is_integrated_gpu():
        logger.info("Creating nv3dsink")
        sink = Gst.ElementFactory.make("nv3dsink", "nv3d-sink")
        if not sink:
            logger.error("Unable to create nv3dsink")
    else:
        if platform_info.is_platform_aarch64():
            logger.info("Creating nv3dsink")
            sink = Gst.ElementFactory.make("nv3dsink", "nv3d-sink")
        else:
            logger.info("Creating EGLSink")
            sink = Gst.ElementFactory.make("nveglglessink", "nvvideo-renderer")
        if not sink:
            logger.error("Unable to create egl sink")

    # Create appsink for callback processing
    logger.info("Creating AppSink")
    appsink = Gst.ElementFactory.make("appsink", "appsink")
    if not appsink:
        logger.error("Unable to create appsink")

    logger.info(f"Playing file {args[1]}")
    source.set_property('location', args[1])
    streammux.set_property('width', 1920)
    streammux.set_property('height', 1080)
    streammux.set_property('batch-size', 1)
    streammux.set_property('batched-push-timeout', MUXER_BATCH_TIMEOUT_USEC)

    #Set properties of pgie and sgie
    pgie.set_property('config-file-path', "dstest2_pgie_config.txt")
    sgie1.set_property('config-file-path', "dstest2_sgie1_config.txt")
    sgie2.set_property('config-file-path', "dstest2_sgie2_config.txt")

    #Set properties of tracker
    config = configparser.ConfigParser()
    config.read('dstest2_tracker_config.txt')
    config.sections()

    for key in config['tracker']:
        if key == 'tracker-width' :
            tracker_width = config.getint('tracker', key)
            tracker.set_property('tracker-width', tracker_width)
        if key == 'tracker-height' :
            tracker_height = config.getint('tracker', key)
            tracker.set_property('tracker-height', tracker_height)
        if key == 'gpu-id' :
            tracker_gpu_id = config.getint('tracker', key)
            tracker.set_property('gpu_id', tracker_gpu_id)
        if key == 'll-lib-file' :
            tracker_ll_lib_file = config.get('tracker', key)
            tracker.set_property('ll-lib-file', tracker_ll_lib_file)
        if key == 'll-config-file' :
            tracker_ll_config_file = config.get('tracker', key)
            tracker.set_property('ll-config-file', tracker_ll_config_file)

    logger.info("Adding elements to Pipeline")
    pipeline.add(source)
    pipeline.add(h264parser)
    pipeline.add(decoder)
    pipeline.add(streammux)
    pipeline.add(pgie)
    pipeline.add(tracker)
    pipeline.add(sgie1)
    pipeline.add(sgie2)
    pipeline.add(nvvidconv)
    pipeline.add(nvosd)
    pipeline.add(tee)
    pipeline.add(queue_display)
    pipeline.add(queue_appsink)
    pipeline.add(sink)
    pipeline.add(appsink)

    # we link the elements together
    # file-source -> h264-parser -> nvh264-decoder ->
    # nvinfer -> nvvidconv -> nvosd -> video-renderer
    logger.info("Linking elements in the Pipeline")
    source.link(h264parser)
    h264parser.link(decoder)

    sinkpad = streammux.request_pad_simple("sink_0")
    if not sinkpad:
        logger.error("Unable to get the sink pad of streammux")
    srcpad = decoder.get_static_pad("src")
    if not srcpad:
        logger.error("Unable to get source pad of decoder")
    srcpad.link(sinkpad)
    streammux.link(pgie)
    pgie.link(tracker)
    tracker.link(sgie1)
    sgie1.link(sgie2)
    sgie2.link(nvvidconv)
    nvvidconv.link(nvosd)
    nvosd.link(tee)

    # Link tee to display branch
    tee_display_pad = tee.request_pad_simple("src_%u")
    if not tee_display_pad:
        logger.error("Unable to get tee display pad")
    queue_display_sinkpad = queue_display.get_static_pad("sink")
    tee_display_pad.link(queue_display_sinkpad)
    queue_display.link(sink)

    # Link tee to appsink branch
    tee_appsink_pad = tee.request_pad_simple("src_%u")
    if not tee_appsink_pad:
        logger.error("Unable to get tee appsink pad")
    queue_appsink_sinkpad = queue_appsink.get_static_pad("sink")
    tee_appsink_pad.link(queue_appsink_sinkpad)
    queue_appsink.link(appsink)

    # Configure appsink
    appsink.set_property("emit-signals", True)
    appsink.set_property("sync", False)
    appsink.connect("new-sample", appsink_callback)


    # create and event loop and feed gstreamer bus mesages to it
    loop = GLib.MainLoop()

    bus = pipeline.get_bus()
    bus.add_signal_watch()
    bus.connect ("message", bus_call, loop)

    # Lets add probe to get informed of the meta data generated, we add probe to
    # the sink pad of the osd element, since by that time, the buffer would have
    # had got all the metadata.
    osdsinkpad = nvosd.get_static_pad("sink")
    if not osdsinkpad:
        logger.error("Unable to get sink pad of nvosd")
    osdsinkpad.add_probe(Gst.PadProbeType.BUFFER, osd_sink_pad_buffer_probe, 0)


    logger.info("Starting pipeline")
    
    # start play back and listed to events
    pipeline.set_state(Gst.State.PLAYING)
    try:
      loop.run()
    except:
      pass

    # cleanup
    pipeline.set_state(Gst.State.NULL)

if __name__ == '__main__':
    sys.exit(main(sys.argv))

%YAML:1.0
################################################################################
# SPDX-FileCopyrightText: Copyright (c) 2019-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################

BaseConfig:
  minDetectorConfidence: 0   # If the confidence of a detector bbox is lower than this, then it won't be considered for tracking

TargetManagement:
  enableBboxUnClipping: 1   # In case the bbox is likely to be clipped by image border, unclip bbox
  maxTargetsPerStream: 150  # Max number of targets to track per stream. Recommended to set >10. Note: this value should account for the targets being tracked in shadow mode as well. Max value depends on the GPU memory capacity

  # [Creation & Termination Policy]
  minIouDiff4NewTarget: 0.5   # If the IOU between the newly detected object and any of the existing targets is higher than this threshold, this newly detected object will be discarded.
  minTrackerConfidence: 0.02   # If the confidence of an object tracker is lower than this on the fly, then it will be tracked in shadow mode. Valid Range: [0.0, 1.0]
  probationAge: 3 # If the target's age exceeds this, the target will be considered to be valid.
  maxShadowTrackingAge: 30  # Max length of shadow tracking. If the shadowTrackingAge exceeds this limit, the tracker will be terminated.
  earlyTerminationAge: 1   # If the shadowTrackingAge reaches this threshold while in TENTATIVE period, the target will be terminated prematurely.

TrajectoryManagement:
  useUniqueID: 0   # Use 64-bit long Unique ID when assignining tracker ID. Default is [true]

DataAssociator:
  dataAssociatorType: 0 # the type of data associator among { DEFAULT= 0 }
  associationMatcherType: 0 # the type of matching algorithm among { GREEDY=0, GLOBAL=1 }
  checkClassMatch: 1  # If checked, only the same-class objects are associated with each other. Default: true

  # [Association Metric: Thresholds for valid candidates]
  minMatchingScore4Overall: 0.0   # Min total score
  minMatchingScore4SizeSimilarity: 0.6  # Min bbox size similarity score
  minMatchingScore4Iou: 0.0       # Min IOU score
  minMatchingScore4VisualSimilarity: 0.7  # Min visual similarity score

  # [Association Metric: Weights]
  matchingScoreWeight4VisualSimilarity: 0.6  # Weight for the visual similarity (in terms of correlation response ratio)
  matchingScoreWeight4SizeSimilarity: 0.0    # Weight for the Size-similarity score
  matchingScoreWeight4Iou: 0.4   # Weight for the IOU score

StateEstimator:
  stateEstimatorType: 1  # the type of state estimator among { DUMMY=0, SIMPLE=1, REGULAR=2 }

  # [Dynamics Modeling]
  processNoiseVar4Loc: 2.0    # Process noise variance for bbox center
  processNoiseVar4Size: 1.0   # Process noise variance for bbox size
  processNoiseVar4Vel: 0.1    # Process noise variance for velocity
  measurementNoiseVar4Detector: 4.0    # Measurement noise variance for detector's detection
  measurementNoiseVar4Tracker: 16.0    # Measurement noise variance for tracker's localization

VisualTracker:
  visualTrackerType: 1 # the type of visual tracker among { DUMMY=0, NvDCF=1 }

  # [NvDCF: Feature Extraction]
  useColorNames: 1     # Use ColorNames feature
  useHog: 0            # Use Histogram-of-Oriented-Gradient (HOG) feature
  featureImgSizeLevel: 2  # Size of a feature image. Valid range: {1, 2, 3, 4, 5}, from the smallest to the largest
  featureFocusOffsetFactor_y: -0.2 # The offset for the center of hanning window relative to the feature height. The center of hanning window would move by (featureFocusOffsetFactor_y*featureMatSize.height) in vertical direction

  # [NvDCF: Correlation Filter]
  filterLr: 0.075 # learning rate for DCF filter in exponential moving average. Valid Range: [0.0, 1.0]
  filterChannelWeightsLr: 0.1 # learning rate for the channel weights among feature channels. Valid Range: [0.0, 1.0]
  gaussianSigma: 0.75 # Standard deviation for Gaussian for desired response when creating DCF filter [pixels]

Output:

2026-02-05 23:27:35,657 - INFO - ------ Frame Number=316 Number of Objects=19 Vehicle_count=15 Person_count=4 
2026-02-05 23:27:35,657 - DEBUG - 
--- Review Data ---
2026-02-05 23:27:35,657 - DEBUG - streamId=0 surfaceStreamID=0 numAllocated=150 numFilled=4
2026-02-05 23:27:35,657 - DEBUG - numobj=3 uniqueId= 181 classId=0 objLabel=car
2026-02-05 23:27:35,657 - DEBUG - trackletState:TRACKER_STATE.TENTATIVE  frameNum: 313 confidence:0.50 age:1 [left:1182.5 width:75.8 top:479.4 height:20.8]
2026-02-05 23:27:35,657 - DEBUG - trackletState:TRACKER_STATE.TENTATIVE  frameNum: 314 confidence:0.90 age:2 [left:1179.0 width:76.6 top:483.5 height:20.1]
2026-02-05 23:27:35,657 - DEBUG - trackletState:TRACKER_STATE.TENTATIVE  frameNum: 315 confidence:0.76 age:3 [left:1190.4 width:62.6 top:485.8 height:24.7]
2026-02-05 23:27:35,657 - DEBUG - numobj=1 uniqueId= 162 classId=0 objLabel=car
2026-02-05 23:27:35,657 - DEBUG - trackletState:TRACKER_STATE.INACTIVE  frameNum: 315 confidence:0.55 age:50 [left:1182.1 width:89.8 top:484.4 height:32.9]
2026-02-05 23:27:35,657 - DEBUG - numobj=1 uniqueId= 167 classId=0 objLabel=car
2026-02-05 23:27:35,657 - DEBUG - trackletState:TRACKER_STATE.INACTIVE  frameNum: 315 confidence:0.47 age:39 [left:647.9 width:23.6 top:475.3 height:15.3]
2026-02-05 23:27:35,657 - DEBUG - numobj=1 uniqueId= 129 classId=0 objLabel=car
2026-02-05 23:27:35,658 - DEBUG - trackletState:TRACKER_STATE.INACTIVE  frameNum: 315 confidence:0.49 age:112 [left:552.3 width:19.4 top:473.0 height:24.3]
2026-02-05 23:27:35,658 - DEBUG - 
--- NVDS Frame Meta ---
2026-02-05 23:27:35,658 - DEBUG - Frame: pad_index=0 batch_id=0 frame_num=316 buf_pts=10533333228 ntp_timestamp=1770334055652587000 source_id=0 num_surfaces_per_frame=1
2026-02-05 23:27:35,658 - DEBUG -   source_frame_width=1280 source_frame_height=720 surface_type=0 surface_index=0
2026-02-05 23:27:35,658 - DEBUG -   num_obj_meta=19 bInferDone=1
2026-02-05 23:27:35,658 - DEBUG -    Object: class_id=2 object_id=122 confidence=0.44 tracker_confidence=0.94
2026-02-05 23:27:35,658 - DEBUG -       rect: left=82.9 top=474.4 width=123.1 height=297.0
2026-02-05 23:27:35,658 - DEBUG -       detector: left=82.9 top=474.4 width=123.1 height=297.0
2026-02-05 23:27:35,658 - DEBUG -       tracker: left=82.9 top=474.4 width=123.1 height=297.0
2026-02-05 23:27:35,658 - DEBUG -    Object: class_id=2 object_id=165 confidence=0.67 tracker_confidence=0.75
2026-02-05 23:27:35,658 - DEBUG -       rect: left=451.6 top=490.1 width=21.0 height=71.7
2026-02-05 23:27:35,658 - DEBUG -       detector: left=451.6 top=490.1 width=21.0 height=71.7
2026-02-05 23:27:35,658 - DEBUG -       tracker: left=451.6 top=490.1 width=21.0 height=71.7

Thanks

[kesong]

We need test video to reproduce the issue and tuning tracker parameters. We also have tuning guide in here if you can’t share the video: Troubleshooting — DeepStream documentation

[tracker.debugging]

Hello Kesong,

Using the sample video for the example. The tracker configuration: config_tracker_NvDCF_perf.yml is pasted on the message above.

python3 deepstream_test_2.py /workspace/deepstream-8.0/samples/streams/sample_720p.h264 

Thanks,

[kesong]

We prefer use config_tracker_NvDCF_accuracy.yml during check flickering BBox issue. Are you see BBox flickering in deepstream_test_2.py with config_tracker_NvDCF_accuracy.yml?

[tracker.debugging]

Yes, it has the same behavior as commented. What is the the issue happening? It seems like the tracker is not including the detections even when the confidence is high enough.

Is there an explanation for this? Am I missing something?

Thanks

[kesong]

Can you share the video to show the flickering BBox issue? Better to use service maker for your application: Service Maker for Python Developers — DeepStream documentation