Get tracker ID and corresponding bounding box using Deepstream Python

• Hardware Platform (Jetson / GPU) = Jetson Xavier Nx
• DeepStream Version = 6.0.1
• JetPack Version (valid for Jetson only) = 35.1.0
• TensorRT Version = 8.4.1.5
• CUDA Version (valid for GPU only) = 11.4
• Issue Type( questions, new requirements, bugs) = Question

Hi, I am using the python binding of Deepstream to perform detection and tracking on a video. For detection, I am using the YOLOv5 model and for tracking, I am using the Deepstream plugin.
(I am using this repo NVIDIA DeepStream SDK 6.1 / 6.0.1 / 6.0 configuration for YOLO-v5 & YOLO-v7 models · GitHub to perform detection a nd tracking using Deepstream pipeline)
I can observe the seamless detection of vehicles with corresponding tracking ID on the OSD window as shown below.

I have two targets:
1. Get the bounding box coordinates of each detected vehicle.
2. Get the tracking ID of the corresponding bounding box coordinates.

Example: {id1: bbox1, id2: bbox2, . . .}

To achieve target 1, I am using pyds.NvDsObjectMeta.cast(l_obj.data).rect_params, it gives me the bounding box coordinates of each detection.
To verify whether the coordinate values that I get from the pyds.NvDsObjectMeta.cast(l_obj.data).rect_params is correct or not, I draw the bounding box on the same frame using the CV2 function. The result is shown below. Here, the red box is originally drawn on the OSD window and the pink boxes are coordinates returned by pyds.NvDsObjectMeta.cast(l_obj.data).rect_params, drawn by me using CV2.

As we can observe, the coordinate values we receive are correct.
Ques 1. Is there a way to get the tracking ID also corresponding to each bounding box coordinates using the pyds.NvDsObjectMeta.cast(l_obj.data).rect_params ?

To achieve target 2, I am using pyds.NvDsPastFrameObjStream.list(trackobj). This function does return tracking IDs of all detected boxes in the frame but it doesn’t return tracking ID to a specific bounding box.

Ques 2. How do I map the coordinates received from pyds.NvDsObjectMeta.cast(l_obj.data).rect_params, and tracking ID received from pyds.NvDsPastFrameObjStream.list(trackobj)?

Also, when I used pyds.NvDsPastFrameObjList.list(pastframeobj) to map the bounding box coordinates with tracking ID, the bounding box coordinate values do not match the real bounding boxes in the image. I verified this by drawing these coordinates on the frame using CV2. The resulting image is shown below. Here, the red box is originally drawn on the OSD window and the blue boxes are coordinates returned by pyds.NvDsPastFrameObjList.list(pastframeobj), drawn by me using CV2.

In both 1 and 2, I am using deepstream_imagedata-multistream.py sample app for drawing the bounding boxes on frame.

To reproduce 1, use the following snippet inside the osd_sink_pad_buffer_probe definition in one of the python sample apps of Deepstream:

while l_obj is not None:
    try: 
            # Casting l_obj.data to pyds.NvDsObjectMeta
            obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
            
            ''' Added here '''
            class_id=obj_meta.class_id
            obj_id=obj_meta.object_id
            obj_conf=obj_meta.confidence
            rect_params=obj_meta.rect_params
            top=int(rect_params.top)
            left=int(rect_params.left)
            width=int(rect_params.width)
            height=int(rect_params.height)
          
            # to draw on frame
            n_frame = pyds.get_nvds_buf_surface(hash(gst_buffer), frame_meta.batch_id)
            n_frame = draw_bounding_boxes(n_frame, obj_meta, obj_meta.confidence)
. . .

To reproduce 2, use the following snippet, use the following snippet inside the osd_sink_pad_buffer_probe definition in one of the python sample apps of Deepstream:

#past traking meta data
past_tracking_meta[0]=1
if(past_tracking_meta[0]==1):
    l_user=batch_meta.batch_user_meta_list
    # print("l_user ============= ", l_user)
    while l_user is not None:
        try:
            # Note that l_user.data needs a cast to pyds.NvDsUserMeta
            # The casting is done by pyds.NvDsUserMeta.cast()
            # The casting also keeps ownership of the underlying memory
            # in the C code, so the Python garbage collector will leave
            # it alone
            user_meta=pyds.NvDsUserMeta.cast(l_user.data)
        except StopIteration:
            break
        if(user_meta and user_meta.base_meta.meta_type==pyds.NvDsMetaType.NVDS_TRACKER_PAST_FRAME_META):
            try:
                # Note that user_meta.user_meta_data needs a cast to pyds.NvDsPastFrameObjBatch
                # The casting is done by pyds.NvDsPastFrameObjBatch.cast()
                # The casting also keeps ownership of the underlying memory
                # in the C code, so the Python garbage collector will leave
                # it alone
                pPastFrameObjBatch = pyds.NvDsPastFrameObjBatch.cast(user_meta.user_meta_data)
            except StopIteration:
                break
            for trackobj in pyds.NvDsPastFrameObjBatch.list(pPastFrameObjBatch):
                for pastframeobj in pyds.NvDsPastFrameObjStream.list(trackobj):
                    print("uniqueId=",pastframeobj.uniqueId) # TO GET TRACKER IDS IN A FRAME
                    print("classId=",pastframeobj.classId)  # TO GET CLASS ID  FOR THE CORRESPONDING TRACKER ID 
                    print("objLabel=",pastframeobj.objLabel) # TO GET OBJECT LABEL FOR THE CORRESPONDING  TRACKER ID                                
                    for objlist in pyds.NvDsPastFrameObjList.list(pastframeobj):                                   
                    ''' HERE I THOUGHT I WILL GET THE BBOX COORDINATES FOR EACH TRACKER ID
                        BUT THE FOLLOWING CORDINATES GIVE ME THE WRONG VALUE AS I SHOWED IN THE THIRD 
                        DIAGRAM  '''
                        print('tBbox.left:', objlist.tBbox.left)
                        print('tBbox.width:', objlist.tBbox.width)
                        print('tBbox.top:', objlist.tBbox.top)
                        print('tBbox.right:', objlist.tBbox.height)

                        n_frame = pyds.get_nvds_buf_surface(hash(gst_buffer), frame_meta.batch_id)
                        n_frame = draw_bounding_boxes(n_frame, objlist, pastframeobj, objlist.confidence)
        try:
            l_user=l_user.next
        except StopIteration:
            break
. . . 

Please put forth your suggestions as to how I can get bounding box coordinates corresponding to a specific tracker ID.
I’d be grateful for your inputs and suggestions.

Thank you.

object_id in pyds.NvDsObjectMeta is the track ID when nvtracker is enabled.
https://docs.nvidia.com/metropolis/deepstream/dev-guide/python-api/PYTHON_API/NvDsMeta/NvDsObjectMeta.html

This is not a proper way.

They are for past frames but not current frame.

1 Like

Thank you very much @Fiona.Chen . I can now access the tracker ID using the obj_id parameter.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.