• Hardware Platform (Jetson / GPU) = Jetson Xavier Nx
• DeepStream Version = 6.0.1
• JetPack Version (valid for Jetson only) = 35.1.0
• TensorRT Version = 8.4.1.5
• CUDA Version (valid for GPU only) = 11.4
• Issue Type( questions, new requirements, bugs) = Question
Hi, I am using the python binding of Deepstream to perform detection and tracking on a video. For detection, I am using the YOLOv5 model and for tracking, I am using the Deepstream plugin.
(I am using this repo NVIDIA DeepStream SDK 6.1 / 6.0.1 / 6.0 configuration for YOLO-v5 & YOLO-v7 models · GitHub to perform detection a nd tracking using Deepstream pipeline)
I can observe the seamless detection of vehicles with corresponding tracking ID on the OSD window as shown below.
I have two targets:
1. Get the bounding box coordinates of each detected vehicle.
2. Get the tracking ID of the corresponding bounding box coordinates.
Example: {id1: bbox1, id2: bbox2, . . .}
To achieve target 1, I am using pyds.NvDsObjectMeta.cast(l_obj.data).rect_params, it gives me the bounding box coordinates of each detection.
To verify whether the coordinate values that I get from the pyds.NvDsObjectMeta.cast(l_obj.data).rect_params is correct or not, I draw the bounding box on the same frame using the CV2 function. The result is shown below. Here, the red box is originally drawn on the OSD window and the pink boxes are coordinates returned by pyds.NvDsObjectMeta.cast(l_obj.data).rect_params, drawn by me using CV2.
As we can observe, the coordinate values we receive are correct.
Ques 1. Is there a way to get the tracking ID also corresponding to each bounding box coordinates using the pyds.NvDsObjectMeta.cast(l_obj.data).rect_params ?
To achieve target 2, I am using pyds.NvDsPastFrameObjStream.list(trackobj). This function does return tracking IDs of all detected boxes in the frame but it doesn’t return tracking ID to a specific bounding box.
Ques 2. How do I map the coordinates received from pyds.NvDsObjectMeta.cast(l_obj.data).rect_params, and tracking ID received from pyds.NvDsPastFrameObjStream.list(trackobj)?
Also, when I used pyds.NvDsPastFrameObjList.list(pastframeobj) to map the bounding box coordinates with tracking ID, the bounding box coordinate values do not match the real bounding boxes in the image. I verified this by drawing these coordinates on the frame using CV2. The resulting image is shown below. Here, the red box is originally drawn on the OSD window and the blue boxes are coordinates returned by pyds.NvDsPastFrameObjList.list(pastframeobj), drawn by me using CV2.
In both 1 and 2, I am using deepstream_imagedata-multistream.py sample app for drawing the bounding boxes on frame.
To reproduce 1, use the following snippet inside the osd_sink_pad_buffer_probe definition in one of the python sample apps of Deepstream:
while l_obj is not None:
try:
# Casting l_obj.data to pyds.NvDsObjectMeta
obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
''' Added here '''
class_id=obj_meta.class_id
obj_id=obj_meta.object_id
obj_conf=obj_meta.confidence
rect_params=obj_meta.rect_params
top=int(rect_params.top)
left=int(rect_params.left)
width=int(rect_params.width)
height=int(rect_params.height)
# to draw on frame
n_frame = pyds.get_nvds_buf_surface(hash(gst_buffer), frame_meta.batch_id)
n_frame = draw_bounding_boxes(n_frame, obj_meta, obj_meta.confidence)
. . .
To reproduce 2, use the following snippet, use the following snippet inside the osd_sink_pad_buffer_probe definition in one of the python sample apps of Deepstream:
#past traking meta data
past_tracking_meta[0]=1
if(past_tracking_meta[0]==1):
l_user=batch_meta.batch_user_meta_list
# print("l_user ============= ", l_user)
while l_user is not None:
try:
# Note that l_user.data needs a cast to pyds.NvDsUserMeta
# The casting is done by pyds.NvDsUserMeta.cast()
# The casting also keeps ownership of the underlying memory
# in the C code, so the Python garbage collector will leave
# it alone
user_meta=pyds.NvDsUserMeta.cast(l_user.data)
except StopIteration:
break
if(user_meta and user_meta.base_meta.meta_type==pyds.NvDsMetaType.NVDS_TRACKER_PAST_FRAME_META):
try:
# Note that user_meta.user_meta_data needs a cast to pyds.NvDsPastFrameObjBatch
# The casting is done by pyds.NvDsPastFrameObjBatch.cast()
# The casting also keeps ownership of the underlying memory
# in the C code, so the Python garbage collector will leave
# it alone
pPastFrameObjBatch = pyds.NvDsPastFrameObjBatch.cast(user_meta.user_meta_data)
except StopIteration:
break
for trackobj in pyds.NvDsPastFrameObjBatch.list(pPastFrameObjBatch):
for pastframeobj in pyds.NvDsPastFrameObjStream.list(trackobj):
print("uniqueId=",pastframeobj.uniqueId) # TO GET TRACKER IDS IN A FRAME
print("classId=",pastframeobj.classId) # TO GET CLASS ID FOR THE CORRESPONDING TRACKER ID
print("objLabel=",pastframeobj.objLabel) # TO GET OBJECT LABEL FOR THE CORRESPONDING TRACKER ID
for objlist in pyds.NvDsPastFrameObjList.list(pastframeobj):
''' HERE I THOUGHT I WILL GET THE BBOX COORDINATES FOR EACH TRACKER ID
BUT THE FOLLOWING CORDINATES GIVE ME THE WRONG VALUE AS I SHOWED IN THE THIRD
DIAGRAM '''
print('tBbox.left:', objlist.tBbox.left)
print('tBbox.width:', objlist.tBbox.width)
print('tBbox.top:', objlist.tBbox.top)
print('tBbox.right:', objlist.tBbox.height)
n_frame = pyds.get_nvds_buf_surface(hash(gst_buffer), frame_meta.batch_id)
n_frame = draw_bounding_boxes(n_frame, objlist, pastframeobj, objlist.confidence)
try:
l_user=l_user.next
except StopIteration:
break
. . .
Please put forth your suggestions as to how I can get bounding box coordinates corresponding to a specific tracker ID.
I’d be grateful for your inputs and suggestions.
Thank you.