Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU): GeForce RTX 3090
• DeepStream Version: 6.1
• TensorRT Version: 8.2
• NVIDIA GPU Driver Version (valid for GPU only): 510
• Issue Type( questions, new requirements, bugs): question
I understand that the tiler merges all frames in a batched buffer into a single frame, and the doc says the tiler combines “NvDsBatchMeta
and NvDsFrameMeta
in row-major order (starting from source 0, left to right across the top row, then across the next row)” so object index change before and after the tiler plugin is expected.
In my pipeline I have 2 sources, I noticed that the order of the objects in frame_meta->obj_meta_list
is reversed after the tiler plugin. Please see example bellow.
Where:
- batch = current batched buffer
- batch_id =
frame_meta->batch_id
- src_id =
frame_meta->source_id
- obj_index = object index in
frame_meta->obj_meta_list
of current frame inbatch_meta->frame_meta_list
- x =
obj_meta->detector_bbox_info.org_bbox_coords.top
- y =
obj_meta->detector_bbox_info.org_bbox_coords.left
Case 1: frame_meta->batch_id
match frame_meta->source_id
----------
before tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 0] x = 453.75, y = 312.69
before tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 1] x = 277.85, y = 379.27
before tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 2] x = 169.19, y = 201.79
before tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 3] x = 2.62, y = 693.85
before tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 4] x = 216.08, y = 869.71
before tiler [batch = 19, batch_id = 1, src_id = 1, obj_index = 0] x = 0.00, y = 378.50
before tiler [batch = 19, batch_id = 1, src_id = 1, obj_index = 1] x = 156.22, y = 890.23
before tiler [batch = 19, batch_id = 1, src_id = 1, obj_index = 2] x = 344.46, y = 331.87
before tiler [batch = 19, batch_id = 1, src_id = 1, obj_index = 3] x = 36.58, y = 863.97
----------
----------
after tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 0], x = 36.58, y = 863.97
after tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 1], x = 344.46, y = 331.87
after tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 2], x = 156.22, y = 890.23
after tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 3], x = 0.00, y = 378.50
after tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 4], x = 216.08, y = 869.71
after tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 5], x = 2.62, y = 693.85
after tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 6], x = 169.19, y = 201.79
after tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 7], x = 277.85, y = 379.27
after tiler [batch = 19, batch_id = 0, src_id = 0, obj_index = 8], x = 453.75, y = 312.69
----------
Case 2: frame_meta->batch_id
does not match frame_meta->source_id
----------
before tiler [batch = 0, batch_id = 0, src_id = 1, obj_index = 0] x = 0.00, y = 425.23
before tiler [batch = 0, batch_id = 0, src_id = 1, obj_index = 1] x = 203.79, y = 936.85
before tiler [batch = 0, batch_id = 0, src_id = 1, obj_index = 2] x = 38.64, y = 865.28
before tiler [batch = 0, batch_id = 1, src_id = 0, obj_index = 0] x = 152.98, y = 68.65
before tiler [batch = 0, batch_id = 1, src_id = 0, obj_index = 1] x = 202.32, y = 423.64
before tiler [batch = 0, batch_id = 1, src_id = 0, obj_index = 2] x = 2.01, y = 688.54
before tiler [batch = 0, batch_id = 1, src_id = 0, obj_index = 3] x = 211.92, y = 889.19
----------
----------
after tiler [batch = 0, batch_id = 0, src_id = 0, obj_index = 0], x = 211.92, y = 889.19
after tiler [batch = 0, batch_id = 0, src_id = 0, obj_index = 1], x = 2.01, y = 688.54
after tiler [batch = 0, batch_id = 0, src_id = 0, obj_index = 2], x = 202.32, y = 423.64
after tiler [batch = 0, batch_id = 0, src_id = 0, obj_index = 3], x = 152.98, y = 68.65
after tiler [batch = 0, batch_id = 0, src_id = 0, obj_index = 4], x = 38.64, y = 865.28
after tiler [batch = 0, batch_id = 0, src_id = 0, obj_index = 5], x = 203.79, y = 936.85
after tiler [batch = 0, batch_id = 0, src_id = 0, obj_index = 6], x = 0.00, y = 425.23
----------
Focus on batch_id, src_id, obj_index before and after the tiler:
Case 1:
(0, 0, 0) → (0, 0, 8)
(0, 0, 1) → (0, 0, 7)
(0, 0, 2) → (0, 0, 6)
(0, 0, 3) → (0, 0, 5)
(0, 0, 4) → (0, 0, 4)
(1, 1, 0) → (0, 0, 3)
(1, 1, 1) → (0, 0, 2)
(1, 1, 2) → (0, 0, 1)
(1, 1, 3) → (0, 0, 0)
Conceptually, the order of object index is reverse based on batch id (instead of source id) from last to first batch (instead of first to last):
[0]->[1]->[2]->[3]->[4]->[0]->[1]->[2]->[3]
[8]<-[7]<-[6]<-[5]<-[4]<-[3]<-[2]<-[1]<-[0]
Case 2:
(0, 1, 0) → (0, 0, 6)
(0, 1, 1) → (0, 0, 5)
(0, 1, 2) → (0, 0, 4)
(1, 0, 0) → (0, 0, 3)
(1, 0, 1) → (0, 0, 2)
(1, 0, 2) → (0, 0, 1)
(1, 0, 3) → (0, 0, 0)
Same as case 1, conceptually, the order of object index is reverse based on batch id (instead of source id) from last to first batch (instead of first to last):
[0]->[1]->[2]->[0]->[1]->[2]->[3]
[6]<-[5]<-[4]<-[3]<-[2]<-[1]<-[0]
Why does the order of object metadatas within each original frame is completely reversed after the tiler? Is this expected behaviour?
I also noticed that this reversing order starts in version 6.1, in version 5.1 the order of objects within each original frame is preserved by the tiler plugin.