Hi. I’m running a deepstream pipeline containing pgie detector + sgie classifier. Given detection metadata extracted from some frame, I’m not always able to extract its corresponding classification metadata.
PIPELINE: The pipeline I’m using is the following (parsed with Gst.parse_launch)
BUFFER_PROBE_CALLBACK: Metadata extracted just like in the sample callback up to this point, attached to the monitor element. Then:
...
obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
report_obj_meta_counter(obj_meta)
classifier_meta_objects = obj_meta.classifier_meta_list # THIS IS NONE SOMETIMES, but generally a `pyds.GList`
while classifier_meta_objects is not None:
try:
classifier_metadata = pyds.NvDsClassifierMeta.cast(classifier_meta_objects.data)
except StopIteration:
break
label_info_list = obj_meta.label_info_list # I think we've got to iterate again here for the multilabel case...
while label_info_list is not None:
try:
label_info = pyds.NvDsLabelInfo.cast(label_info_list.data)
except StopIteration:
break
report_label_info_counter(label_info)
try:
label_info_list = label_info_list.next
except StopIteration:
break
try:
classifier_meta_objects = classifier_meta_objects.next
except StopIteration:
break
...
• NOTES
Using the snippet above, the number of classifications (15249 as reported by report_label_info_counter) is slightly lower than the number of detections (16496 as reported by report_obj_meta_counter). There are cases where obj_meta.classifier_meta_list is None, instead of a pyds.GList.
The same number of frames (1088) are being processed if I change the decoder from jpegdec to nvjpegdec. If I turn on raw-output-tensors, both the pgie and sgie output 1088, consistent with the number of frames. However, I loose 11 detections (from 16496 to 16485. On the other hand, the number of classifications goes up from 15249 to 15252. These numbers are consistent across >10 runs each.
I can manually enforce the SGIE to “skip” detections, eg by increasing the input-object-min-width to 200. In that case, when also enabling raw-output-tensors, the number of sgie output tensors decreases (to 365), which makes me doubt its a sgie config file issue…
Other
Using dla does not affect the numbers.
network-mode does not affect the numbers
Using jpegdec vs nvjpegdec does, but slightly (maybe a sync or a flush thing?), and does not solve the problem.
• QUESTIONS
What happened to those obj_meta which do not have corresponding classifier_meta_list?
I have not. Can I use it from python? I’m not comfortable with C++, but if it ends up being a pyds issue, I’m not closed to considering other alternatives. I’ll take a look in the meantime.
Hello there, any updates regarding this issue. I am facing similar thing using C++ deepstream.
I am using a primary model to detect people and a custom secondary model (classifier). However, not all detected objects have metadata attached from the secondary although setting classification-threshold to 0.1 and even 0
Hey @mohammad1 , so it’s not related to python, seems caused by meta data management, would you mind to share your repro with us, I would like to use the native c/c++ sample to debug.
It is the same deepstream-test2 example (vehicle + color + type + make) with small modifications. I am printing the output of the classification of each object. As shown in the image attached, classification exits for some objects but not all and for some classification models but not all.
which, accordfing to nvbufsurftransform.h, mean “NvBufSurfTransformError_Invalid_Params” and “NvBufSurfTransformError_Execution_Error”
when it did not crash, it got 795 (same value for both detector and classifier!). I think this might have happened regardless of the macro change If I had tried enough times…
This got me thinking:
The docs say this Integer 0: Platform default – GPU (dGPU), VIC (Jetson) 1: GPU 2: VIC (Jetson only)
The log asked to use GPU
sgie → scaling-compute-hw=1 => 16496 detections and 16496 classifications!!!
Thanks again for the insights @bcao , I’ll run more tests to ensure this is reproducible with different values, but this workaround completely fulfills my needs. If I am able to reproduce this after more experimentation, I’ll come back and mark as solved :)
I’ve tried changing the macros all the way down to 0, and scaling-compute-hw=1 allows correct execution.
NOTE: the smallest detections I got from the sample data had minimum bboxes in the 8-16 range, so while I can confirm this works for smaller bboxes than the original setting (lower than 16), I have not tested everything works /won’t crash in case the detection produces bboxes with dimensions lower than 8.
