Secondary gie0 depends on the output of primary gie, secondary gie1 depends on the output of secondary gie0,but gie1 got the wrong result

**• Hardware Platform (Jetson / GPU)Tesla T4
**• DeepStream Version 5.0.1
• JetPack Version (valid for Jetson only)
**• TensorRT Version 7.0
**• NVIDIA GPU Driver Version (valid for GPU only)440.118.02

Primary gie is a yolov5 detector model, secondary gie0 and gie1 are resnet50 classification model,I can get the results of primary gie and secondary gie0 correctly, but when gie0 is passed to gie1, I cannot get the correct result of gie1,as far as I know, deepstream does not support the results of a classification model to be passed to another classification model in secondary -gie.Is there any workaround to solve this problem?

Below are my configuration file:


primary-gie config file:

secondary-gie0 config file:

secondary-gie1 config file:

Sorry for the late response, we will investigate this issue and do the update soon.

Thanks

What is your SGIE0’s output? What is your SGIE1’s function?

The output of sgie0’s refers to the inference result of secondary gie0, and the sgie1 function is in the screenshot below


I refer to link Secondary gie depend on another secondary gie, I modify should_infer_object function and get a error result.

What is the sgie1’s input? A object image or others?
Does sgie0 output bbox?

yes,sgie0 output bbox,sgie0 can get a correct result,the input of sgie1 is based on the output of the classification result of sgie0

Can PGIE also output bbox? Are the class id different with pgie and sgie0?

Sorry for the late reply ,PGIE and SGIE0 can get correct result, PGIE is a yolov5 detect model,the input of SGIE0 is based on the output of the detect result of PGIE,deepstream does not execute SGIE1 by default.,when I modify the should_infer_object function, SGIE1 can output bbox, but it is an out-of-order result.

@17396156679 Can you explain what is your PGIE, SGIE0 and SGIE1, the input and output of them in details?

For example, the PGIE is a detector which detect persons in picture, SGIE0 is a detector which detect helmet from one person picture, SGIE1 is a classifier which identify the color of the helmet from SGIE0. Can you describe your models in this way?

Can you explain what do you mean by “when I modify the should_infer_object function, SGIE1 can output bbox, but it is an out-of-order result.”?

And this question is very important, please answer it : Are the class id different with pgie and sgie0?

I did not understand the meaning of the sentence whether the class id of pgie and sgie0 are different .PGIE detects two categories, namely class id0: helmet, class id1: no_helmet. When class id is equal to 0, the SGIE0 classification model is executed, otherwise the detection result is not passed to SGIE0. SGIE0 has three categories, namely class id0: belt_helmet, class id1: right_helmet, class id2: wrong_helmet, when the classification class id is equal to 0, the result is passed to SGIE1 to continue classification, otherwise the result is output directly, SGIE1 has two categories, class is0: rightBelt_helmet, class id1: wrongBelt_helmet, but the default The SGIE1 classification model will not be executed.
Config File:

According to your config files, PGIE is a detector which can output bbox, SGIE0 and SGIE1 are classifiers which can only output classification labels.

So it is no meaning to set “operate-on-gie-id=2” in SGIE1’s config file. Currently SGIE only support filtering by class id(which is “operate-on-class-ids” parameter in config file), but classifier does not generate or change any class id. The class id and bbox are all generated by PGIE which is a detector.

If you want to filter by SGIE0 identified class, you may try to change the class_id in NvDsObjectMeta to a new value after SGIE0 inference.

To read the source code of gst-nvinfer can help you to understand the relationship of PGIE and SGIE.

I understand what you mean and I will try. Thank you for your reply.