Using secondry detector as separate TRT based custom code rather than back-to-back detector

[sharma.rahul98912]

• Hardware Platform Nvidia Tesla T4
• DeepStream Version 5.1

• TensorRT Version 7.2.2.3
• NVIDIA GPU Driver Version (valid for GPU only) 460.32…03
• Issue Type( questions, new requirements, bugs) question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
use person detector as primary and face detector as separate trt based in gst-dsexample.

Hi, continuing to my previous post “Using detectors on condition based for different streams”, i have few querries-

To clarify more here are the steps that i used in gst-dsexample-
*) Created secondry detector instance in “gst_ivcore_start”.
*) Got frame in ds_example->process_full_frame through get_converted_mat.
*) Did secondry detector inferencing and got the bboxes and info over there.

Querry-

1. After bbox info, how can i use tracker with to get tracker IDs for the same?

2. If i use back-to-back detector then in gst-dsexample plugin, can i get secondary detector information to modify?

3. in back-to-back detector, secondary detection runs after all primary detections sequentially. Is this correct? if yes So is there way where i can define sources for secondary detector to be run? like what my main goal is.

4. If i want to use back-to-back detector, i need to use this implementation in deepstream-app.c to get more plugins usage like tracker, gst plugin etc… So do i need to modify secondary_gie.cpp? or in need to modify primary_gie_bin?

5. If somehow it is implemented, Does tracker gives ID’s for both primary and secondary detections?

Thanks.

[Fiona.Chen]

@sharma.rahul98912

• After bbox info, how can i use tracker with to get tracker IDs for the same?
• If i use back-to-back detector then in gst-dsexample plugin, can i get secondary detector information to modify?
• in back-to-back detector, secondary detection runs after all primary detections sequentially. Is this correct? if yes So is there way where i can define sources for secondary detector to be run? like what my main goal is.
• If i want to use back-to-back detector, i need to use this implementation in deepstream-app.c to get more plugins usage like tracker, gst plugin etc… So do i need to modify secondary_gie.cpp? or in need to modify primary_gie_bin?
• If somehow it is implemented, Does tracker gives ID’s for both primary and secondary detections?

1. You must make sure the face object class id is different to person object class id. You can use nvdstracker after any detector.
2. I don’t understand “If i use back-to-back detector then in gst-dsexample plugin”. Do you mean using ds-example as the secondary detector but not nvinfer?
3. PGIE and SGIE is working sequentially. You can not designate specific source to work on for either PGIE or SGIE. If you don’t want some sources be inferenced, please do not put them in batch.
4. No need to modify deepstream-app. deepstream-app already supports PGIE+SGIE mode.
5. No. One tracker only tracking the output from one detector.

[sharma.rahul98912]

Thanks for reply.
You can use nvdstracker after any detector.
Is there simple API call or i need to initiallize tracker in some ways?

I don’t understand “If i use back-to-back detector then in gst-dsexample plugin”. Do you mean using ds-example as the secondary detector but not nvinfer?
i mean i have to use gst-dsexample as to modify the output for my application, So if i use back-to-back detector and i am getting two bboxes info (primary and secondary). So in gst plugin how can i use both detectors output ? as in ObjMeta, only primary detector info is there.

No need to modify deepstream-app. deepstream-app already supports PGIE+SGIE mode.
SGIE is classifier i guess. Can i use this as detector also by simply enable [secondary_gie] and providing the detector model path? Parser logic might be different inside i guess.

[Fiona.Chen]

@sharma.rahul98912

Is there simple API call or i need to initiallize tracker in some ways?

You need to use gst-nvtracker plugin.

@sharma.rahul98912

i mean i have to use gst-dsexample as to modify the output for my application, So if i use back-to-back detector and i am getting two bboxes info (primary and secondary). So in gst plugin how can i use both detectors output ? as in ObjMeta, only primary detector info is there.

In object meta, unique_component_id is different. I think there is already sample in back-to-back code deepstream_reference_apps/back_to_back_detectors.c at master · NVIDIA-AI-IOT/deepstream_reference_apps · GitHub, please read the code carefully.

@sharma.rahul98912

SGIE is classifier i guess. Can i use this as detector also by simply enable [secondary_gie] and providing the detector model path? Parser logic might be different inside i guess.

No. SGIE means secodary GIE(GPU inference engine), it can be classifier, detector or others.

[sharma.rahul98912]

Thanks alot.

[sharma.rahul98912]

@Fiona.Chen

nvtracker

hi,
before end this discussion, i just wanted to reconfirm about tracker-usage .

You told me to use gst-nvtracker plugin. How can i use that in gst-dsexample? What i found is i have to use with proper pipeline initialization like gst_element_factory_make (“nvtracker”, “tracker”)…

But here i wanted to implement simple tracker code which can be used as simple API in between detected object in dsexample code. Does gst-nvtracker gives that liberty?

[Fiona.Chen]

gst-dsexample s also a plugin, you can not use gst-nvtracker inside gst-dsexample. There are several deepstream samples of how to use gst-nvtracker in pipeline in deepstream package.