How to link multiple detectors to tracker?

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 6.2
• NVIDIA GPU Driver Version (valid for GPU only) 550.54.15
• Issue Type( questions, new requirements, bugs) questions
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I was able to plugin multiple primary detectors in the pipeline, however I’m facing issue while linking multiple detector output to the tracker plugin. I’ve attached an image of the current pipeline. I’m currently facing issue in the primary gie bin, where I’ve created a tee to add multiple detectors in the app. May I know how to link the primary gie bin to the tracker bin instead of redirecting the output to the fake sink?

what are the three models used to do respectively? is there any dependency relation? which same are you testing or referring to?

I have two detectors in the pipeline not three. They don’t have any dependency relation. They are just regular yolov5. The app that I’m currently modifying is deepstream-app

currently deepstream-app can only support one pgie and multiple sgie. but deepsream-app is opensource. you can modify the code to customize. the path is in create_pipeline of \opt\nvidia\deepstream\deepstream-7.0\sources\apps\sample_apps\deepstream-app\deepstream_app.c.
you can use this kind of “…pgie->pgie->nvtracker->…”.

I understand deepstream-app supports only one pgie, which is why I was modifying the source code. While I was doing this I was trying to follow how multiple sgie’s were connected to a tee and tried doing the same to pgie which is where I encountered the issue of not being able to link up the tee + pgie to tracker.

Is there any benefit of one over the other i.e, pgie linked serially to parallelly?

Just was wondering why there was a difference in the way sgie’s are linked to the suggested approach of linking pgie serially?


In your case, the two models are detection type, they need to insert serially before tracker because tracker needs to track the objects.
inserting sgie parallelly can improve performance because the main branch “…pgie->tracker” does not need to wait the sgie(classification) inference results.

But doesn’t the same hold for pgie too? Technically the pgie will infer on frames regardless of the results from the tracker, the tracker element fetches results if the frame is not inferenced by a pgie when the interval isn’t set or am I misinterpreting things here?

In my last comment, sgie means classification model. tracker is based on the bboxes got from detection models. in your case, the both detection models need to to added before tracker, and bboxes need to be got before tracker, so the two models need to added serially before tracker.

Heyy I don’t think you’re understanding my point at all here. I’ve been working on deepstream for quite some time now, so I’m aware of the basic terminologies. Also sgie’s can be detectors too! since it’s just another nvinfer plugin.

What I’m trying to achieve is in the pgie bin, you could have multiple pgie elements either serially or parallely, I’ve kinda done the serial linking of pgie in the past, however I wanted to try out the pgie to be linked parallelly using a tee and the pgie bin to be linked the tracker. I’m not sure how to attach a source pad from the pgie bin when the pipeline is split using a tee to the sink pad to the tracker. This is the problem I’m right now trying to solve

yes, sgie can be a detection model. for example, In this sample, sgie is a face detection.
please refer to NVIDIA ready-made parallel inference sample deepstream_parallel_inference_app.
if you still need to use own code, the current pipeline can run. but it is a little unreasonable. we can 't make sure the bboxes are generated before tracker and osd.