Multi primary detecter inference for one camera, is it possible?

[zyctimes]

• Hardware Platform: Jetson NX
• DeepStream Version: 5.0
• JetPack Version: 4.4
• TensorRT Version: 7.1.0

Hi Sir/Madam:

Days ago, I raised a question: Multi primary detecter inference for different cameras, is it possible?. Many thanks for your response.

A similar, but different question under the reply from “chunleonglee”:
An extension of the above question, what if for each stream of a few different models will be used.
E.g. use pretrained vehicle detect, human detect on the same video stream , how will it be given that is only one pgie?

I am also not clear of this question. Although Deepstream supports one pgie and multiple sgie, I am not sure whether those sgie can be regarded as pgie(s) and how to actually use them (as pgies). Many thanks if any examples can be provided.

Best
Yichao

[Fiona.Chen]

You can use multiple pgie, but please make sure they are assigned different unique-id and the detected class id should be different too.

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html#

The better way is to use one model to detect multiple classes.

[zyctimes]

@Fiona.Chen

unique-id

So, that means, we could attach/add one pgie after another, and then Gst-nvmultistreamtiler, osd, etc. And this pipeline should run. Also the processing time is [pgie1]+[pgie2]+…+[pgieX]_[other normal modules].
Right?

It looks like the output of pgie could also be the input of next pgie.

[Fiona.Chen]

@zyctimes

It looks like the output of pgie could also be the input of next pgie.

No. The output of pgie is bboxes. They will never be the input of the next pgie

[zyctimes]

bboxes or related metadata will not input to next nvinfer. but I mean, The other output of nvinfer is Gst Buffer. Can it be the input of next nvinfer? since it is the input of nvtracker.

Now I am a bit confused your initial comments “You can use multiple pgie, but please make sure they are assigned different unique-id and the detected class id should be different too.” How exactly second or third pgie are attached?

Is it possible if you could make an example? I think many people are curious of how multiple models are used in Deepstream pipeline in details. I will also try in weeks and publish results here if success, but official examples are always more convincing, you know :)

Best
Yichao

[Fiona.Chen]

GstBuffer will be the input of all element in the pipeline.

Currently we don’t have such sample. It may take some time to write the sample. We will consider the requirement in the future.

[zyctimes]

Hi Fiona:

Thank you so much for your comment.
I think many people are interested to see this sample. Because, if multiple models can be implemented (they could even be processed by GPU parallelly I guess), then much more implementation areas can be covered. Also, you do not need to build multiple pipelines for multiple pgies.
It is even a milestone in my mind to make multiple pgies work in one pipeline. Very meaningful and worth try.

Again, thank you so much if there can be an official sample published in the future. I will also try and update the results here if success.

Best
Yichao