Nvinfer multi-threading

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson
• DeepStream Version 5.1
• JetPack Version (valid for Jetson only) 4.5.1
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hi,

I have one primary model and 8 other models running on deep-stream, the fps very low. Currently i am working on the optimization to increase the speed.
the primary model has two output (A and B )
five of the secondary models work on A output from primary
and the rest work on B output.

Currently they are working on sequential, how to guarantee the secondary models are working on parallel?
does tee or queue help us on this?

3

Yes, please have a try with tee.

4

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_troubleshooting.html#the-deepstream-application-is-running-slowly

5

Could you please share an example of using tee?

6

You can search GStreamer tee example on Google. Can you share your use case? So we can have a check if tee can resolve your issue.

7

image
image593×845 63.5 KB

Secondary models are not dependent on each other ? can we run them in parallel?

8

Have you tried setting operate-on-class-ids parameter?
I think that should be able to do what you’re trying. But only difference is that it is not parallel like your diagram.

Say for secondary_model5, secondary_model6, secondary_model7 if you set operate-on-class-ids=2 and secondary_model1, secondary_model2, secondary_model3, secondary_model4 have operate-on-class-ids=1 then I think you should have the desired behaviour.

9

Yes this is what is implemented currently, but in sequential not in parallel. I want the same result on parallel

1 Like
10

Can you share more on call Frame Parser Function?

12

Do you have any requirement to sync between the parallel pipeline? Or the parallel pipeline will link to fakesink.

13

parallel pipeline will link to fakesink.

14

Got it, seems your design is fine.

15

For me is very slow. the question is can I make it faster by run the models in parallel?? and how ? because now it is sequential.

16

Below is for performance tuning:

1 Like
18

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.