Deepstream run multiple model engines concurrently

Hello, could I know if it is possible to run on deepstream, multiple model engines (yolo) simultaneously on one source video and produce an output .mp4 file with all the model tracking results (all model tracking results combined into 1 output .mp4 file)
Could you please provide a sample for it.

What are these models? Detectors, classifier, segmentations, … What is the relationship between these models?

Take our deepstream-test2 as an example, https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_C_Sample_Apps.html#c-c-sample-apps-source-details, the pipeline runs 4 models on one stream. One detector model to detect cars in the video, three classifier models to identify the manufacture, color and type of the cars.

Thanks for the reply.
These are 2 detectors for different object detection.
The source video would be same. I tried to create two sources ( the same video file though).One source should work on a network engine of one model and another one works on a different network engine of another model. Similar to this post: Deepstream multiple network engines plugin problems
1.Can I run them simultaneously and produce the 2 output detection videos files? Instead of implementing two deepstream-apps, is it possible in one deepstream-app to have independent detectors?
I have used back to back detectors but second detector runs on detected frames of first detector. How could both the detectors run on full frame, which solution could be followed for this?
2. Can I run both the detectors engines simultaneously on full frame to detect different objects in one pipeline (It is like using two primary detectors)

  1. deepstream-app does not support such case, you need to develop your own app. PGIE and SGIE are configured with nvinfer config file. Take back to back as an example, [secondary_detector_config.txt]secondary_detector_config.txt is the nvinfer config file for SGIE, “process-mode=2” in the config file means to do the inference on objects but not the full frames. We have document which describes all parameters meaning for nvinfer config file https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html#gst-nvinfer-file-configuration-specifications, please read it carefully.

  2. Yes. You can enable two detectors by configing two PGIE nvinfer plugins in the pipeline.

Got it. Thank you.
For the 2nd question, to enable two PGIE nvinfer plugins in the pipeline: does it mean to add two primary gie’s in the deepstream_app_config.txt and two primary_detector_config.txt files in the pipeline? Sorry, but could I please know how to achieve this?

deepstream_app does not support multiple pgies. Please write your own app. back to back can be a reference. Another suggestion is to train a new model which can detect both kinds of objects, one model is enough.

Thank you very much for clarifying. So, Deepstream does not support concurrent loading of multiple network engines simultaneously on full frame right?

deepstream-app does not support multiple pgies but deepstream can support. deepstream is a SDK, deepstream-app is a sample app.

1 Like

Thank you very much