# Multiple input streams with multiple primary and secondary inference **URL:** \ **Category:** DeepStream SDK\ **Created:** [January 9, 2023, 8:02am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006 "2023-01-09T08:02:48Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Ni\_Fury](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Ni\_Fury](https://forums.developer.nvidia.com/u/Ni_Fury)\ **Post date:** [January 9, 2023, 8:02am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/1 "2023-01-09T08:02:48Z")
Please provide complete information as applicable to your setup. **• Hardware Platform (Jetson / GPU)** **• DeepStream Version** **• JetPack Version (valid for Jetson only)** **• TensorRT Version** **• NVIDIA GPU Driver Version (valid for GPU only)** **• Issue Type( questions, new requirements, bugs)** **• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)** **• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)** Hi, I have a question! So lets consider that I have 13 input streams, and I want to have the first 5 input streams to infer person config, and the next 5 streams to infer vehicle, and the last 3 frames to infer animals… So, how do I create a pipeline or script for the same where different streams work with different primary, and secondary inference? ---
**Author:** ![miguel.taylor](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/miguel.taylor/32/273049_2.png) [@miguel.taylor](https://forums.developer.nvidia.com/u/miguel.taylor)\ **Post date:** [January 9, 2023, 8:43pm UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/3 "2023-01-09T20:43:42Z")
Hi, You will need to create multiple pipelines. > So lets consider that I have 13 input streams, and I want to have the first 5 input streams to infer person config, and the next 5 streams to infer vehicle, and the last 3 frames to infer animals… For each primary inference you want to make you will need a separate pipeline. Batch the first 5 streams and use them as input for the person primary, then in another pipeline batch the next streams and pass them to the next primary, and so on. If you need to process all the streams afterwards, you will need to do some metadata manipulation to join all metadata and transfer it to another pipeline with a batch of all the streams. We have some example DeepStream pipelines: [DeepStream pipelines - RidgeRun Developer Wiki](https://developer.ridgerun.com/wiki/index.php/DeepStream_pipelines) I would need more information of your application to be able to give a more detailed answer. ---
**Author:** ![yingliu](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/yingliu/32/136703_2.png) [@yingliu](https://forums.developer.nvidia.com/u/yingliu)\ **Post date:** [January 17, 2023, 4:02am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/4 "2023-01-17T04:02:38Z")
@miguel.taylor , Thank you for you reply! @Ni_Fury Kindly let us know if you still need support on this topic, thanks. ---
**Author:** ![Ni\_Fury](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Ni\_Fury](https://forums.developer.nvidia.com/u/Ni_Fury)\ **Post date:** [January 19, 2023, 5:46am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/5 "2023-01-19T05:46:12Z")
> [@miguel.taylor](#): > > I would need more information of your application to be able to give a more detailed answer. In openvino ,we can share the model instance id through the streams, so I want to know if, once the models are loaded on the gpu, and if I can use the model’s instance-id sort in deepstream to avoid loading of models on the gpu multiple times. I hope I am clear. ---
**Author:** ![Ni\_Fury](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Ni\_Fury](https://forums.developer.nvidia.com/u/Ni_Fury)\ **Post date:** [January 19, 2023, 5:50am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/6 "2023-01-19T05:50:52Z")
> [@yingliu](#): > > Kindly let us know if you still need support on this topic, thanks. yes I do! ---
**Author:** ![Fiona.Chen](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/fiona.chen/32/15508_2.png) [@Fiona.Chen](https://forums.developer.nvidia.com/u/Fiona.Chen)\ **Post date:** [January 19, 2023, 7:02am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/7 "2023-01-19T07:02:26Z")
You can refer to the parallel multiple models sample: [NVIDIA-AI-IOT/deepstream\_parallel\_inference\_app: A project demonstrating how to use nvmetamux to run multiple models in parallel. (github.com)](https://github.com/NVIDIA-AI-IOT/deepstream_parallel_inference_app) ---
**Author:** ![Ni\_Fury](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Ni\_Fury](https://forums.developer.nvidia.com/u/Ni_Fury)\ **Post date:** [January 19, 2023, 8:57am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/8 "2023-01-19T08:57:50Z")
Here every parallel model is different… But, what I want is that, If one camera is running “Vehicle, and Licence plate”… Now that the vehicle model is already loaded… and the second camera is running “Vehicle and vehicleType”. How to use the same Vehicle model for the second camera which is already loaded by the first camera… Instead of loading it again? ---
**Author:** ![miguel.taylor](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/miguel.taylor/32/273049_2.png) [@miguel.taylor](https://forums.developer.nvidia.com/u/miguel.taylor)\ **Post date:** [January 19, 2023, 5:07pm UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/9 "2023-01-19T17:07:09Z")
@Ni_Fury You can pass all the camera streams through all the secondaries, so both cameras would have “Vehicle, Licence plate, and vehicleType”. I assume you don’t want to do this to avoid unnecessary processing, but it is the easiest solution. A cool feature to solve this kinds of problems would be an extra property added to `nvinfer` to specify the batch id we want to operate, something like `operate-on-gie-id` for the frame index inside a batch. But currently there is no such option on DeepStream. As far as I know you would need to operate on all streams or have separate pipelines loading the models used on each pipeline. There is a third option using `nvinferserver` and Triton to serve the models and consume them from multiple pipelines, effectively loading each model once. We tried doing this on DeepStream 5.0 but each `nvinferserver` element still used its own model instance. I don’t know if this changed in more recent DeepStream versions. ---
**Author:** ![Ni\_Fury](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Ni\_Fury](https://forums.developer.nvidia.com/u/Ni_Fury)\ **Post date:** [January 31, 2023, 7:06am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/10 "2023-01-31T07:06:13Z")
> [@Fiona.Chen](#): > > You can refer to the parallel multiple models sample: [NVIDIA-AI-IOT/deepstream\_parallel\_inference\_app: A project demonstrating how to use nvmetamux to run multiple models in parallel. (github.com)](https://github.com/NVIDIA-AI-IOT/deepstream_parallel_inference_app) Is there any python implementation for the same?? ---
**Author:** ![Fiona.Chen](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/fiona.chen/32/15508_2.png) [@Fiona.Chen](https://forums.developer.nvidia.com/u/Fiona.Chen)\ **Post date:** [January 31, 2023, 7:25am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/11 "2023-01-31T07:25:22Z")
> [@Ni\_Fury](#): > > Is there any python implementation for the same?? No. ---
**Author:** ![Ni\_Fury](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Ni\_Fury](https://forums.developer.nvidia.com/u/Ni_Fury)\ **Post date:** [January 31, 2023, 7:36am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/12 "2023-01-31T07:36:24Z")
> [@Fiona.Chen](#): > > No. Any other help that I can get for my reference? To build the pipeline in python? ---
**Author:** ![Fiona.Chen](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/fiona.chen/32/15508_2.png) [@Fiona.Chen](https://forums.developer.nvidia.com/u/Fiona.Chen)\ **Post date:** [January 31, 2023, 7:41am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/13 "2023-01-31T07:41:41Z")
Yes, you can transfer the c/c++ app to python with python bindings. [NVIDIA-AI-IOT/deepstream\_python\_apps: DeepStream SDK Python bindings and sample applications (github.com)](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps) ---
**Author:** ![Ni\_Fury](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Ni\_Fury](https://forums.developer.nvidia.com/u/Ni_Fury)\ **Post date:** [January 31, 2023, 8:01am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/14 "2023-01-31T08:01:43Z")
I am supposed to clone the parallel repo inside the pythonapps and then run the steps for binding right? ---
**Author:** ![Fiona.Chen](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/fiona.chen/32/15508_2.png) [@Fiona.Chen](https://forums.developer.nvidia.com/u/Fiona.Chen)\ **Post date:** [January 31, 2023, 8:41am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/15 "2023-01-31T08:41:09Z")
No. You need to rewrite the app with python bindings. ---
**Author:** ![Ni\_Fury](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Ni\_Fury](https://forums.developer.nvidia.com/u/Ni_Fury)\ **Post date:** [January 31, 2023, 9:04am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/16 "2023-01-31T09:04:13Z")
Is there any detailed explanation on tee, and queues… How to use them in the pipeline with proper demux and streammux? ---
**Author:** ![Fiona.Chen](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/fiona.chen/32/15508_2.png) [@Fiona.Chen](https://forums.developer.nvidia.com/u/Fiona.Chen)\ **Post date:** [February 1, 2023, 8:47am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/17 "2023-02-01T08:47:53Z")
They are all provided by GStreamer community. Please refer to GStreamer document and source code. [queue (gstreamer.freedesktop.org)](https://gstreamer.freedesktop.org/documentation/coreelements/queue.html?gi-language=python) [tee (gstreamer.freedesktop.org)](https://gstreamer.freedesktop.org/documentation/coreelements/tee.html?gi-language=python) ---
**Author:** ![Ni\_Fury](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Ni\_Fury](https://forums.developer.nvidia.com/u/Ni_Fury)\ **Post date:** [February 6, 2023, 7:00am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/18 "2023-02-06T07:00:31Z")
I am trying to implement it this way… my code is attached below… Can you please tell me where am I going wrong, I am unable to see any output… [parallel\_pipeline.py](https://forums.developer.nvidia.com/uploads/short-url/ryKmkM1mgSYUywZjmcmGJA3bJcR.py) (10.8 KB) ---
**Author:** ![Ni\_Fury](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Ni\_Fury](https://forums.developer.nvidia.com/u/Ni_Fury)\ **Post date:** [February 6, 2023, 7:21am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/19 "2023-02-06T07:21:54Z")
The output:: Creating Pipeline Creating streamux Creating source\_bin 0 Creating source\_bin 1 Creating Tee Creating nvstreamdemux vehiclecount defaultdict(\, {‘vehiclecount’: [0, 1]}) LINKINGGGGGG 0 LINKINGGGGGG 1 Creating the pgie OUTSIDE LOOP Creating gst-dsmetamux Creating tiler Creating nvvidconv Creating nvosd Creating H265 Encoder Creating H265 rtppay Adding elements to Pipeline ---
**Author:** ![Fiona.Chen](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/fiona.chen/32/15508_2.png) [@Fiona.Chen](https://forums.developer.nvidia.com/u/Fiona.Chen)\ **Post date:** [February 7, 2023, 1:37am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/20 "2023-02-07T01:37:20Z")
@Ni_Fury Seems you have created new topic, so this topic is closed. ---
**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\ **Post date:** [February 21, 2023, 1:38am UTC](https://forums.developer.nvidia.com/t/multiple-input-streams-with-multiple-primary-and-secondary-inference/239006/21 "2023-02-21T01:38:06Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.