Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU) • DeepStream Version • JetPack Version (valid for Jetson only) • TensorRT Version • NVIDIA GPU Driver Version (valid for GPU only) • Issue Type( questions, new requirements, bugs) • How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) • Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
Hi, I have a question!
So lets consider that I have 13 input streams, and I want to have the first 5 input streams to infer person config, and the next 5 streams to infer vehicle, and the last 3 frames to infer animals…
So, how do I create a pipeline or script for the same where different streams work with different primary, and secondary inference?
So lets consider that I have 13 input streams, and I want to have the first 5 input streams to infer person config, and the next 5 streams to infer vehicle, and the last 3 frames to infer animals…
For each primary inference you want to make you will need a separate pipeline. Batch the first 5 streams and use them as input for the person primary, then in another pipeline batch the next streams and pass them to the next primary, and so on. If you need to process all the streams afterwards, you will need to do some metadata manipulation to join all metadata and transfer it to another pipeline with a batch of all the streams.
In openvino ,we can share the model instance id through the streams, so I want to know if, once the models are loaded on the gpu, and if I can use the model’s instance-id sort in deepstream to avoid loading of models on the gpu multiple times.
Here every parallel model is different…
But, what I want is that, If one camera is running “Vehicle, and Licence plate”… Now that the vehicle model is already loaded… and the second camera is running “Vehicle and vehicleType”.
How to use the same Vehicle model for the second camera which is already loaded by the first camera… Instead of loading it again?
@Ni_Fury You can pass all the camera streams through all the secondaries, so both cameras would have “Vehicle, Licence plate, and vehicleType”. I assume you don’t want to do this to avoid unnecessary processing, but it is the easiest solution.
A cool feature to solve this kinds of problems would be an extra property added to nvinfer to specify the batch id we want to operate, something like operate-on-gie-id for the frame index inside a batch. But currently there is no such option on DeepStream. As far as I know you would need to operate on all streams or have separate pipelines loading the models used on each pipeline.
There is a third option using nvinferserver and Triton to serve the models and consume them from multiple pipelines, effectively loading each model once. We tried doing this on DeepStream 5.0 but each nvinferserver element still used its own model instance. I don’t know if this changed in more recent DeepStream versions.
I am trying to implement it this way… my code is attached below…
Can you please tell me where am I going wrong, I am unable to see any output… parallel_pipeline.py (10.8 KB)