RTSP stream (RGBA) → meta transfer detections 1 + 2 → model-3 → detections 1 + 2 + 3 over RGBA stream
We created a custom GStreamer element called dsmetatransfer that transfers DeepStream meta from one stream to another.
You can do something similar with meta manipulation to join the meta from the 2 individual pipelines and transfer it to another pipeline where the 2 streams are batched. That way you can use the multistream tiler.
However, I think it would be easier just to create your own overlay element that joins the outputs from the two DeepStream pipelines and displays them.
One way or another, you will need to add something custom. With the default DeepStream components you can’t perform multiple primary inferences over different streams from the same batch.
@miguel.taylor thanks for your sharing, could you share more about your use case? could you share the whole pipeline with dsmetatransfer? could you elaborate the dsmetatransfer plugin, how to transfers DeepStream meta from one stream to another? thanks!
@rahul17 please refer to deepstream_parallel_inference_app, which is similar to your use case, it can support inference parallelly, for example: some sources will go to model1, some sources will go to model2, then use dsmetamux plugin to mux the meta.
But then we switched to our own custom metadata. We serialize the values we need from the DS meta into a JSON. Then, we send that JSON around using NvDsEventMsgMeta, we had to do this because some DeepStream elements were only copying DS meta and deleting our custom meta type. We also have separate custom elements to transform a serialized meta back to DeepStream so that it will be displayed with the overlay.