Please provide complete information as applicable to your setup.
• Hardware Platform: Jetson
• DeepStream Version: 6.2
• JetPack Version: 5.1.0
We have a multi camera setup where we are running inference on only selet streams and then we tile all streams for display and logging.
We have implemented the version below with a single streammux (where we try to get the GIE to only infer on selected streams by using the src-ids parameter on the preprocess element.
[group-0]
src-ids=0;1
here we expect the inference to only run on streams 0 and 1 and just pass streams 2 and 3 over to the display without going through the PGIE (or the PGIE src pad buffer where we add some extra state information to the object meta, nvosd display elements etc.
we expect these special custom overlays we create on GIE padprobes to only appear on inferred streams (as those object metas are only in the context of being inferred on.
here is a block diagram.
when we use this we immidiately notice two things;
-
our frame rate drops significantly (lets say eatch source produces frames at at least 30fps and works very well with a single stream at 30 with no drop in FPS but with this setup we drop to less than half of that)
-
The Bounding boxes of one stream keeps jumping and flickering (does not happen with single stream)
this may be because the
batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
also the PGIE srcpad migt be including non inferred frames (into the padprobe function that adds the extra metadata) as they get passed through the PGIE
I don’t really mind using multiple muxes and a demux to entirely prevent non inferred frames from getting into the PGIE
for exmple look at the layout below (not tested)
What I would like to know primarily is,
-
Is the method 1. the right approach to such a problem
- if so what can I do to avoid the exessive delay i am experiencing (I have tried both multi batched engines and single batched engine files but saw no improvement on faster processing times so I suspect I may be making a mistake there)
-
Is there a known standard way to to this (multi stream in-out with selective inferencing) which I may not know. if there is something like that already existing can you kindly point me in that direction.
Thanks.
Please let me know if this problem discription is unclear or you need more information.
Cheers,
Ganindu.
EDIT:
Not sutre this is because of some video compression artefacts (I am running two cameras and two file sources to simulate 4 inputs and I am only running inference on the file streams)