interference between tiler and stream demux

Accelerated Computing Intelligent Video Analytics DeepStream SDK
#1

Hi,

In recent work we ran into a case that a pipeline would generate both tiled and separated video streams from multiple input streams.
The pipeline looks like

source --> GIE --> tee --> tiler --> osd --> sink
                     ` --> demux --> osd --> sink
                               | --> osd --> sink
                               | --> osd --> sink
                               ......

This pipeline works as intended if either tiler or demux is present.

However when both are enabled, the detected boxes drawn in demuxed sinks are altered (seems by the tiler) even if the batched stream is split before both elements.
By printing the address of NvDsObjectMeta the metas look like different objects in each branch, but the boxes are always the same and shifted from what is printed right after GIE.
For example:

../src/face_probe_manager.cc:539]]] [frame 0x7f587c03e780] detected object: 0x7f587c013f00 [819, 168, 33, 42]
../src/pipeline.cc:148]]] [4|sep_stream_3] batch: 0x7f5a5401a4b0
../src/pipeline.cc:150]]] [4|sep_stream_3] frame: 0x7f5a54013430
../src/pipeline.cc:157]]] [4|sep_stream_3] object: 0x7f5a541d9160 [913, 416, 11, 14]
../src/pipeline.cc:148]]] [0|tiled_stream] batch: 0x7f5a2804fb30
../src/pipeline.cc:150]]] [0|tiled_stream] frame: 0x7f5a28054fa0
../src/pipeline.cc:157]]] [0|tiled_stream] object: 0x7f5a28053810 [913, 416, 11, 14]

where the box is printed as [x, y, w, h].
The coordinates are correctly shifted from 1920x1080 to right-bottom 1/4 part of tiled 1280x720, which is expected in tiled_stream but not in sep_stream_3.

This is not urgent but hope it could be resolved. Maybe more information about meta copying/sharing mechanism is available somewhere?

#2

Hi
Sorry for a late response, Have you solved this issue, if not, can you share one sample code which can demonstrate this issue for further check?

#3

Hi amycao,

Thanks for reply.
Unfortunately the problem is not solved yet, and the pipeline code is too complex to share.
Later I would work out a sample pipeline.

As update I’ve made some detailed log to see when the metadata is copied/changed.
From the printed addresses in various probes, it seems the NvDsBatchMeta (and its meta children) is consistently shared through tee/queue/nvosd elements. The copy is made in tiler/demux/nvvideoconvert as one would expect, e.g.

../src/probe_manager_face.cc:398]]] [detector] object: 0x7fe244021920 (140609780324640) [162, 246, 48, 57] on stream 3
../src/post_process_sink.hh:34]]] [tee_sink] object: 0x7fe244021920 (140609780324640) [162, 246, 48, 57]
../src/post_process_sink.hh:34]]] [tiler_queue_sink] object: 0x7fe244021920 (140609780324640) [162, 246, 48, 57]
../src/post_process_sink.hh:34]]] [demux_queue_sink] object: 0x7fe244021920 (140609780324640) [162, 246, 48, 57]
../src/post_process_sink.hh:34]]] [tiler_bin_sink] object: 0x7fe244021920 (140609780324640) [162, 246, 48, 57]
../src/post_process_sink.hh:34]]] [demux_sink] object: 0x7fe244021920 (140609780324640) [162, 246, 48, 57]
../src/post_process_sink.hh:34]]] [demux_src_3] object: 0x7fe42404c9b0 (140609780324640) [694, 442, 16, 19]
../src/post_process_sink.hh:34]]] [sep_3_sink] object: 0x7fe42404c9b0 (140609780324640) [694, 442, 16, 19]
../src/post_process_sink.hh:34]]] [tiler_bin_src] object: 0x7fe414032170 (140609780324640) [694, 442, 16, 19]
../src/post_process_sink.hh:34]]] [combined_sink] object: 0x7fe414032170 (140609780324640) [694, 442, 16, 19]

Here demux_src_3 --> sep_3_sink and tiler_bin_src --> combined_sink are directly linked.
The long integer inside () is a unique ID attached to object_meta->misc_obj_info[0] upon creation.
So it seems the copy in demux is affected by the coordinate shift in tiler.

#4

Hi
Sorry for a late reply, this is not supported in current version, and will be supported in future release.

#5