How to run two different segmentation visualization plugins and display result in a tiled display?

Hi,
I have developed a segmentation visualization plugin based on “nvsegvisual”.
I want to make these two plugins running at the same time and show the result size by size on a tiled display.

I tried the following command:(use only one nvinfer and tee its output to two different segvisual plugins)

gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream-4.0/samples/streams/sample_720p.mjpeg ! jpegparse ! nvv4l2decoder mjpeg=1 ! m.sink_0 nvstreammux name=m width=512 height=512 batch-size=1 batched-push-timeout=4000000 ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream-4.0/sources/apps/sample_apps/deepstream-segmentation-overlay/dstest_segmentation_config_semantic.txt batch-size=1 !
tee name=t ! nvsegvisual batch-size=1 width=512 height=512 ! n.sink_0
t. ! jwsegvisual batch-size=1 width=512 height=512 ! n.sink_1
nvstreammux name=n width=512 height=512 batch-size=2 !
nvmultistreamtiler rows=1 columns=2 width=1024 height=512 !
nvegltransform ! nveglglessink async=false

But there are two problems:

  1. the framerate is very low, about 1 fps
  2. segfault after few seconds.

Any idea how to split the result of nvinfer to multiple post-processing plugins and then merge the results into a tiled display?

Thanks.

/jason

Why do you need 2 streammux and 2 segvisual ?

Hi,

  1. I am trying to compare the visualization of two segvisual plugins and want to see them showing side by side.

  2. I actually tested some different combinations but could not find any working. I used 2 streammux, the first one before nvinfer is needed because without it, nvinfer complains about missing NvDsBatchMeta and quit, see the log below. The second streammux is used to merge the two streams of visualization result from these 2 sevisual plugins.

0:00:13.647784504 28567   0x7f1c080990 WARN                 nvinfer gstnvinfer.cpp:1201:gst_nvinfer_process_full_frame:<nvinfer0> error: NvDsBatchMeta not found for input buffer.
ERROR: from element /GstPipeline:pipeline0/GstNvInfer:nvinfer0: NvDsBatchMeta not found for input buffer.
Additional debug info:
/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(1201): gst_nvinfer_process_full_frame (): /GstPipeline:pipeline0/GstNvInfer:nvinfer0
Execution ended after 0:00:00.182359331
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
0:00:13.648644068 28567   0x55b7325ed0 WARN               baseparse gstbaseparse.c:3611:gst_base_parse_loop:<jpegparse0> error: Internal data stream error.
0:00:13.648673733 28567   0x55b7325ed0 WARN               baseparse gstbaseparse.c:3611:gst_base_parse_loop:<jpegparse0> error: streaming stopped, reason error (-5)
Setting pipeline to NULL ...
Freeing pipeline ...

Segmentation sample supports multiple streams. Did you try?
Don’t suggest use 2 streammux in one pipeline.

Do you mean:
create 2 file sources of the same input file, merge the streams into the first streammux => nvinfer => split into two streams with tee and then one stream per each segvisual pulgin ?
I still have a question: how to merge the result of two segvisual plugins if not using the second streammux.

segvisual pulgin also supports to display multiple masks.

./deepstream-segmentation-app <config_file> [file2] … [fileN]

yes, I know segvisual support displaying multiple masks.
But the question is not that it cannot display multiple masks.
The question is how to display multiple segvisual plugin.
I have my own segvisual plugin which output not masks but overlay.

You can create 2 file sources of the same stream and run inference on each. After that you need to use nvstreamdemux component and feed the output of each demux stream to different segmentation visualization plugins and then mux the output of those to feed it to nvtiler

how do i add a pixel mask to the deepstream-segmenation-test output using semantic segmentation?

Hi lucasssss,

Please open a new topic for your issue. Thanks

Hi, kayccc
New topic created https://devtalk.nvidia.com/default/topic/1072654/deepstream-sdk/-jetson-xavier-deepstream-segmentation-test-with-custom-pixel-mask/

please answer my question.