How to use nvstreamdemux and nvstreammux to process multiple live videos input and output

Please provide complete information as applicable to your setup.

• Hardware Platform (jetson xavier nx devkit)
• DeepStream Version5.0
• JetPack Version (valid for Jetson only)4.4
• TensorRT Version7.1

Hey guys,I’m working on a pipeline that process a few live videos and output them with rstp server respectively.
Now I have a simple test based on deepstream-test3(python),I added a nvstreamdemux after nvdsosd and created two nvegltransform&nveglglessink,the pipeline like :

uridecoderbin1 \ _______________________________________________ / nvegltransform1 ->nveglglessink1
______________ nvstreammux ->nvinfer ->… ->nvdsosd ->nvstreamdemux
uridecoderbin2 / _______________________________________________ \nvegltransform2 ->nveglglessink2

and I got :
Decodebin child added: source

Decodebin child added: source

Decodebin child added: decodebin0

Decodebin child added: decodebin1

Decodebin child added: rtppcmadepay0

Decodebin child added: rtph264depay0

Decodebin child added: alawdec0

Decodebin child added: h264parse0

In cb_newpad

gstname= audio/x-raw
Decodebin child added: capsfilter0

Decodebin child added: nvv4l2decoder0

Seting bufapi_version

Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f9eeae048 (GstCapsFeatures at 0x7edc08da00)>
reference in DPB was never decoded
Frame Number= 0 Number of Objects= 1 Vehicle_count= 0 Person_count= 1
Segmentation fault (core dumped)

update:
I changed nvstreamdemux after nvinfer,segmentation fault disappeared ,but stream fps is down to 0.2 and just show picture in one nveglglessink and the other one just no picture.
echo like:
Frame Number=0 Number of Objects=1 Vehicle_count=1 Person_count=0
Frame Number=0 Number of Objects=1 Vehicle_count=1 Person_count=0

and just stucked,
after ctrl+c,echo like:

Frame Number=1 Number of Objects=1 Vehicle_count=1 Person_count=0
Frame Number=1 Number of Objects=1 Vehicle_count=1 Person_count=0
stream 0 fps is 0.2********
stream 1 fps is 0.2********
here is my pipeline:

How can I use nvstreamdemux to get multiple stream output properly?
location of nvstreamdemux is special in a pipeline?
I find nvstreamdemux in deepstream5.0 guide,only input and output,it’s a reverse of nvstreammux?

I cant tell what your actual question is… The pipeline graph seems to show that you have the streamdemuxer set correctly. Are you getting errors?

Thanks for help ,I can open two python3 windows normally to show,but one is just stuck and another one just has no picture,and print is stuck too ,like below


after ctrl+c

I’m not sure… you could try putting queue’s after each branch of streamdemux - thats what I have in my C deepstream apps and the demuxer works just fine.

Most sample pipelines have the OSD component in there as well and prior to the a nvvideoconvert from NV12 to RGBA - not sure if that would help either…

@cy_workmail Can you refer to the sample codes of deepstream-app in Deepstream SDK package? There are details of using nvstreammux and nvstreamdemux.

Thanks for help!I added queue after nvstreamdemux and it worked,but it’s not perfect.
1.when playing streams,one stream mixed with some pictures from another stream,and now is no nvdsosd

2.then I want to add nvdsosd,but when I add nvdsosd before nvstreamdemux,but segmentation fault come back again,
here is the pipeline

3.so I change nvdsosd to behind nvstreamdemux,it worked but streams are still mixed,pipeline like below

comparing pipelines, different inputs and outputs are more likely to be the reason for segmentaion fault?
and what do you think about problem stream-mixed ,is setting incorrect?

Thanks for your response,I’ll read it later. Can you help me with these now?

and I find a bug report in forums and someone said that there is a little bug about multiple sinks in deepstream5.0.
can it cause segmentation fault?and fix plan?

@cy_workmail Do you mean the “segment fault” crash with different source id sinks? You can use the patch in BUG REPORT: the deepstream5.0 reference application process sub_bins index incorrectly in demux setting. This solution will be released with next version Deepstream SDK.

Don’t cross the streams. It would be bad! Try to imagine all life as you know it stopping instantaneously and every molecule in your body exploding at the speed of light.

Make sure that async = false and sync = false on your sinks.

sorry,what does ‘cross the streams’ mean ? I set async = false and sync = false both two sinks,but stream-picture -mix is still,and I find streams’ fps decreased from 25->16->0.2 after playing for a while ,but utilization of cpu and gpu are low ,are buffers stuck?

@cy_workmail Can you upload your codes so that we can take a view and reproduce your problem?

Hi Fiona, I sent you a massage which includes my code and problems.

The user send message with https://forums.developer.nvidia.com/t/re-how-to-use-nvstreamdemux-and-nvstreammux-to-process-multiple-live-videos-input-and-output/144376, https://forums.developer.nvidia.com/t/re-how-to-use-nvstreamdemux-and-nvstreammux-to-process-multiple-live-videos-input-and-output/142430, and it is answered.