Different pipeline for different stream after nvstreamdemux

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi,
I am having some issues while using nvstreamdemux. The pipeline works fine when I use the same set of elements for both the source after nvstreamdemux but if I use some other set of elements for different stream the pipeline gets stuck(does not change state to playing).

Here I tried:
gst-launch-1.0 nvstreammux name=mux batch-size=2 width=1920 height=1080 ! nvinfer config-file-path=dstest1_pgie_config.txt ! nvstreamdemux name=demux filesrc location=…/…/…/…/samples/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_0 filesrc location=…/…/…/…/samples/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_1 demux.src_0 ! “video/x-raw(memory:NVMM), format=NV12” ! queue ! nvvideoconvert ! “video/x-raw(memory:NVMM), format=RGBA” ! nvdsosd ! nvvideoconvert ! “video/x-raw, format=RGBA” ! videoconvert ! “video/x-raw, format=NV12” ! x264enc ! qtmux ! filesink location=./out3.mp4 demux.src_1 ! fakesink

3

DeepStream supports multiple inputs with different sources, could you check if there is any difference from yours?

Thanks!

4

Hi @mchi,
I have been using deepstream container to do my experiments on deepstream. Yes, it supports multiple inputs with different sources.

For example this works, if that is what you are asking:
gst-launch-1.0 nvstreammux name=mux batch-size=2 width=1920 height=1080 ! nvinfer config-file-path=dstest2_pgie_config.txt ! nvstreamdemux name=demux filesrc location=/home/neil/use/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_0 filesrc location=/home/neil/use/streams/sample_1080p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_1 demux.src_0 ! queue ! nvvideoconvert ! nvdsosd ! nvvideoconvert ! videoconvert ! x264enc ! qtmux ! filesink location=./out3.mp4 demux.src_1 ! queue ! nvvideoconvert ! nvdsosd ! nvvideoconvert ! videoconvert ! x264enc ! qtmux ! filesink location=./out4.mp4

My actual motivation is to have the same path for the initial part of the pipeline(for different streams) then vary the pipeline depending on the stream id. So I am using nvdemux to separate the streams.
The pipeline I want to realise is:

uridecodebin → nvstreammux → nvinfer → nvstreamdemux name=demux demux → path1(stream1) demux → path2(stream2)

So the different cases where this worked:
1st
path1: → queue → nvconv → nvosd → nvconv → vc → x264enc → rtph → udp
path2: → queue → nvconv → nvosd → nvconv → vc → x264enc → rtph → udp

2nd
path1: → queue → nvconv → nvosd → nvconv → vc → x264enc → qtmux → filesink
path2: → queue → nvinfer(secondary classifier) → nvconv → nvosd → nvconv → vc → x264enc → rtph → udp

3rd
path1: → queue → nvconv → nvosd → msgconv → msgbroker
path2: → queue → nvconv → nvosd → msgconv → msgbroker

4th
path1: → queue → nvconv → nvosd → msgconv → msgbroker
path2: → queue → nvconv → nvosd → fakesink

The Cases where this did not work:

1st
path1: → queue → nvconv → nvosd → nvconv → vc → x264enc → rtph → udp
path2: → fakesink or → queue → fakesink

2nd
path1: → queue → nvconv → nvosd → nvconv → vc → x264enc → qtmux → filesink
path2: → fakesink or → queue → fakesink

3rd
path1: → queue → nvconv → nvosd → nvconv → vc → x264enc → rtph → udp
path2: → queue → nvconv → nvosd → msgconv → msgbroker

4th
path1: → queue → nvconv → nvosd → nvconv → vc → x264enc → qtmux → filesink
path2: → queue → nvconv → nvosd → msgconv → msgbroker

5th
path1: → queue → nvinfer(secondary classifier) → nvconv → nvosd → nvconv → vc → x264enc → rtph → udp
path2: → queue → nvconv → nvosd → msgconv → msgbroker

So 4th and 5th(did not work) are critical for my use case.

I had put a probe on the source pad of nvinfer(primary infer) so it displays the output for 1st few frames and then comes to a halt for the cases it does not work. So the frames are not
being consumed by path1 and path2 where it halts.

Is this pipeline restricted to some pipelines or there is some proper way to use nvdemux this?
Is there some other way to have a separate path in the pipeline depending on stream id?

Thanks.

7

Hi @duttaneil16,
Sorry for delay!

Shouldn’t use tee instead of demux as seems you want to split the data to two streams?

8

Hi @mchi,
Here by stream I mean two different sources(different videos) each having separate path after some point in the pipeline.
From what I have seen from the documentation and examples I could only find a way to separate two data sources which have been muxed together by the use of nvstreamdemux.
Can this be done using tee?
If use, how can the two sources be separated at tee using source id ?

9

Hi @duttaneil16
Sorry for long delay!

Checked internally, you are right, it’s rignt for your case to use nvstreamdemux to seperate the source.

Some questions:
Did you set “sink=0” for the demuxer ?
In “The Cases where this did not work:”, x264enc is using CPU encoding, right? did you try replacing it with nvv4l2h264enc ?

Thanks!

10

Hi @mchi

Thanks, the nvv4l2h264enc plugin worked. The plugin x264enc(gstreamer plugin) I have used for several pipeline with deepstream but have never have faced a problem.
What seems to be a problem with gstreamer plugins, is compatibility an issue?

11

Maybe, is it possible to share us a repo?

Thanks!

12

Hi @mchi,
Sorry, could not share any repo due to my employer’s policy. But the pipeline described above are as it is, with all the plugins in the same order as given. I have used all Nvidia provided models for these particular testings of pipelines. I can provide particular config for plugin if required. The pipelines worked just by replacing x264enc with nvv4l2h264enc.
Thanks.