Pipelines with new `nvstreammux` and `nvstreamdemux` fail to play correctly in DS 6.3

[prominence_ai]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
Jetson
• DeepStream Version
6.3
• JetPack Version (valid for Jetson only)
JetPack 5.1.2 GA
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
Pipelines with new nvstreammux and nvstreamdemux are failing to play correctly after moving to DeepStream 6.3
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

I’ve tried to reduce my application to the simplest of use cases as follows,

uridecode-0 -> sink_pad_0 |                        | src_pad_0 -> 3d-sink
                          | new-streammux -> demux |
uridecode-1 -> sink_pad_1 |                        | src_pad_1 -> 3d-sink

However, the sources, streammux, demux, and sinks plugins are setup within a hierarchy of gst-bins… and it is difficult for me to remove this extra complexity.

The failure seems to be arbitrary but most of the time one of the demuxer branches fails to negotiate caps correctly. Sometimes it works correctly, other times both branches fail.

Here’s a zip file of the png files show below
png-files.zip (2.1 MB)

---

Here’s the png image of the failing case (one branch)

Here’s the log of a failing case
new_streammux_failing.log (735.4 KB)
Close up image of the demuxer - failing case

---

Here’s the png file of the working case

Here’s the log of a working case
new_streammux_working.log (626.9 KB)
Close up image of the demuxer - working case

---

I’ve tried with and without Streammux configuration files… although, with identical file sources, I would think the default streammux settings would be fine.

If I revert to the old Streammux this Pipeline runs correctly… and I’m unable to reproduce any failure.

I don’t recall having any problems in DeepStream 6.2, but I no longer have a 6.2 setup to confirm on.

Has anyone else experienced similar issues? Perhaps I’m doing something wrong and have just been getting away with it using the old streammux?

Regards,
Robert.

[Fiona.Chen]

I’ve tried the following pipeline on Jetson with DeepStream 6.2, it works.

export USE_NEW_NVSTREAMMUX=yes

gst-launch-1.0 uridecodebin uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h265.mp4 ! mux.sink_0 nvstreammux batch-size=2 name=mux ! queue ! nvstreamdemux name=demux demux.src_0 ! queue ! nv3dsink uridecodebin uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h265.mp4 ! mux.sink_1 demux.src_1 ! nv3dsink

[prominence_ai]

@Fiona.Chen this command fails for me about 20% of the time. The Pipeline transitions to the ready state and freezes with one window rendered. Here’s the Pipeline graph as it’s stuck in the ready state.

Adding a second queue for the second 3dsink seems to fix it.

gst-launch-1.0 uridecodebin uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h265.mp4 ! mux.sink_0 nvstreammux batch-size=2 name=mux ! queue ! nvstreamdemux name=demux demux.src_0 ! queue ! nv3dsink uridecodebin uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h265.mp4 ! mux.sink_1 demux.src_1 ! queue ! nv3dsink

But this doesn’t seem to be the issue I’m running into with my application. I’m just wondering if it has anything to do with the GstBin hierarchy that I’m using? I guess I’m going to have to try and reproduce this with a simple C example.

Thanks for the reply.

[Fiona.Chen]

No. It is not the bin’s problem. It seems some issue with new nvstreammux on DeepStream 6.3. I can reproduce the freeze with the pipeline on DeepStream 6.3 and DeepStream 6.4. We will investigate this issue.

[prominence_ai]

My issue seems a little different… It is not a freeze but a failure to negotiate the caps correctly. It’s pretty clear from this image.

[lukaslorenc.sk]

• Hardware Platform (Jetson / GPU)
GPU
• DeepStream Version
6.4 with docker 6.4-gc-triton-devel
• JetPack Version (valid for Jetson only)

• TensorRT Version
as in docker image 6.4-gc-triton-devel
• NVIDIA GPU Driver Version (valid for GPU only)
535.161.07
• Issue Type( questions, new requirements, bugs)
Same issue cannot play pipeline with new nvstreammux in DS 6.2 it is running correctly
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

gst-launch-1.0 videotestsrc do-timestamp=true ! "video/x-raw,width=640,height=480,format=RGBA,framerate=5/1" ! nvvideoconvert ! "video/x-raw(memory:NVMM),format=NV12,width=640,height=480" ! m.sink_0 nvstreammux name=m nvstreammux.config batch-size=32 sync-inputs=false ! fakesink async=false sync=false

Config:

[property]
overall-max-fps-n=10
overall-max-fps-d=1
overall-min-fps-n=6
overall-min-fps-d=1
max-same-source-frames=2
adaptive-batching=1
max-fps-control=0

Output:
It will freeze on 0:00:00.2 and also hangs on statechange to NULL

max_fps_dur 8.33333e+06 min_fps_dur 2e+08
Setting pipeline to PAUSED ...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
max_fps_dur 1e+08 min_fps_dur 1.66667e+08
Redistribute latency...
0:00:00.2 / 99:99:99.

Do you have any hotfix for this issue?

[lukaslorenc.sk]

Well I just tried multiple times with the provided gst-launch command in docker image 6.4-gc-triton-devel and it clearly does not work did you tried to reproduce the issue with docker image 6.4-gc-triton-devel?

[Fiona.Chen]

It is fixed internally. Please wait for the new release.

[lukaslorenc.sk]

OK thank you. Can you share patch/hotfix? Can you disclose when the next release date is going to be?

[Fiona.Chen]

@lukaslorenc.sk

Can you share patch/hotfix? Can you disclose when the next release date is going to be?

Sorry, we are not allowed to provide such things in the forum.