Dynamically Added NvStreamDemux doesn't work

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 6.1.1
• TensorRT Version 8.4.3.1
**• NVIDIA GPU Driver Version 515.105.01
• Issue Type( questions, new requirements, bugs) bug
**• How to reproduce the issue ? Dynamically add an nvstreamdemux to a running pipeline (use block pad probes), and request src_0 pad, you will find that no data comes out it

Dynamically added nvstreamdemux used to work fine in DS 6.0 but didn’t work in DS 6.1.1 when added dynamically yet it works fine it is added at the beginning. I think it doesn’t negotiate/set the src caps properly in such case.

Find below a log file for running this pipeline:

GST_DEBUG=5 gst-launch-1.0 -e nvstreammux name=mux batch-size=1 width=1920 height=1080 ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt batch-size=1 ! nvstreamdemux name=nvdemux filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! queue ! mux.sink_0  nvdemux.src_0 ! "video/x-raw(memory:NVMM), format=NV12" !  queue ! nvvideoconvert  ! nvdsosd ! nvvideoconvert ! nveglglessink async=0

init-with-nvstreamdemux-log.txt (34.2 KB)

Here’s the log file when it’s added dynamically in our app:
runtime-nvstreamdemux-addition-log.txt (12.0 KB)

We also tried to start the pipeline with an already existing nvstreamdemux and it worked but the dynamically added one didn’t work and the upcoming graph shows a caps issue.

Best Regards,
Eslam

Could you attach your demo code for dynamically added NvStreamDemux? Also could you try it with our latest DeepStream version 6.2?

1 Like

Kindly, Find the demo code attached below tested against DS 6.1.1:
The code adds at most one additional source to the pipeline as well as a new branch containing nvstreamdemux.
runtime_source_add_delete.zip (80.1 KB)

Also here’s the full graph, you will find that the src caps of nvstreamdemux is not properly negotiated as showed earlier.
pipeline.dot (44.5 KB)

I would like to hear your opinion about this before testing against DS 6.2

Best Regards,
Eslam

What’s your command like? I use ./deepstream-test-rt-src-add-del rtsp://xxx 0 filesink 1 to run the app attached, it’s not running properly.

Sorry for the late reply. the input supported is a h264 elementary stream video file uri.
So you can use the following cli args ./deepstream-test-rt-src-add-del file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 0 nveglglessink 0

FYI: the code sample was originally found here

I know you are referring to our source code, and the source code runs properly in our docker. But when I use the code you attached(the zip file) with the command line: ./deepstream-test-rt-src-add-del file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 0 nveglglessink 0. It cannot run properly and it’s not the issue about nvstreamdemux.
Is the code you attached that you actually used?
Additionally, from the code you attached. You don’t need to create multiple nvstreamdemux, you just need to create one plugin and add new “src_%u” pads on it.

What do you mean by it doesn’t run properly? what log did you see? What version did you run against?

Also I am not creating multiple nvstreamdemux. The only one being created is the one being added in add_sources at runtime and the total sources set to 2.

The app starts by showing 1 stream, a few seconds later, another source is added as well as the branch in which the nvstreademux is present which is later attached to a different nveglglessink. Problem is that this sink never receives any frames. This is shown by the caps being no set/negotiated in the fully attached graph below.

pipeline
pipeline.dot (44.5 KB)

Yes, it’s the same code, tested against DS 6.1.1

To Clarify the use case, it’s an on-demand separation of a particular stream at runtime so that it can be served over a udpsink and later by an rtsp server.

Because we will not provide patches separately on previous DeepStream version, I run that on DeepStream6.2. It reports Fatal Error in nvmultistreamtiler plugin.

Error Log

0:00:18.795157977 1970 0x55c3df0c1400 WARN nvmultistreamtiler gstnvtiler.cpp:739:gst_nvmultistreamtiler_transform: error: GstNvTiler: FATAL ERROR; NvTiler::Composite failed
ERROR from element nvtiler: GstNvTiler: FATAL ERROR; NvTiler::Composite failed
Error details: gstnvtiler.cpp(739): gst_nvmultistreamtiler_transform (): /GstPipeline:dstest-pipeline/GstNvMultiStreamTiler:nvtiler
[NvTiler::ScaleAndPlaceAllMeta] ERROR: 175; scaleX=0.019271 scaleY=0.000000 frameMeta=0x7f3e040211f0
[NvTiler::Composite] ERROR: 343; ScaleAndPlaceAllMeta failed (0x7f3e8c001940;0x7f3e8c001c30)

I have tried the same code on DeepStream 6.0, it runs properly. We’ll debug the reasons why the latest version does not run properly. Thanks

1 Like

Hi @eslam.ahmed, I modified your source code: tiler_rows = 1; tiler_columns=MAX_NUM_SOURCES. It work well with DeepStream 6.2. So we suggest you use the DeepStream 6.2 and set the right parameters for nvmultistreamtiler plugin. Thanks

Again there is no issue with the nvmultistreamtiler Who said that there was one to begin with?

I am talking about the second branch being added in the get_branch_bin method which as the nvstreamdemux and the additional nveglglessink. The expected output of this app is to see 2 separate nveglglessink windows and not one!

Also, I have tested against Deepstream 6.2 and the issue persists!

Note: the 2 screenshots above a part of the same pipeline and are generated from the code sample uploaded!

Thanks!

OK. We’ll analyze the issue of nvstreamdemux not being able to linked correctly in this scenario. And we’ll updated promptly once it is resolved. Thanks

1 Like

@yuweiw We are also stuck on the same. Is there any update on this issue or a possible workaround?

@naveen.pnd Not yet. We are analyzing it, and will reply promptly if there is a conclusion.

We have recently tested against DeepStream 6.3 and the issue still persists.