Runtime add/delete error with nvstreamdemux

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): 3090
• DeepStream Version: 6.2
• TensorRT Version: 8.5.2
• NVIDIA GPU Driver Version (valid for GPU only): 525.89.02
• Issue Type( questions, new requirements, bugs): questions

I want to develop a runtime source re-add application with nvstreamdemux based on the official deepstream_test_rt_src_add_del.c. I build a simple demo for two sources and the pipeline shows below:

runtime_add
runtime_add13898×1154 837 KB

The pipeline use new streammux. The demo logic is that when one source stream (like source_id 0) is over and EOS is captured in bus_call, sink_0 pad of streammux is released in stop_release_source, then new source and sink_0 pad of streammux are created in add_sources. But the newly created source is not handled and throws the error

[ERROR push 333] push failed [-3]

I do the following tests:

  • without nvstreamdemux
    The demo runs well

  • unlink (not remove) src_0 of demux and relink in add_sources
    The problem is the same

The demo source code:
runtime_add.txt (28.6 KB)

3

Can you upgrade to the latest DeepStream 7.0 version?

4

Thank you for your attention.

Since the tracker engine in my real application is currently developed on DS 6.0, there is currently no plan to upgrade to DS7.0.

But I have tested the demo code before on DS 7.0 (with the image deepstream:7.0-gc-triton-devel). The error is gone but new source is continuously created and deleted.

frame_num: 359

frame_num: 360

decodebin new pad video/x-raw

frame_num: 361

Decodebin linked to pipeline

** Message: 08:28:13.672: get message from element decodebin2 : missing-plugin

WARNING from element source-bin-00: No decoder available for type ‘audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2’.

Warning: No decoder available for type ‘audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2’.

** Message: 08:28:14.419: get message from element stream-muxer : stream-eos

Got EOS from stream 1

STATE CHANGE SUCCESS

STATE CHANGE SUCCESS 0x7fbfd4005020

Calling Start 1

creating uridecodebin for [file:///workspace/runtime_source_add_delete/sample_1080p_h265.mp4]

decodebin child added source

decodebin child added decodebin3

STATE CHANGE ASYNC

decodebin child added qtdemux3

decodebin child added multiqueue3

decodebin child added h265parse1

decodebin child added capsfilter3

decodebin child added nvv4l2decoder3

decodebin new pad video/x-raw

Decodebin linked to pipeline

** Message: 08:28:14.540: get message from element decodebin3 : missing-plugin

WARNING from element source-bin-01: No decoder available for type ‘audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2’.

Warning: No decoder available for type ‘audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2’.

** Message: 08:28:15.297: get message from element stream-muxer : stream-eos

Got EOS from stream 1

STATE CHANGE SUCCESS

STATE CHANGE SUCCESS 0x7fbfd4431160

Calling Start 1

creating uridecodebin for [file:///workspace/runtime_source_add_delete/sample_1080p_h265.mp4]

decodebin child added source

decodebin child added decodebin4

STATE CHANGE ASYNC

decodebin child added qtdemux4

decodebin child added multiqueue4

decodebin child added h265parse2

decodebin child added capsfilter4

decodebin child added nvv4l2decoder4

decodebin new pad video/x-raw

Decodebin linked to pipeline

** Message: 08:28:15.420: get message from element decodebin4 : missing-plugin

WARNING from element source-bin-01: No decoder available for type ‘audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2’.

Warning: No decoder available for type ‘audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2’.

** Message: 08:28:16.179: get message from element stream-muxer : stream-eos

Got EOS from stream 1

STATE CHANGE SUCCESS

The whole log
1.log (42.1 KB)

5

What is your issue with such result?

6

I would like to seek advice on modifications I need to make for the purpose of runtime add/delete with nvstreamdemux, or the reason why it cannot be used in this way.

The log on DS7.0 shows the new created stream is not processed at all and EOS is immediately reached. The new source is then repeatedly added and deleted.

7
  1. From your code, it is no meaning to unlink the sink elements from the nvstreamdemux since when you send eos through the nvstreammux sink pad, the nvstreamdemux will release the corresponding sink, and the eos should be handled by the nvstreamdemux and the sink elements themselves to make the context be cleared correctly.
  2. What you should do is just to check and record the status of the nvstreademux and create new src pad only when you are sure the pad is free and the corresponding source has been connected.
  3. When there is no source in the pipeline, the pipeline state should be changed to NULL and the pipeline should be restarted after the new source is created and connected.
8

Much appreciate, I will try your advice later.

9

I have implemented runtime add/delete with nvstreammux for one stream. I post my method for others to refer to.

  1. when one stream is over, remove src_0 of nvstreamdemux, remove elements after nvstreamdemux.
  2. recreate elements after nvstreamdemux before new added source_bin and request src_0 of nvstreamdemux.

The problem encountered is that when I feed 1000 videos one by one, the program’s memory is gradually increased, and finally increased by 40M, resulting in a memory leak.