Audio pipeline got stuck after EOS

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (GPU)
• DeepStream Version 6.1

I got this problem need your help, I am still debugging but cannot fix by myself so far

I am running a deepstream pipeline with video and audio inferencing (like image below)
image

I use uridecodebin and loop video with seek_decode (using DS source code). I use 2 different videos, one is 10 secs, the other is 1 min for example. When 10-sec video got EOS, it can loop but it causes only audio pipelines of other cameras (1-min video) stuck, video pipelines keep running normally.

I have to set sync to true to sync audio and video pipeline
audio_sink = gst_element_factory_make (“fakesink”, “audio_fakesink”);
g_object_set (G_OBJECT (audio_sink), “sync”, TRUE, “async”, TRUE, NULL);

Do you have any ideas? Thank you so much.

3

1.Can you update to DS-7.0 ? Maybe some problems have fixed on latest version.

2.If you want loop the file. you can try nvurisrcbin and set property of file-loop as true.

uri_decode_bin = gst_element_factory_make ("nvurisrcbin", "uri-decode-bin");
 g_object_set (G_OBJECT (uri_decode_bin), "file-loop", TRUE, NULL);

3.Have you set the drop-pipeline-eos property of nvstreammux to true?
Can you provide sample code to reproduce the problem?

This is necessary, no problem.

1 Like
4

I’ve already tried all your suggestions but it seems not work in my case (DS 6.1)

This is my sample code extracted from my product
sample_code.zip (5.7 MB)
command to run:

./sdkds_app file:///test2_10s.mp4 file:///test2.mp4

Could you take a look at my create_uridecodesrc_bin function at line 540, please?

image
image1047×393 67.9 KB

g_signal_connect inside that function at line 658:

image
image1168×87 20.2 KB

This is my audio pipeline running along with video pipeline

image
image1157×143 27.6 KB

After cam0 got EOS, cam1 got stuck with audio pipeline, video pipeline for both cameras still worked well

image
image1854×837 196 KB

I really need your help. Thank you so much.

5

After some modifications I can run your code fine

1.The wrong buffer size here will cause stack overflow, so it needs to be increased to 32 bytes.

GstCaps *caps = NULL, *caps1 = NULL;
gchar bin_name[32] = { };
int decoder_type = TYPE_DECODER_NVV4L2;

2.The value of nvbuf-memory-type uses the default value, which usually does not affect

  //g_object_set (nvvidconv, "gpu-id", GPU_ID, "nvbuf-memory-type", 3, NULL);
  g_object_set (nvvidconv, "gpu-id", GPU_ID, "nvbuf-memory-type", 0, NULL);

I tested it on DS-7.0.


(sdkds_app:27231): GStreamer-WARNING **: 10:55:38.278: ../gst/gstpad.c:4416:gst_pad_chain_data_unchecked:<nvv4l2decoder1:sink> Got data flow before segment event
[pipeline][cam1][f871] Received audio buffer of size 744 bytes
[pipeline][cam1][f872] Received audio buffer of size 742 bytes
[pipeline][cam1][f873] Received audio buffer of size 744 bytes
[pipeline][cam1][f874] Received audio buffer of size 742 bytes
[pipeline][cam1][f875] Received audio buffer of size 744 bytes
[pipeline][cam1][f876] Received audio buffer of size 742 bytes
[pipeline][cam1][f877] Received audio buffer of size 744 bytes
[pipeline][cam0][info] (file:///root/sample_code/test2_10s.mp4) got EOSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS

(sdkds_app:27231): GStreamer-WARNING **: 10:55:38.446: ../gst/gstpad.c:4416:gst_pad_chain_data_unchecked:<nvv4l2decoder1:sink> Got data flow before segment event
[pipeline][cam1][f878] Received audio buffer of size 742 bytes
[pipeline][cam1][f879] Received audio buffer of size 744 bytes
[pipeline][cam0][info] Start seek decode reset video fileeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee ...

(sdkds_app:27231): GStreamer-WARNING **: 10:55:38.486: ../gst/gstpad.c:4416:gst_pad_chain_data_unchecked:<nvv4l2decoder0:sink> Got data flow before segment event
[pipeline][cam0][f880] Received audio buffer of size 680 bytes
[pipeline][cam0][f881] Received audio buffer of size 742 bytes
[pipeline][cam1][f882] Received audio buffer of size 742 bytes
[pipeline][cam0][f883] Received audio buffer of size 744 bytes
[pipeline][cam0][f884] Received audio buffer of size 742 bytes
[pipeline][cam0][f885] Received audio buffer of size 744 bytes
[pipeline][cam0][f886] Received audio buffer of size 742 bytes
[pipeline][cam0][f887] Received audio buffer of size 744 bytes

(sdkds_app:27231): GStreamer-WARNING **: 10:55:38.614: ../gst/gstpad.c:4416:gst_pad_chain_data_unchecked:<nvv4l2decoder1:sink> Got data flow before segment event
[pipeline][cam0][f888] Received audio buffer of size 742 bytes
[pipeline][cam0][f889] Received audio buffer of size 744 bytes
[pipeline][cam0][f890] Received audio buffer of size 742 bytes
1 Like
6

Thank for your suggestion, but as I check

  1. I don’t see any cam1 printed after EOS signal, could you show more log?
    From my side,

    image
    image1852×965 227 KB

  2. Base docker image does not support audio decode, so how do you install the environment? For me, I have to install ffmpeg and gstreamer1.0-plugins-bad

7

do you have any updates?

8

In the docker image, there is a script called /opt/nvidia/deepstream/deepstream/user_additional_install.sh. This will help you install additional plugins.

Since I don’t know how you modified deepstream_source_bin.c, I didn’t find out the problem.

1 Like
9

currently I do not modify any line of code in deepstream_source_bin.c as you see I don’t include it to build executable file in CMakeLists.txt

10

Most of the source bin in your project is copied from deepstream_source_bin.c.I think it might be due to the difference.

1 Like
11

Could you modify something with my custom source bin to make it work?
And do you think problems come from current DS version with no support for my case?

Thank you so much.

12

I still ran into this problem.
Have you known the reason yet?

13

Haven’t found the problem yet. If you use deepstream-app, do you have similar problems?

14

totally using deepstream-app is not my use case, so it does not really help. Thank you so much.

15

I will look into it when I have time. There are too many topics need to deal with.

By the way, why deepstream-app cannot match your requriements ? Maybe we can improve it in subsequent versions.

1 Like
16

coz I dont see any config setup for running both video and audio with looping