Smart Record can record audio in DeepStream 6.0.1, but can't record audio in DeepStream 6.1.1

ta.sugiyama · March 14, 2023, 9:53am

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) dGPU
• DeepStream Version 6.1.1
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only) 515.65.01
• Issue Type( questions, new requirements, bugs) bugs
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

SmartRecord can record audio in DeepStream 6.0.1,
but can’t record audio in DeepStream 6.1.1

How to reproduce the issue

prepare rtsp

docker run --rm --name rtspserver -itd --network=host -e RTSP_PROTOCOLS=tcp aler9/rtsp-simple-server
ffmpeg -re -stream_loop -1 -i sample_1080p_h264.mp4 -c:v libx264 -r 30 -bf 0 -g 1 -c:a copy -f rtsp -rtsp_transport tcp rtsp://localhost:8554/mystream

DS 6.0.1 works

nvidia-docker run --rm -it nvcr.io/nvidia/deepstream:6.0.1-triton
cd /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-testsr/
CUDA_VER=11.4 make -j
URI=rtsp://127.0.0.1:8554/mystream
./deepstream-testsr-app $URI --bbox-enable=0 --enc-type=1 --sink-type=1 --sr-mode=0

Log is like below and the output mp4 contains video+audio

Running...
In cb_newpad
In cb_newpad
Recording started..
Recording done.
Recording started..

DS6.1.1 try1 does not work

nvidia-docker run --rm -it nvcr.io/nvidia/deepstream:6.1.1-triton
cd /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-testsr/
CUDA_VER=11.7 make -j
URI=rtsp://127.0.0.1:8554/mystream
./deepstream-testsr-app $URI --bbox-enable=0 --enc-type=1 --sink-type=1 --sr-mode=0

The log is like below.

Running...
Recording started..
In cb_newpad
In cb_newpad
ERROR from element audio-parser-pre-recordbin: Your GStreamer installation is missing a plug-in.
Error details: gstparsebin.c(3486): gst_parse_bin_expose (): /GstPipeline:dstest-sr-pipeline/GstParseBin:audio-parser-pre-recordbin:
no suitable plugins found:
Missing parser: MPEG-4 AAC (audio/mpeg, mpegversion=(int)4, stream-format=(string)raw, codec_data=(buffer)1190)

Returned, stopping playback
Deleting pipeline

DS 6.1.1 try2 does not work (apt install then run the binary)

apt-get install --reinstall --no-install-recommends -y libavcodec58 libavcodec-dev libavformat58 libavformat-dev libavutil56 libavutil-dev gstreamer1.0-libav
./deepstream-testsr-app $URI --bbox-enable=0 --enc-type=1 --sink-type=1 --sr-mode=0

The output mp4 contains video but does not contain audio.
The log is like below.

Running...
In cb_newpad
In cb_newpad
Recording started..
Recording done.
** ERROR: <RunUserCallback:207>: No video stream found
** ERROR: <RunUserCallback:212>: No audio stream found
Recording started..

DS 6.1.1 try3

I changed container type from mp4 to mkv but the output mkv does not contain audio.

Running...
In cb_newpad
In cb_newpad
Recording started..
Recording done.
** ERROR: <RunUserCallback:212>: No audio stream found
Recording started..

Thanks.

yuweiw · March 14, 2023, 11:57am

Could you update the DeepStream Version to 6.2 and have a try?

ta.sugiyama · March 15, 2023, 1:46am

@yuweiw

I tested DS6.2 and could not record (tried try1/try2).
ds62-try2.txt (3.8 KB)
ds62-try1.txt (5.1 KB)

# ./deepstream-testsr-app $URI --bbox-enable=0 --enc-type=1 --sink-type=1 --sr-mode=0

(gst-plugin-scanner:44): GStreamer-WARNING **: 00:10:01.911: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:44): GStreamer-WARNING **: 00:10:01.959: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstopenmpt.so': libmpg123.so.0: cannot open shared object file: No such file or directory
(gst-plugin-scanner:44): GStreamer-WARNING **: 00:10:02.018: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstmpeg2enc.so': libmpeg2encpp-2.1.so.0: cannot open shared object file: No such file or directory
(gst-plugin-scanner:44): GStreamer-WARNING **: 00:10:02.107: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstmpeg2dec.so': libmpeg2.so.0: cannot open shared object file: No such file or directory
(gst-plugin-scanner:44): GStreamer-WARNING **: 00:10:02.111: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstmpg123.so': libmpg123.so.0: cannot open shared object file: No such file or directory
(gst-plugin-scanner:44): GStreamer-WARNING **: 00:10:02.131: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstchromaprint.so': libavcodec.so.58: cannot open shared object file: No such file or directory

I found that

The log tell that it failed to load plugin libmpg123.so.0, libmpeg2encpp-2.1.so.0, libmpeg2.so.0, libmpg123.so.0, libavcodec.so.58due to missing shared library
smartrecord tells missing parser:Missing parser: MPEG-4 AAC (audio/mpeg, mpegversion=(int)4, stream-format=(string)raw, codec_data=(buffer)1190)
I could not load mpegaudioparse by gst-inspect-1.0

After I run below apt command, video+audio record sometimes success and sometimes fail

apt-get install --reinstall --no-install-recommends -y libavcodec58 libavcodec-dev libavformat58 libavformat-dev libavutil56 libavutil-dev gstreamer1.0-libav
apt install --reinstall gstreamer1.0-plugins-good
apt install --reinstall libvpx6 libx264-155 libx265-179 libmpg123-0 libmpeg2-4 libmpeg2encpp-2.1-0

ds62-ok.txt (2.0 KB)
ds62-ng-sometimes.txt (2.1 KB)

This behavior is the same in DespStream6.1.1, after the above command.
Error log is like below (See attached file for detail). This happens when cb_newpad_audio_parsebin() try to link to asink of recordbin.

(deepstream-testsr-app:1438): GStreamer-CRITICAL **: 01:45:09.635: gst_pad_link_full: assertion 'GST_IS_PAD (sinkpad)' failed
Elements not linked. Exiting.

My question is that

Is it OK to reinstall gstreamer1.0-plugins-good? (DeepStream may be intented to use modified version of gst-plugins-good)
Is missing library in container considered a bug?
How to fix “Elements not linked” error? I suspect it’s a problem inside recordbin.
When record success, I see the below warning. What is it?

 WARN: <audio_queue_src_pad_probe:301>: Invalid PTS found in stream. Stream recorded by Smart Record might have issues

ta.sugiyama · March 15, 2023, 2:59am

“Elements not linked” often happens in DeepStream 6.2 rather than 6.1.1

GST_DEBUG=3 shows some warning.

Running...
Recording started..
In cb_newpad
0:00:08.154329034   421 0x56467c0a6de0 FIXME               basesink gstbasesink.c:3246:gst_base_sink_default_event:<nvvideo-renderer> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:08.162414819   421 0x56467c0a69e0 FIXME               basesink gstbasesink.c:3246:gst_base_sink_default_event:<src_filesink0> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:08.162549910   421 0x56467c0a69e0 WARN                   qtmux gstqtmux.c:2981:gst_qt_mux_start_file:<mux_elem0> Robust muxing requires reserved-moov-update-period to be set
In cb_newpad
cb_newpad_audio_parsebin() Link audio start
0:00:08.262805423   421 0x7f79b4062060 WARN                   qtmux gstqtmux.c:6367:gst_qt_mux_request_new_pad:<mux_elem0> Not providing request pad after stream start.

(deepstream-testsr-app:421): GStreamer-CRITICAL **: 02:58:56.330: gst_pad_link_full: assertion 'GST_IS_PAD (sinkpad)' failed
Elements not linked. Exiting.

yuweiw · March 15, 2023, 8:00am

OK, I can repro your problem in my Deepstream 6.2 env. We’ll check and fix it. Please look forward to the next version. Thanks

system · April 3, 2023, 8:01am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.