Video and Audio Recording

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi,
Using the below cmd, I am able to record the video and audio in same time.

gst-launch-1.0 v4l2src ! ‘video/x-raw, width=1920, height=1080, framerate=30/1, format=UYVY’ ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=I420’ ! queue ! omxh264enc bitrate=8000000 ! h264parse ! matroskamux name=mux ! filesink location=oct22_1230pm.mkv -e alsasrc ! ‘audio/x-raw, format=(string)S16LE, layout=(string)interleaved, channels=(int)2, channel-mask=(bitmask)0x0000000000000003, rate=(int)48000’ ! queue ! audioconvert ! lamemp3enc ! mux.

But in Audio, I am getting the noise.

The log is:

gst-launch-1.0 v4l2src ! ‘video/x-raw, width=1920, height=1080, framerate=30/1, format=UYVY’ ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=I420’ ! queue ! omxh264enc bitrate=8000000 ! h264parse ! matroskamux name=mux ! filesink location=oct22_1230pm.mkv -e alsasrc ! ‘audio/x-raw, format=(string)S16LE, layout=(string)interleaved, channels=(int)2, channel-mask=(bitmask)0x0000000000000003, rate=(int)48000’ ! queue ! audioconvert ! lamemp3enc ! mux.
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstAudioSrcClock
Redistribute latency…
Redistribute latency…
Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
===== MSENC blits (mode: 1) into tiled surfaces =====
WARNING: from element /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: Can’t record audio fast enough
Additional debug info:
gstaudiobasesrc.c(869): gst_audio_base_src_create (): /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0:
Dropped 205920 samples. This is most likely because downstream can’t keep up and is consuming samples too slowly.

-Thanks.

2

Are you able to get good audio via wavenc?

gst-launch-1.0 alsasrc num-buffers=1000 ! wavenc ! filesink location=a.wav
3 
gst-launch-1.0 alsasrc num-buffers=1000 ! wavenc ! filesink location=a.wav

seems to record good audio.

Could you provide the overall sequence to combine audio + video → to a mkv file, simultaneously?

4

Hi Andrey1984,
Here is a command for your reference:

$ gst-launch-1.0 nvcamerasrc num-buffers=300 ! omxh264enc ! queue ! mux. alsasrc num-buffers=1000 ! voaacenc ! queue ! mux. qtmux name=mux ! filesink location=a.mp4

Is generates mp4. You should change accordingly to get mkv.

5

DaneLLL,
Thank you for your response.

gst-launch-1.0 nvcamerasrc num-buffers=300 ! omxh264enc ! queue ! mux. alsasrc num-buffers=1000 ! voaacenc ! queue ! mux. qtmux name=mux ! filesink location=a.mp4
WARNING: erroneous pipeline: no element "voaacenc"

It seems to require “gstreamer element faac” / voaacenc that belongs to bad-plugins 0.10
I shall try build GitHub - GStreamer/gst-plugins-bad: 'Bad' GStreamer plugins and helper libraries. This module has been merged into the main GStreamer repo for further development.

apt-get install gstreamer1.0-plugins-bad

gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvtee ! nvvidconv flip-method=0 ! omxh264enc ! queue ! mux. alsasrc num-buffers=1000 ! voaacenc ! queue ! mux. qtmux name=mux ! filesink location=a.mp4 -e

will the following be the best way to record with the devboad onboard sensor?
 
[code] gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM), width=(int)2592, height=(int)1944, format=(string)I420, framerate=(fraction)30/1' ! nvtee ! nvvidconv flip-method=0 ! omxh264enc ! queue ! mux. alsasrc num-buffers=1000 ! voaacenc ! queue ! mux. qtmux name=mux ! filesink location=a.mp4 -e
Setting pipeline to PAUSED ...

Available Sensor modes : 
2592 x 1944 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
2592 x 1458 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1280 x 720 FR=120.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 0 WxH = 2592x1944 FrameRate = 30.000000 ...

New clock: GstAudioSrcClock
Redistribute latency...
Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4 
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
NvH264MSEncSetCommonStreamAttribute: LevelIdc conformance violation
NvH264MSEncSetCommonStreamAttribute: LevelIdc conformance violation
===== MSENC blits (mode: 1) into tiled surfaces =====
^Chandling interrupt.
Interrupt: Stopping pipeline ...
EOS on shutdown enabled -- Forcing EOS on the pipeline
Waiting for EOS...
Got EOS from element "pipeline0".
EOS received - stopping pipeline...
Execution ended after 0:00:18.452010195
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Audio seems to fail on 2 cases of 3

$ gst-launch-1.0 nvcamerasrc num-buffers=300 ! omxh264enc ! queue ! mux. alsasrc num-buffers=1000 ! voaacenc ! queue ! mux. qtmux name=mux ! filesink location=a.mp4

seems to abrupt on 10 seconds 7 times of 10.

Any idea how to debug audio loss in most of recordings?

What causes

Got EOS from element "pipeline0".
Execution ended after 0:00:10.221296455

Reference:
https://devtalk.nvidia.com/default/topic/973277/jetson-tx1/how-to-quit-nvgstcapture-in-full-screen-mode/post/5005055/#5005055

6

Hi Andrey, is the issue seen with audiotestsrc?

7

Thank you for your question.
It seems I shall find a way to execute audiotestsrc. It seems to be a plugin of gstreamer. Doing google search on how to execute it…

gst-launch-1.0 audiotestsrc
Setting pipeline to PAUSED ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
ERROR: from element /GstPipeline:pipeline0/GstAudioTestSrc:audiotestsrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstAudioTestSrc:audiotestsrc0:
streaming task paused, reason not-linked (-1)
Execution ended after 0:00:00.000109698
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
8

Q --“is the issue seen with audiotestsrc?”
A: The issue is rather seen with audiotestsrc.
Update: the issue has been resolved.