Audio recording with video

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hi

I am using the below pipeline to encode a 1080p video stream with audio and preview.

gst-launch-1.0 -e v4l2src device=/dev/video0 io-mode=2 do-timestamp=true ! tee name=t2 t2. ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,width=1920,height=1080,framerate=30/1 ! nvvidconv ! queue ! "video/x-raw(memory:NVMM),width=1920,height=1080,format=NV12" ! nvv4l2h264enc maxperf-enable=1 bitrate=4000000 profile=4 ! queue ! h264parse ! queue ! mux. alsasrc device="hw:2,0" ! tee name=a1 ! queue ! audio/x-raw ! queue ! audioresample ! "audio/x-raw,rate=44100" ! queue ! voaacenc ! queue ! mp4mux name=mux ! filesink location=encoded.mp4 t2. ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,width=1920,height=1080,framerate=30/1 ! nvvidconv ! queue ! "video/x-raw(memory:NVMM),width=1920,height=1080,format=NV12" ! queue ! nvvidconv ! nvoverlaysink

The above pipeline gets stuck with the hdmi to usb conveter device used. But work fine with a logitech webcam.

Above pipeline also works fine without the audio capture part with the hdmi to usb converter used. When audio is added only the pipeline gets stuck and I am also using jetson nano in MAXN mode with ‘jetson_clocks’ on

What can be the reason for this?

My requirement is to record 2 1080p 30fps MJPG streams with audio using jetson nano.

Thanks in advanced

3

Hi,
You may break down the pipeline to try video-only and audio-only commands. To clarify which part triggers the issue. And the MJPEG stream is decoded twice. May try to put tee after nvv4l2decoder mjpeg=1:

gst-launch-1.0 -e v4l2src device=/dev/video0 io-mode=2 do-timestamp=true  ! image/jpeg,width=1920,height=1080,framerate=30/1 ! jpegparse ! nvv4l2decoder mjpeg=1 ! tee name=t2 ! queue ! ...
1 Like
4

Hi @DaneLLL ,

I tried the video and audio command separately they are working fine, but noticed that when encoding the audio only from logitech webcam the the cpu usage is low, but when encoding the audio only from othe usb source one of the cpu cores goes to 90% and more.

Also tried placing the tee after decorder but the pipeline did not start. The pipeline worked only when tee is placed just before the decorder (this was teted without the audio from the other usb device)

5

Hi,
Probably the device is not in card 2, device 0. Please run arecord -l to confirm the index.

May also remove do-timestamp=true for a try.

1 Like