Multiple video and audio recording

Autonomous Machines Jetson & Embedded Systems Jetson TX2
, ,
1

Hey,

I have 3 cameras and one mic connected.

I want to record videos from all 3 cameras to HLS, but the first one to capture and mix in the audio from the mic as well.

When I run the following pipeline (video only, no audio) everything works well:

nvarguscamerasrc sensor_id=0 sensor-mode=0 tcp-port=85540 gamma=0.45 ee-mode=0 awblock=0 aelock=0 exposuretimerange=“500000 12500000” exposurecompensation=0.0 gainrange=“1 22” alc-mode=0 ispdigitalgainrange=“1 1” ! video/x-raw(memory:NVMM), width=4056, height=3040, format=(string)NV12 ! nvvidconv interpolation-method=4 top=272 bottom=2768 left=76 right=3980 ! video/x-raw(memory:NVMM), width=3216, height=2064, framerate=30/1, format=NV12 ! queue ! nvv4l2h264enc maxperf-enable=true control-rate=1 bitrate=50000000 iframeinterval=30 insert-sps-pps=true ! video/x-h264, stream-format=(string)byte-stream ! queue ! h264parse ! queue ! mpegtsmux name=muxcam0 ! hlssink location=/user/recordings/%06dcam0.ts target-duration=5 nvarguscamerasrc sensor_id=1 sensor-mode=0 tcp-port=85550 gamma=0.45 ee-mode=0 awblock=0 aelock=0 exposuretimerange=“500000 12500000” exposurecompensation=0.0 gainrange=“1 22” alc-mode=0 ispdigitalgainrange=“1 1” ! video/x-raw(memory:NVMM), width=4056, height=3040, format=(string)NV12 ! nvvidconv interpolation-method=4 top=272 bottom=2768 left=76 right=3980 ! video/x-raw(memory:NVMM), width=3216, height=2064, framerate=30/1, format=NV12 ! queue ! nvv4l2h264enc maxperf-enable=true control-rate=1 bitrate=50000000 iframeinterval=30 insert-sps-pps=true ! video/x-h264, stream-format=(string)byte-stream ! queue ! h264parse ! queue ! mpegtsmux ! hlssink location=/user/recordings/%06dcam1.ts target-duration=5 nvarguscamerasrc sensor_id=2 sensor-mode=0 tcp-port=85560 gamma=0.45 ee-mode=0 awblock=0 aelock=0 exposuretimerange=“500000 12500000” exposurecompensation=0.0 gainrange=“1 22” alc-mode=0 ispdigitalgainrange=“1 1” ! video/x-raw(memory:NVMM), width=4056, height=3040, format=(string)NV12 ! nvvidconv interpolation-method=4 top=272 bottom=2768 left=76 right=3980 ! video/x-raw(memory:NVMM), width=3216, height=2064, framerate=30/1, format=NV12 ! queue ! nvv4l2h264enc maxperf-enable=true control-rate=1 bitrate=50000000 iframeinterval=30 insert-sps-pps=true ! video/x-h264, stream-format=(string)byte-stream ! queue ! h264parse ! queue ! mpegtsmux ! hlssink location=/user/recordings/%06dcam2.ts target-duration=5

But when I add audio in camera 0 (sensor_id 0, 85540) using the following pipeline, I get errors:

nvarguscamerasrc sensor_id=0 sensor-mode=0 tcp-port=85540 gamma=0.45 ee-mode=0 awblock=0 aelock=0 exposuretimerange=“500000 12500000” exposurecompensation=0.0 gainrange=“1 22” alc-mode=0 ispdigitalgainrange=“1 1” ! video/x-raw(memory:NVMM), width=4056, height=3040, format=(string)NV12 ! nvvidconv interpolation-method=4 top=272 bottom=2768 left=76 right=3980 ! video/x-raw(memory:NVMM), width=3216, height=2064, framerate=30/1, format=NV12 ! queue ! nvv4l2h264enc maxperf-enable=true control-rate=1 bitrate=50000000 iframeinterval=30 insert-sps-pps=true ! video/x-h264, stream-format=(string)byte-stream ! queue ! h264parse ! queue ! mpegtsmux name=muxcam0 ! hlssink location=/user/recordings/%06dcam0.ts target-duration=5 alsasrc device=hw:tegrasndt186ref,1 ! audio/x-raw, format=(string)S32LE, layout=(string)interleaved, rate=(int)48000, channels=(int)1 ! queue ! audioconvert ! voaacenc ! muxcam0. nvarguscamerasrc sensor_id=1 sensor-mode=0 tcp-port=85550 gamma=0.45 ee-mode=0 awblock=0 aelock=0 exposuretimerange=“500000 12500000” exposurecompensation=0.0 gainrange=“1 22” alc-mode=0 ispdigitalgainrange=“1 1” ! video/x-raw(memory:NVMM), width=4056, height=3040, format=(string)NV12 ! nvvidconv interpolation-method=4 top=272 bottom=2768 left=76 right=3980 ! video/x-raw(memory:NVMM), width=3216, height=2064, framerate=30/1, format=NV12 ! queue ! nvv4l2h264enc maxperf-enable=true control-rate=1 bitrate=50000000 iframeinterval=30 insert-sps-pps=true ! video/x-h264, stream-format=(string)byte-stream ! queue ! h264parse ! queue ! mpegtsmux ! hlssink location=/user/recordings/%06dcam1.ts target-duration=5 nvarguscamerasrc sensor_id=2 sensor-mode=0 tcp-port=85560 gamma=0.45 ee-mode=0 awblock=0 aelock=0 exposuretimerange=“500000 12500000” exposurecompensation=0.0 gainrange=“1 22” alc-mode=0 ispdigitalgainrange=“1 1” ! video/x-raw(memory:NVMM), width=4056, height=3040, format=(string)NV12 ! nvvidconv interpolation-method=4 top=272 bottom=2768 left=76 right=3980 ! video/x-raw(memory:NVMM), width=3216, height=2064, framerate=30/1, format=NV12 ! queue ! nvv4l2h264enc maxperf-enable=true control-rate=1 bitrate=50000000 iframeinterval=30 insert-sps-pps=true ! video/x-h264, stream-format=(string)byte-stream ! queue ! h264parse ! queue ! mpegtsmux ! hlssink location=/user/recordings/%06dcam2.ts target-duration=5

The errors:
CONSUMER: 1662452847.723 - Report for camera 0: missing frames: 1, missing time:0.067
CONSUMER: 1662452847.806 - Report for camera 1: missing frames: 1, missing time:0.067
CONSUMER: 1662452847.854 - Report for camera 2: missing frames: 5, missing time:0.200
WARNING | Error: gst-core-error-quark: Can’t record audio fast enough (13): gstaudiobasesrc.c(849): gst_audio_base_src_create (): /GstPipeline:pipeline0/GstPipeline:pipeline1/GstAlsaSrc:alsasrc0:
Dropped 217088 samples. This is most likely because downstream can’t keep up and is consuming samples too slowly.
Error: gst-core-error-quark: Can’t record audio fast enough (13): gstaudiobasesrc.c(849): gst_audio_base_src_create (): /GstPipeline:pipeline0/GstPipeline:pipeline1/GstAlsaSrc:alsasrc0:

Iv’e been searching and trying to fix this for couple of days now, any help would be appreciated!

3

Hi,
You may try audotestsrc and see if it works. To clarify if the pipeline is valid by replacing alsasrc with audotestsrc.

We don’t try such complex case. Maybe you can begin with one video + one audio and then increase to two, three videos. Here is a simple pipeline for reference:
Mixing audio from 2 usb audio sources - #3 by DaneLLL

4

You may also try adding aacparse :

... audioconvert ! voaacenc ! aacparse ! queue ! muxcam0.

If not enough, you may also see similar case :

6

Thanks for the help, audioconvert ! voaacenc did the trick.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.