Record Audio and Video Syncronized

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

I would like to record with gstreamer rtsp from IP camera and Audio from USB Alsa .
I’m able to record both but audio is not syncornized with video.
This is the command for record only video:
gst-launch-1.0 -e rtspsrc location=rtsp://admin:password@192.168.11.23/MediaInput/h264 ! queue ! rtpjitterbuffer latency=100 ! rtph264depay ! h264parse ! splitmuxsink muxer=mp4mux location=file%02d.mp4 max-size-time=6000000000
this is for record audio :
gst-launch-1.0 alsasrc device=hw:1,0 ! audio/x-raw,channels=1 ! volume volume=5 ! opusenc audio-type=voice ! .audio_%u splitmuxsink location=qqqqaaa%03d.ogg max-size-time=6000000000 muxer=oggmux

Is there anyone expert that can help me to record both and also syncronized?

2

You would use a single command where the video pipeline and the audio pipeline are connected to the mux such as:

gst-launch-1.0 -e splitmuxsink name=mux muxer=mp4mux location=file%02d.mp4    videotestsrc ! nvvidconv ! omxh264enc ! h264parse ! queue ! mux.video    audiotestsrc ! audio/x-raw,channels=1 ! volume volume=5 ! opusenc audio-type=voice ! queue ! mux.audio_0

You would adjust the video source to rtsp and audio to your USB source.

3

I lunched this command and get this error :
gst-launch-1.0 -e splitmuxsink name=mux muxer=mp4mux location=file%02d.mp4 rtspsrc location=rtsp://admin:password@192.168.11.23/MediaInput/h264 ! h264parse ! queue ! mux.video alsasrc device=hw:1,0 ! audio/x-raw,channels=1 ! volume volume=5 ! opusenc audio-type=voice ! queue ! mux.audio_0

WARNING: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0: Delayed linking failed.
Additional debug info:
./grammar.y(510): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0:
failed delayed linking some pad of GstRTSPSrc named rtspsrc0 to some pad of GstH264Parse named h264parse0
ERROR: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0/GstUDPSrc:udpsrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0/GstUDPSrc:udpsrc0:
streaming stopped, reason not-linked (-1)
EOS on shutdown enabled – waiting for EOS after Error
Waiting for EOS…

4

Your pipeline lacks rtph264depay bteween rtspsrc and h264parse.

5

gst-launch-1.0 -e splitmuxsink name=mux muxer=mp4mux location=file%02d.mp4 ! rtspsrc location=rtsp://admin:password@192.168.11.23/MediaInput/h264 ! rtph264depay ! h264parse ! queue ! mux.video ! alsasrc device=hw:1,0 ! audio/x-raw,channels=1 ! volume volume=5 ! opusenc audio-type=voice ! queue ! mux.audio_0
WARNING: erroneous pipeline: syntax error

6

This time there are extra ‘!’. This command launches 3 pipelines, one for video, one for audio and one for muxer Try:

gst-launch-1.0 -e splitmuxsink name=mux muxer=mp4mux location=file%02d.mp4                  rtspsrc location=rtsp://admin:password@192.168.11.23/MediaInput/h264 ! rtph264depay ! h264parse ! queue ! mux.video            alsasrc device=hw:1,0 ! audio/x-raw,channels=1 ! volume volume=5 ! opusenc audio-type=voice ! queue ! mux.audio_0
7

I found the way for record with this command:
gst-launch-1.0 -e splitmuxsink name=mux muxer=mp4mux location=file%02d.mkv max-size-time=30000000000 rtspsrc location=rtsp://admin:password@192.168.11.23/MediaInput/h264 ! queue ! rtpjitterbuffer latency=15 ! rtph264depay ! h264parse ! mux.video alsasrc device=hw:1,0 ! audio/x-raw,channels=1 ! volume volume=5 ! opusenc audio-type=voice ! queue ! mux.audio_0

I have now one problem. The splitted files timeline does not start from 0. I need also to restream this ip camera with audio syncronzied on rtp. Can you help me?

8

I found way also for restream , but audio and video are not syncronized
./test-launch --gst-debug=3 ‘( rtspsrc location=rtsp://user:pwd!@192.168.11.23/MediaInput/h264 latency=15 ! rtph264depay ! h264parse ! rtph264pay name=pay0 pt=96 alsasrc device=“hw:1,0” ! queue ! audioconvert ! audioresample ! avenc_aac ! rtpmp4apay name=pay1 pt=98 )’

10

Hi,
It looks like the video source(rtspsrc) and audio source(alsasrc) are not synchronized. You may check if the sources are synchronized in saving to a MP4 file. A reference pipeline of saving to mp4: