Save 4 rtsp stream from 4 camera to 1 concatenated video

Robotics & Edge Computing Jetson Systems Jetson Nano
1

Hi, I’m trying to save 4 different rtsp stream to one video file. I want to concatenate the 4 stream resolution of 1280x720 to one video of 2560x1440.

I succeed to concatenate the 4 stream to one video file but the video file generated is not compressed. It makes 200Mo for 10secondes. It is too big.
When I checked the codec of the video file created it says ‘Uncompressed packed YUV’

Here is my command :

gst-launch-1.0 -ev \ videomixer name=mix \ sink_1::xpos=0 sink_1::ypos=0 \ sink_2::xpos=1280 sink_2::ypos=0 \ sink_3::xpos=0 sink_3::ypos=720 \ sink_4::xpos=1280 sink_4::ypos=720 \ ! qtmux ! filesink location=mosaic.mp4 \ rtspsrc location='rtsp://192.168.1.19:554/1/h264major' \ ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv \ ! video/x-raw,width=1280,height=720 \ ! mix.sink_1 \ rtspsrc location='rtsp://192.168.1.19:554/1/h264major' \ ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv \ ! video/x-raw,width=1280,height=720 \ ! mix.sink_2 \ rtspsrc location='rtsp://192.168.1.20:554/1/h264major' \ ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv \ ! video/x-raw,width=1280,height=720 \ ! mix.sink_3 \ rtspsrc location='rtsp://192.168.1.20:554/1/h264major' \ ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv \ ! video/x-raw,width=1280,height=720 \ ! mix.sink_4 \
&

PID=$!

sleep 61

kill -INT $PID`

If you have any advice which can help me to reduce the size, to make the file compressed, Thanks!

2

You may try using nvcompositor instead of videomixer.
You may also need to use RTSP2.0 for EOS management.

gst-launch-1.0 -e nvcompositor name=mix background-w=2560 background-h=1440 \
    sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720  \
    sink_1::xpos=1280 sink_1::ypos=0 sink_1::width=1280 sink_1::height=720 \
    sink_2::xpos=0 sink_2::ypos=720 sink_2::width=1280 sink_2::height=720 \
    sink_3::xpos=1280 sink_3::ypos=720 sink_3::width=1280 sink_3::height=720 \
    ! 'video/x-raw(memory:NVMM),format=RGBA' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! nvv4l2h264enc insert-vui=1 ! h264parse ! matroskamux ! filesink location=test.mkv sync=false  \
    rtspsrc location=rtsp://127.0.0.1:8554/test default-rtsp-version=32 ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! 'video/x-raw(memory:NVMM),format=NV12' ! queue ! mix.sink_0      \
    rtspsrc location=rtsp://127.0.0.1:8554/test default-rtsp-version=32 ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! 'video/x-raw(memory:NVMM),format=NV12' ! queue ! mix.sink_1      \
    rtspsrc location=rtsp://127.0.0.1:8554/test default-rtsp-version=32 ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! 'video/x-raw(memory:NVMM),format=NV12' ! queue ! mix.sink_2      \
    rtspsrc location=rtsp://127.0.0.1:8554/test default-rtsp-version=32 ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! 'video/x-raw(memory:NVMM),format=NV12' ! queue ! mix.sink_3
3

Thanks!
I tried to use your command but it says this :

canva-photo-editor
canva-photo-editor742×439 87.2 KB

How can I use rtsp2.0 ?

4

Maybe the rtsp server in your camera doesn’t support this, but it may not be mandatory.
RTSP1.0 seems working for me for this case:

gst-launch-1.0 -e nvcompositor name=mix background-w=2560 background-h=1440 \
    sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720  \
    sink_1::xpos=1280 sink_1::ypos=0 sink_1::width=1280 sink_1::height=720 \
    sink_2::xpos=0 sink_2::ypos=720 sink_2::width=1280 sink_2::height=720 \
    sink_3::xpos=1280 sink_3::ypos=720 sink_3::width=1280 sink_3::height=720 \
    ! 'video/x-raw(memory:NVMM),format=RGBA' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvv4l2h264enc insert-vui=1 profile=2 ! h264parse ! matroskamux ! filesink location=test.mkv sync=false  \
    rtspsrc location=rtsp://127.0.0.1:8554/test ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! 'video/x-raw(memory:NVMM),format=NV12' ! queue ! mix.sink_0      \
    rtspsrc location=rtsp://127.0.0.1:8554/test ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! 'video/x-raw(memory:NVMM),format=NV12' ! queue ! mix.sink_1      \
    rtspsrc location=rtsp://127.0.0.1:8554/test ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! 'video/x-raw(memory:NVMM),format=NV12' ! queue ! mix.sink_2      \
    rtspsrc location=rtsp://127.0.0.1:8554/test ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! 'video/x-raw(memory:NVMM),format=NV12' ! queue ! mix.sink_3
5

Thanks a lot !! It works as I wish

6

Hi, the code you give me works but does not work every time. It gives me this error.

ERROR: from element /GstPipeline:pipeline0/GstMatroskaMux:matroskamux0: Could not multiplex stream.
Additional debug info:
matroska-mux.c(986): gst_matroska_mux_video_pad_setcaps (): /GstPipeline:pipeline0/GstMatroskaMux:matroskamux0:
Caps changes are not supported by Matroska
EOS on shutdown enabled – waiting for EOS after Error
Waiting for EOS…
ERROR: from element /GstPipeline:pipeline0/GstMatroskaMux:matroskamux0: Could not multiplex stream.
Additional debug info:
matroska-mux.c(986): gst_matroska_mux_video_pad_setcaps (): /GstPipeline:pipeline0/GstMatroskaMux:matroskamux0:
Caps changes are not supported by Matroska

Do you have an idea why sometimes it crashes ?
Thanks!

7

The error message says that caps changed and matroskamux doesn’t support this.
You may try to scale each decoded source to 720p, set fixed caps and add -v flag, so that you’ll get some details about what caps changed before the error:

gst-launch-1.0 -ev nvcompositor name=mix \
    sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720  \
    sink_1::xpos=1280 sink_1::ypos=0 sink_1::width=1280 sink_1::height=720 \
    sink_2::xpos=0 sink_2::ypos=720 sink_2::width=1280 sink_2::height=720 \
    sink_3::xpos=1280 sink_3::ypos=720 sink_3::width=1280 sink_3::height=720 \
    ! 'video/x-raw(memory:NVMM),format=RGBA,width=2560,height=1440,framerate=30/1' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvv4l2h264enc insert-vui=1 profile=2 \
    ! h264parse ! video/x-h264,width=2560,height=1440,framerate=30/1 ! matroskamux ! filesink location=test.mkv sync=false  \
    rtspsrc location=rtsp://127.0.0.1:8554/test ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12,width=1280,height=720,pixel-aspect-ratio=1/1' ! queue ! mix.sink_0      \
    rtspsrc location=rtsp://127.0.0.1:8554/test ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12,width=1280,height=720,pixel-aspect-ratio=1/1' ! queue ! mix.sink_1      \
    rtspsrc location=rtsp://127.0.0.1:8554/test ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12,width=1280,height=720,pixel-aspect-ratio=1/1' ! queue ! mix.sink_2      \
    rtspsrc location=rtsp://127.0.0.1:8554/test ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12,width=1280,height=720,pixel-aspect-ratio=1/1' ! queue ! mix.sink_3
8

Thanks !
It sends no error, but when I want to stop the programm with Ctrl+C it sends this error :

ERROR: from element /GstPipeline:pipeline0/GstQueue:queue1: Internal data stream error.
Additional debug info:
gstqueue.c(988): gst_queue_handle_sink_event (): /GstPipeline:pipeline0/GstQueue:queue1:
streaming stopped, reason not-negotiated (-4)
An error happened while waiting for EOS
Execution ended after 0:00:21.379088927
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

Is there a way to stop properly exit a gstreamer pipeline ?

I’m using this :

nvvidconv ! ‘video/x-raw(memory:NVMM),format=NV12,width=1280,height=720,pixel-aspect-ratio=1/1’ ! queue ! mix.sink_3
&

PID=$!

sleep 21

kill -INT $PID

9

You may try removing -e or sending signal twice.

10

Thanks it working !

Now when I play the video file created it doesn’t show the total duration like this :

oie_pmkwFWtsDDrt
oie_pmkwFWtsDDrt1226×76 53.1 KB

The information of the file is that :

oie_cTqIW11udCkW
oie_cTqIW11udCkW724×186 39.3 KB

Do you have an idea why my video file is “corrupted” ?

By using the command in your post of the april 29 :

Maybe the rtsp server in your camera doesn’t support this, but it may not be mandatory.
RTSP1.0 seems working for me for this case:

The file has a duration set.

Do you have any idea why one is working and the other one no?

Thanks in advance