Using Gstreamer to record three sources into one file

Greetings,

I have three image sensors accessible at /dev/video0, /dev/video1 and /dev/video2. I want to record these three streams side-by-side into one file. I have been able to display the three streams side-by-side with nvoverlaysink, but have had no luck with filesink yet.

Hi,
Please share your current working pipeline with nvoverlaysink so that we can base on it to suggest next.

Here is the script displaying three streams at once:

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! \
"video/x-raw(memory:NVMM), format=(string)I420, width=(int)$VIDEO_WIDTH, height=(int)$VIDEO_HEIGHT" ! \
 nvoverlaysink overlay-x=0 overlay-y=200 overlay-h=$VIDEO_HEIGHT overlay-w=$VIDEO_WIDTH overlay=1 overlay-depth=0 sync=false \
v4l2src device=/dev/video1 ! "video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! \
"video/x-raw(memory:NVMM), format=(string)I420, width=(int)$VIDEO_WIDTH, height=(int)$VIDEO_HEIGHT" ! \
nvoverlaysink overlay-x=$VIDEO_WIDTH overlay-y=200 overlay-h=$VIDEO_HEIGHT overlay-w=$VIDEO_WIDTH overlay=2 overlay-depth=1 sync=false \
v4l2src device=/dev/video2 ! "video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! \
"video/x-raw(memory:NVMM), format=(string)I420, width=(int)$VIDEO_WIDTH, height=(int)$VIDEO_HEIGHT" ! \
nvoverlaysink overlay-x=$((2*$VIDEO_WIDTH)) overlay-y=200 overlay-h=$VIDEO_HEIGHT overlay-w=$VIDEO_WIDTH overlay=3 overlay-depth=2 sync=false

Here is the command I use to record three sources to three files:

gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw, format=(string)UYVY, width=(int)$SOURCE_WIDTH, height=(int)$SOURCE_HEIGHT" ! nvvidconv ! \
"video/x-raw(memory:NVMM), format=(string)I420, width=(int)$VIDEO_WIDTH, height=(int)$VIDEO_HEIGHT" ! \
omxh264enc qp-range=20,20:20,20:-1,-1 ! matroskamux ! queue ! filesink location=$OUTPUT_FOLDER${1}1.mkv \
v4l2src device=/dev/video1 ! "video/x-raw, format=(string)UYVY, width=(int)$SOURCE_WIDTH, height=(int)$SOURCE_HEIGHT" ! nvvidconv ! \
"video/x-raw(memory:NVMM), format=(string)I420, width=(int)$VIDEO_WIDTH, height=(int)$VIDEO_HEIGHT" ! \
omxh264enc qp-range=20,20:20,20:-1,-1 ! matroskamux ! queue ! filesink location=$OUTPUT_FOLDER${1}2.mkv \
v4l2src device=/dev/video2 ! "video/x-raw, format=(string)UYVY, width=(int)$SOURCE_WIDTH, height=(int)$SOURCE_HEIGHT" ! nvvidconv ! \
"video/x-raw(memory:NVMM), format=(string)I420, width=(int)$VIDEO_WIDTH, height=(int)$VIDEO_HEIGHT" ! \
omxh264enc qp-range=20,20:20,20:-1,-1 ! matroskamux ! queue ! filesink location=$OUTPUT_FOLDER${1}3.mkv

But how do I record three streams to a single file, so it looks the same way as when running the first command? (side-by-side)

Hi,
You can utilize nvcompositor plugin. Below pipeline is for your reference:

$ gst-launch-1.0 videotestsrc ! video/x-raw,format=UYVY ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_0  videotestsrc pattern=1 ! video/x-raw,format=UYVY ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_1  nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_1::xpos=320 sink_1::ypos=0 ! 'video/x-raw(memory:NVMM),width=640,height=240' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvv4l2h264enc ! h264parse ! matroskamux ! filesink location=a.mkv

Thank you for replying!
I have been trying compositor before, but wasn’t able to get it working.

Trying your example gives me:

WARNING: erroneous pipeline: no element "nvv4l2h264enc"

I am using the image from e-con systems, which I guess is one of your partners.
uname -a reports:

Linux tegra-ubuntu 4.4.38-ged8e6c8

Using omxh264enc reports:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Framerate set to : 72 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4 
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:00.259409824
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Hi,
The function is in new release. Please upgrade to r32.2.1.