How to duplicate frame and display side by side Gstreamer

I want to display the following video side by side so that I have 2 1280x720 frames creating a 2560x720 frame (equivalent to using hconcat in OpenCV). Is it possible to do this in a gstreamer pipeline? Here is the pipeline to get the camera stream:

gst-launch-1.0 nvarguscamerasrc sensor_id=0 ! \
   'video/x-raw(memory:NVMM),width=12780, height=720, framerate=60/1, format=NV12' ! \
   nvvidconv flip-method=0 ! 'video/x-raw,width=1280, height=720' ! \
   nvvidconv ! nvegltransform ! nveglglessink -e

Hi,
You can use nvcompositor plugin. Please refer to
Nvcompositor is not working on Jetpack 4.6 - #5 by DaneLLL

I tried this to duplicate the camera stream using nvcompositor but it failed. I don’t think I can open 2 sources of the same camera at once. Is there a way I could use the tee method to send the same stream to sink 1 and 2 and duplicate the stream on screen?

    gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720 sink_0::zorder=1 sink_0::alpha=1 \
    sink_1::xpos=1280 sink_1::ypos=0 sink_1::width=1280 sink_1::height=720 sink_1::zorder=2 sink_1::alpha=1 ! \
    'video/x-raw(memory:NVMM),format=RGBA' ! nvvidconv ! video/x-raw,format=RGBA ! nv3dsink \
    nvarguscamerasrc sensor_id=0 ! 'video/x-raw(memory:NVMM),width=1280, height=720, framerate=60/1, format=NV12' ! comp.sink_0 \
    nvarguscamerasrc sensor_id=0 ! 'video/x-raw(memory:NVMM),width=1280, height=720, framerate=60/1, format=NV12' ! comp.sink_1    
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:736 Failed to create CaptureSession
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 4 
   Output Stream W = 1280 H = 720 
   seconds to Run    = 0 
   Frame Rate = 59.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
Redistribute latency...
pixel_format must be RGBA for blending operation
nvbuffer_composite Failed
Got EOS from element "pipeline0".
Execution ended after 0:00:00.644832697
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Success
Setting pipeline to NULL ...
Freeing pipeline ...

Hi,
You may try to create two windows and put the windows side by side:

~$ gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),format=NV12,width=640,height=360' ! tee name=t ! queue ! nv3dsink window-x=640 sync=0 t. ! queue ! nv3dsink sync=0

Thanks that’s really close to what I’m looking for. Is there a way to combine the tee method with nvcompositor to place the windows side by side?

Actually I think the issue with the nvcompositor is that is expects a RGBA format but I’m sending a NV12 format. How would I perform the conversion to get RGBA?

so this works to convert it (I think):

gst-launch-1.0 nvarguscamerasrc sensor_id=0 !    \
'video/x-raw(memory:NVMM),width=1280, height=720, framerate=60/1, format=NV12' !   \
nvvidconv flip-method=0 ! 'video/x-raw,width=1280, height=720, format=RGBA' ! nv3dsink

But when I run this pipeline:

 gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720 sink_0::zorder=1 sink_0::alpha=1 \
    sink_1::xpos=1280 sink_1::ypos=0 sink_1::width=1280 sink_1::height=720 sink_1::zorder=2 sink_1::alpha=1 ! \
    'video/x-raw(memory:NVMM),format=RGBA' ! nvvidconv ! video/x-raw,format=RGBA ! nv3dsink \
    nvarguscamerasrc sensor_id=0 ! 'video/x-raw(memory:NVMM),width=1280, height=720, framerate=60/1, format=NV12' ! nvvidconv flip-method=0 ! 'video/x-raw,width=1280, height=720, format=RGBA' ! comp.sink_0 \
    nvarguscamerasrc sensor_id=1 ! 'video/x-raw(memory:NVMM),width=1280, height=720, framerate=60/1, format=NV12' ! nvvidconv flip-method=0 ! 'video/x-raw,width=1280, height=720, format=RGBA' ! comp.sink_1  

I get an error saying WARNING: erroneous pipeline: could not link nvvconv1 to comp, comp can't handle caps video/x-raw, width=(int)1280, height=(int)720, format=(string)RGBA

When I run the following pipeline:

    gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720 sink_0::zorder=1 sink_0::alpha=1 \
    sink_1::xpos=1280 sink_1::ypos=0 sink_1::width=1280 sink_1::height=720 sink_1::zorder=2 sink_1::alpha=1 ! \
    'video/x-raw(memory:NVMM),format=RGBA' ! nvvidconv ! video/x-raw,format=RGBA ! nv3dsink \
    nvarguscamerasrc sensor_id=0 ! 'video/x-raw(memory:NVMM),width=1280, height=720, framerate=60/1, format=NV12' ! comp.sink_0 \
    nvarguscamerasrc sensor_id=1 ! 'video/x-raw(memory:NVMM),width=1280, height=720, framerate=60/1, format=NV12' ! comp.sink_1 

The error says:

pixel_format must be RGBA for blending operation

This works using videocompositor but It’s 2 different csi cameras, I’m not sure how to use the tee operation here. It also makes the framerate very low:

gst-launch-1.0 nvarguscamerasrc sensor_id=0 !deo/x-raw(memory:NVMM),format=NV12,width=640,height=360' ! nvvidconv flip-method=0 ! 'video/x-raw,width=1280, height=720, format=RGBA' ! compositor name=comp sink_1::xpos=1280 ! queue ! nv3dsink sync=0 nvarguscamerasrc sensor_id=1 ! 'video/x-raw(memory:NVMM),format=NV12,width=1280,height=720' ! nvvidconv flip-method=0 ! 'video/x-raw,width=1280, height=720, format=RGBA' ! comp.

Sorry for so many replies, this link looks like it might help NVIDIA Jetson Nano GStreamer example pipelines for video transforming - RidgeRun Developer Connection

 gst-launch-1.0 nvarguscamerasrc sensor_id=0 !    'video/x-raw(memory:NVMM),width=1280, height=720, \
framerate=60/1, format=NV12' !   nvvidconv flip-method=0 ! 'video/x-raw,width=1280, height=720, format=RGBA' ! \
queue ! tee name=t t. ! queue ! comp. t. ! queue ! comp. nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 \
sink_0::width=160 sink_0::height=120 sink_1::xpos=160 sink_1::ypos=0 sink_1::width=160 sink_1::height=120 ! \
nv3dsink

That gives a error reason not negotiated (-4) but if I run it with nvoverlaysink there is no error (but obviously it doesn’t display because the sink is depracated)

Got it working!

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720 \
sink_1::xpos=1280 sink_1::ypos=0 sink_1::width=1280 sink_1::height=720 \
! 'video/x-raw(memory:NVMM),format=RGBA' ! nv3dsink \
nvarguscamerasrc sensor_id=0 ! 'video/x-raw(memory:NVMM),width=1280, height=720, framerate=60/1, format=NV12' ! nvvidconv flip-method=0 ! 'video/x-raw(memory:NVMM),width=1280, height=720, format=RGBA' \
! queue ! tee name=t t. ! queue ! comp. t. ! queue ! comp.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.