Gstreamer Image compositor with video streams dosen't start with preview

pulzappcheck890 · November 1, 2022, 6:28am

Hi,
I’m trying the below pipeline as follows:

gst-launch-1.0 -e videotestsrc pattern=black ! video/x-raw,width=320,height=240 ! nvvidconv ! queue ! tee name=t5 ! queue ! comp1.sink_0 v4l2src device=/dev/presentation ! queue ! tee name=t1 ! video/x-raw, width=1920, height=1080, framerate=60/1 ! videorate ! video/x-raw, width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_1 v4l2src device=/dev/exCAM io-mode=2 ! tee name=t2 ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_2 t2. ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_3 gdkpixbufoverlay location=logo-.png ! videoconvert ! imagefreeze ! queue ! comp1.sink_4 nvcompositor name=comp1 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=295 sink_1::width=970 sink_1::height=540 sink_2::xpos=970 sink_2::ypos=295 sink_2::width=960 sink_2::height=540 sink_3::xpos=0 sink_3::ypos=0 sink_3::width=1950 sink_3::height=270 sink_4::ypos=860 sink_4::width=1950 sink_4::height=270 ! nvvidconv ! nvoverlaysink display-id=1

The comp 3 is what i’m trying to display the image

When i try to run the above pipeline, it does not throw an error and it does not show the preview as well. but, when i tried to run the pipeline, the following info appeared

Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 277
NvMMLiteOpen : Block : BlockType = 277
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 277
NvMMLiteBlockCreate : Block : BlockType = 277
^Chandling interrupt.
Interrupt: Stopping pipeline …
EOS on shutdown enabled – Forcing EOS on the pipeline

is there any other method to eliminate this and get the preview?

Thanks in advance !

DaneLLL · November 1, 2022, 8:00am

Hi,
For further check, please try

If it works with 4 videotestsrc sources
If 4 videotestsrc sources work, please replace it with your sources one by one to clarify which one triggers the issue

The issue may be from

gdkpixbufoverlay location=logo-.png ! videoconvert ! imagefreeze ! queue ! comp1.sink_4

Not sure but this one looks to be a single image. Other sources are continuous frames and this source may not work with them properly.

pulzappcheck890 · November 3, 2022, 5:08am

I tried the way you mentioned earlier and now when i try to run the pipeline, i get a fully blacked-out preview .
The used pipeline as follows:

gst-launch-1.0 -e videotestsrc pattern=black ! video/x-raw,width=320,height=240 ! nvvidconv ! queue ! tee name=t5 ! queue ! comp1.sink_0 v4l2src device=/dev/presentation ! queue ! tee name=t1 ! video/x-raw, width=1920, height=1080, framerate=60/1 ! videorate ! video/x-raw, width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_1 v4l2src device=/dev/exCAM io-mode=2 ! tee name=t2 ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_2 t2. ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_3 filesrc location=/home/plz/Desktop/plz-logo-.png ! pngdec ! videoconvert ! imagefreeze ! queue ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12”! queue ! comp1.sink_4 nvcompositor name=comp1 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=295 sink_1::width=970 sink_1::height=540 sink_2::xpos=970 sink_2::ypos=295 sink_2::width=960 sink_2::height=540 sink_3::xpos=0 sink_3::ypos=0 sink_3::width=1950 sink_3::height=270 sink_4::ypos=860 sink_4::width=1950 sink_4::height=270 ! nvvidconv ! nvoverlaysink display-id=1

DaneLLL · November 3, 2022, 5:22am

Hi,
The later command looks identical to the original one. Does it work with 4 videotestsrc? Have you identified which source triggers the issue?

pulzappcheck890 · November 3, 2022, 5:49am

Yes, the 4th video feed works , the problem occurs when i try to insert a picture.
The used pipeline as follows:

gst-launch-1.0 -e videotestsrc pattern=black ! video/x-raw,width=320,height=240 ! nvvidconv ! queue ! tee name=t5 ! queue ! comp1.sink_0 v4l2src device=/dev/presentation ! queue ! tee name=t1 ! video/x-raw, width=1920, height=1080, framerate=60/1 ! videorate ! video/x-raw, width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_1 v4l2src device=/dev/exCAM io-mode=2 ! tee name=t2 ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_2 t2. ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_3 t2. ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_4 t2. ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_5 nvcompositor name=comp1 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=295 sink_1::width=970 sink_1::height=540 sink_2::xpos=970 sink_2::ypos=295 sink_2::width=960 sink_2::height=540 sink_3::xpos=0 sink_3::ypos=0 sink_3::width=1950 sink_3::height=270 sink_4::xpos=0 sink_4::ypos=60 sink_4::width=1950 sink_4::height=270 sink_5::ypos=860 sink_5::width=1950 sink_5::height=270 ! nvvidconv ! nvoverlaysink display-id=1

and this is the pipeline i used to insert the image (as the 4th source):

gst-launch-1.0 -e videotestsrc pattern=black ! video/x-raw,width=320,height=240 ! nvvidconv ! queue ! tee name=t5 ! queue ! comp1.sink_0 v4l2src device=/dev/presentation ! queue ! tee name=t1 ! video/x-raw, width=1920, height=1080, framerate=60/1 ! videorate ! video/x-raw, width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_1 v4l2src device=/dev/exCAM io-mode=2 ! tee name=t2 ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_2 t2. ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_3 t2. ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_4 filesrc location=/home/plz/Desktop/plz-logo-.png ! pngdec ! videoconvert ! imagefreeze ! queue ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! videorate ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_5 nvcompositor name=comp1 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=295 sink_1::width=970 sink_1::height=540 sink_2::xpos=970 sink_2::ypos=295 sink_2::width=960 sink_2::height=540 sink_3::xpos=0 sink_3::ypos=0 sink_3::width=1950 sink_3::height=270 sink_4::xpos=0 sink_4::ypos=60 sink_4::width=1950 sink_4::height=270 sink_5::ypos=860 sink_5::width=1950 sink_5::height=270 ! nvvidconv ! nvoverlaysink display-id=1

DaneLLL · November 3, 2022, 6:28am

Hi,
Single image is no supported in nvcompositor. Please refer to the command to make it as a continuous frame source:

$ gst-launch-1.0 multifilesrc location=src%03d.png loop=1 ! pngdec ! videoparse format=2 framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvoverlaysink

Honey_Patouceul · November 3, 2022, 8:49pm

Not sure I correctly understand your case (some unused queues), but trying to mimic it, this works:

gst-launch-1.0 -e \
videotestsrc pattern=black ! video/x-raw,width=320,height=240,framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,pixel-aspect-ratio=1/1' ! tee name=t5 ! queue ! comp1.sink_0 \
videotestsrc pattern=snow ! video/x-raw,width=320,height=240,framerate=60/1 ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvvidconv ! video/x-raw, width=1920, height=1080 ! tee name=t1 ! queue ! videorate ! video/x-raw,framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12,width=970,height=540,pixel-aspect-ratio=1/1' ! queue ! comp1.sink_1 \
videotestsrc pattern=ball ! video/x-raw,width=320,height=240,framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420,width=1920,height=1080,framerate=30/1,pixel-aspect-ratio=1/1' ! nvjpegenc ! image/jpeg,width=1920,height=1080,framerate=30/1 ! tee name=t2 ! queue ! nvv4l2decoder mjpeg=1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12,width=960,height=540,pixel-aspect-ratio=1/1'  ! queue ! comp1.sink_2 \
t2. ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! 'video/x-raw(memory:NVMM),width=1950,height=270,format=NV12,pixel-aspect-ratio=1/1' ! queue ! comp1.sink_3 \
t2. ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! 'video/x-raw(memory:NVMM),width=1950,height=270,format=NV12,pixel-aspect-ratio=1/1' ! queue ! comp1.sink_4 \
filesrc location=test.png ! image/png,width=1720 ,height=1440,framerate=0/1,pixel-aspect-ratio=1/1 ! pngdec ! videoconvert ! imagefreeze ! video/x-raw,framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),width=1950,height=270,format=NV12,pixel-aspect-ratio=1/1' ! queue ! comp1.sink_5 \
nvcompositor name=comp1 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=295 sink_1::width=970 sink_1::height=540 sink_2::xpos=970 sink_2::ypos=295 sink_2::width=960 sink_2::height=540 sink_3::xpos=0 sink_3::ypos=0 sink_3::width=1950 sink_3::height=270 sink_4::xpos=0 sink_4::ypos=60 sink_4::width=1950 sink_4::height=270 sink_5::ypos=860 sink_5::width=1950 sink_5::height=270 ! nvvidconv ! autovideosink

pulzappcheck890 · November 4, 2022, 3:46am

Thank You. This was the needed output.

pulzappcheck890 · November 4, 2022, 8:17am

Hi,

The below Pipeline works fine. But the problem is the png images does not appear transparent. It has a black background.

gst-launch-1.0 -e videotestsrc pattern=black ! video/x-raw,width=320,height=240 ! nvvidconv ! queue ! tee name=t5 ! queue ! comp1.sink_0 v4l2src device=/dev/presentation ! queue ! tee name=t1 ! video/x-raw, width=1920, height=1080, framerate=60/1 ! videorate ! video/x-raw, width=1920, height=1080, framerate=30/1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_1 v4l2src device=/dev/exCAM io-mode=2 ! tee name=t2 ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12” ! queue ! comp1.sink_2 filesrc location=/home/plz/Downloads/GST/logo-.png ! image/png,width=1720 ,height=1440,framerate=0/1,pixel-aspect-ratio=1/1 ! pngdec ! videoconvert ! imagefreeze ! video/x-raw,framerate=30/1 ! nvvidconv ! ‘video/x-raw(memory:NVMM),width=1950,height=270,format=NV12,pixel-aspect-ratio=1/1’ ! queue ! comp1.sink_3 filesrc location=/home/plz/Downloads/GST/logo-.png ! image/png,width=1720 ,height=1440,framerate=0/1,pixel-aspect-ratio=1/1 ! pngdec ! videoconvert ! imagefreeze ! video/x-raw,framerate=30/1 ! nvvidconv ! ‘video/x-raw(memory:NVMM),width=1950,height=270,format=NV12,pixel-aspect-ratio=1/1’ ! queue ! comp1.sink_4 filesrc location=/home/plz/Downloads/GST/logo-.png ! image/png,width=720 ,height=440,framerate=0/1,pixel-aspect-ratio=60/1 ! pngdec ! videoconvert ! imagefreeze ! video/x-raw,framerate=30/1 ! nvvidconv ! ‘video/x-raw(memory:NVMM),width=1950,height=270,format=NV12,pixel-aspect-ratio=60/1’ ! queue ! comp1.sink_5 nvcompositor name=comp1 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=295 sink_1::width=970 sink_1::height=540 sink_2::xpos=970 sink_2::ypos=295 sink_2::width=960 sink_2::height=540 sink_3::xpos=0 sink_3::ypos=0 sink_3::width=1950 sink_3::height=270 sink_4::xpos=0 sink_4::ypos=290 sink_4::width=150 sink_4::height=70 sink_4::zorder=4 sink_4::alpha=0.5 sink_5::ypos=860 sink_5::width=1950 sink_5::height=270 ! nvoverlaysink

Thanks in advance

DaneLLL · November 4, 2022, 8:29am

Hi,
Transparent blending is not supported in nvcompositor plugin. Please note this.

There is a related discussion for Orin:
Nvcompositor plugs alpha is not work and alpha plugs no work - #24 by DaneLLL

system · November 23, 2022, 6:42am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.