Gstreamer Compositor error

pulzappcheck890 · November 9, 2022, 9:25am

Hi,

I’m trying to use compositor and when i try to run the pipeline it throws an error as below:

WARNING: erroneous pipeline: could not link queue2 to comp

The used pipeline as follows:

gst-launch-1.0 -v v4l2src device=/dev/exCAM io-mode=2 ! tee name=t2 ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw(memory:NVMM),width=960,height=540,format=NV12”! identity ! queue ! comp.sink_0 filesrc location=/home/ums4/Downloads/GST/eduscope-logo-.png ! image/png,width=1720 ,height=1440,framerate=0/1,pixel-aspect-ratio=1/1 ! pngdec ! videoconvert ! imagefreeze ! video/x-raw,framerate=30/1 ! nvvidconv ! ‘video/x-raw(memory:NVMM),width=1950,height=270,format=NV12,pixel-aspect-ratio=1/1’ ! identity ! queue ! comp.sink_1 compositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=320 sink_0::height=240 sink_1::xpos=0 sink_1::ypos=240 sink_1::width=320 sink_1::height=240 ! video/x-raw,width=320,height=480 ! videoconvert ! xvimagesink

Thanks in advance.

pulzappcheck890 · November 9, 2022, 10:17am

When i try to overlay an image over the video, it does not show the preview.

The used pipeline as follows:

gst-launch-1.0 videotestsrc pattern=snow ! video/x-raw,width=1280,height=720,framerate=30/1 ! queue ! comp.sink_0 gdkpixbufoverlay location=/home/ums4/Downloads/GST/eduscope-logo-.png offset-x=-1 offset-y=-1 overlay-width=40 overlay-height=40 ! videoconvert ! queue ! nvvidconv ! queue ! comp.sink_1 compositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::zorder=1 sink_1::xpos=0 sink_1::ypos=0 sink_1::zorder=2 ! videoconvert ! xvimagesink

Honey_Patouceul · November 9, 2022, 4:01pm

You have 2 issues:

First one is that you are using nvvidconv that has at least one of its input or output in NVMM memory. For compositor, the inputs are expected to be in system memory.
Second one is that gdkpixbufoverlay overlays the picture onto a video stream, but in your pipeline it has no input, so it never starts.
You may try:

# With gdkpixbufoverlay:
gst-launch-1.0 videotestsrc pattern=snow ! video/x-raw,width=1280,height=720,framerate=30/1 ! gdkpixbufoverlay location=/usr/share/jetsonpowergui/logo/nv_logo.png offset-x=-1 offset-y=-1 overlay-width=272 overlay-height=203 ! videoconvert ! xvimagesink

# Or with compositor
gst-launch-1.0 \
videotestsrc pattern=snow ! video/x-raw,width=1280,height=720,framerate=30/1 ! queue ! comp.sink_0   \
filesrc location=/usr/share/jetsonpowergui/logo/nv_logo.png ! pngdec ! imagefreeze ! video/x-raw,framerate=30/1 ! videoconvert ! video/x-raw,format=RGBA ! queue ! comp.sink_1  \
compositor name=comp sink_0::width=1280 sink_0::height=720 sink_0::zorder=0 sink_1::xpos=1179 sink_1::ypos=619 sink_1::width=100 sink_1::height=100 sink_1::zorder=1 ! videoconvert ! xvimagesink

pulzappcheck890 · November 10, 2022, 5:04am

Thank You. This was the exact output needed.

pulzappcheck890 · November 10, 2022, 11:31am

Hi,

i tried the above mentioned pipeline with v4lsrc and when i try to run the pipeline with a PNG image, the video feed seems to be lagging. The used p-line as follows:

gst-launch-1.0 videotestsrc pattern=red ! video/x-raw,width=1280,height=720,framerate=30/1 ! queue ! comp.sink_0 v4l2src device=/dev/exCAM io-mode=2 ! tee name=t2 ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw,width=960,height=540,format=NV12” ! queue ! queue ! comp.sink_1 filesrc location=/home/usr/Downloads/GST/logo-.png ! pngdec ! imagefreeze ! video/x-raw,framerate=30/1 ! videoconvert ! comp.sink_2 compositor name=comp sink_0::width=1280 sink_0::height=720 sink_0::zorder=0 sink_1::xpos=1179 sink_1::ypos=619 sink_1::width=100 sink_1::height=100 sink_1::zorder=2 sink_2::xpos=179 sink_2::ypos=619 sink_2::width=500 sink_2::height=100 sink_2::zorder=1 ! videoconvert ! ximagesink

Also, when i try to run without any image, the pipeline seems to be functioning with no lag. The p-line as follows:

gst-launch-1.0 videotestsrc pattern=red ! video/x-raw,width=1280,height=720,framerate=30/1 ! queue ! comp.sink_0 v4l2src device=/dev/exCAM io-mode=2 ! tee name=t2 ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! videorate ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! queue ! “video/x-raw,width=960,height=540,format=NV12” ! queue ! queue ! comp.sink_2 compositor name=comp sink_0::width=1280 sink_0::height=720 sink_0::zorder=0 sink_1::xpos=1179 sink_1::ypos=619 sink_1::width=100 sink_1::height=100 sink_1::zorder=2 sink_2::xpos=179 sink_2::ypos=619 sink_2::width=500 sink_2::height=100 sink_2::zorder=1 ! videoconvert ! ximagesink

How to eliminate this problem?

Thanks in advance.

Honey_Patouceul · November 10, 2022, 12:48pm

You may enclose your code into code blocks starting and closing with a line having just : ```
This would save many time when trying your case.

I don’t have a real V4L JPG camera, so I emulated one:

gst-launch-1.0 \
  videotestsrc ! video/x-raw,width=640,height=480,framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! nvvidconv ! 'video/x-raw(memory:NVMM),width=1920,height=1080' ! nvjpegenc ! image/jpeg,width=1920,height=1080,framerate=30/1 ! tee name=jpgcam \
  videotestsrc pattern=red ! video/x-raw,width=1280,height=720,framerate=30/1,format=RGBA ! queue ! comp.sink_0  \
  jpgcam. ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! 'video/x-raw,width=500,height=500,format=RGBA,pixel-aspect-ratio=1/1' ! queue ! comp.sink_1  \
  filesrc location=/usr/share/jetsonpowergui/logo/nv_logo.png ! pngdec ! imagefreeze ! video/x-raw,framerate=30/1 ! videoscale add-borders=0 ! 'video/x-raw,width=500,height=100,pixel-aspect-ratio=1/1' ! videoconvert ! video/x-raw,format=RGBA ! queue ! comp.sink_2  \
  compositor name=comp sink_0::width=1280 sink_0::height=720 sink_0::zorder=0 sink_1::xpos=1179 sink_1::ypos=619 sink_1::width=100 sink_1::height=100 sink_1::zorder=2 sink_2::xpos=179 sink_2::ypos=619 sink_2::width=500 sink_2::height=100 sink_2::zorder=1 ! videoconvert ! ximagesink -v

pulzappcheck890 · November 11, 2022, 6:07am

The pipeline works by making the image transparent. But, when i try to make the webcam video size bigger,
the frame-rate drops relatively. The P-line as follows:

gst-launch-1.0 videotestsrc pattern=red ! video/x-raw,width=1280,height=720,framerate=30/1,format=RGBA ! queue ! comp.sink_0 v4l2src device=/dev/exCAM io-mode=2 ! queue ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! ‘video/x-raw,width=500,height=500,format=RGBA,pixel-aspect-ratio=1/1’ ! queue ! comp.sink_1 filesrc location=/home/usr/Downloads/GST/-logo-.png ! pngdec ! imagefreeze ! video/x-raw,framerate=30/1 ! videoscale add-borders=0 ! ‘video/x-raw,width=500,height=100,pixel-aspect-ratio=1/1’ ! videoconvert ! video/x-raw,format=RGBA ! queue ! comp.sink_2 compositor name=comp sink_0::width=1280 sink_0::height=720 sink_0::zorder=0 sink_1::xpos=79 sink_1::ypos=19 sink_1::width=970 sink_1::height=540 sink_1::zorder=2 sink_2::xpos=179 sink_2::ypos=19 sink_2::width=500 sink_2::height=100 sink_2::zorder=2 ! videoconvert ! ximagesink -v

How can we achieve this while using NVIDIA encoders to function without frame drop?

Thanks In advance

Honey_Patouceul · November 13, 2022, 9:22pm

Please help me to help you.
Once this is ok, better create a new topic for this new issue.

system · November 27, 2022, 9:22pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.