Problems to embedded my GStreamer pipeline in GTK window

Hello Guys.
I’ve been trying to adapt this pipeline (that work very well with gst-luch-1.0)

rtspsrc location=rtsp://admin:123456@192.168.1.118:554/streaming/channels/101 latency=0 ! rtph265depay ! h265parse ! nvv4l2decoder ! nvvidconv ! capsfilter caps=video/x-raw,format=RGBA ! queue ! tee name=t t. ! queue ! comp.sink_0 t. ! queue ! videocrop top=315 left=520 right=520 bottom=315 ! videoscale ! comp.sink_1 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720 sink_1::xpos=960 sink_1::ypos=0 sink_1::width=320 sink_1::height=180 ! nvoverlaysink

to be embedded in GTK window.

as far i know, gtk have special sink called gtksink. The only problem this sink accept BGRx and BGRA format (nvoverlaysink can work RGBA)

so i tried to adapt my pipeline to:
rtspsrc location=rtsp://admin:123456@192.168.1.118:554/streaming/channels/101 latency=0 ! rtph265depay ! h265parse ! nvv4l2decoder ! nvvidconv ! capsfilter caps=video/x-raw,format=RGBA ! queue ! tee name=t t. ! queue ! comp.sink_0 t. ! queue ! videocrop top=315 left=520 right=520 bottom=315 ! videoscale ! comp.sink_1 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720 sink_1::xpos=960 sink_1::ypos=0 sink_1::width=320 sink_1::height=180 ! nvvidconv ! videoconvert ! capsfilter caps=“video/x-raw,format=BGRx” ! gtksink name=gtksink

( i could figured out if is possible to lunch this pipeline with only gst-luch-1.0)

So I wrote python script (actually Gemini-Pro wrote).

import gi
gi.require_version(‘Gtk’, ‘3.0’)
gi.require_version(‘Gst’, ‘1.0’)
from gi.repository import Gtk, Gst

Gst.init(None)

class GstVBoxWidget(Gtk.VBox):
def init(self, pipeline_description):
super().init()

    self.pipeline = Gst.parse_launch(pipeline_description)
    self.video_sink = self.pipeline.get_by_name('gtksink')

    self.video_area = Gtk.DrawingArea()
    self.pack_start(self.video_area, True, True, 0)

    # Handle the "sink-widget" property change
    self.video_sink.connect("notify::sink-widget", self.on_widget_change)

    self.pipeline.set_state(Gst.State.PLAYING)

def on_widget_change(self, sink, gparam):
    if sink.props.sink_widget:
        self.video_area.realize()  # Make sure the DrawingArea is realized
        window_handle = self.video_area.get_window().get_xid()
        sink.props.sink_widget.set_window_handle(window_handle)

def on_destroy(win):
Gtk.main_quit()

pipeline_desc = “”"
rtspsrc location=rtsp://admin:123456@192.168.1.118:554/streaming/channels/101 latency=0 !
rtph265depay ! h265parse ! nvv4l2decoder ! nvvidconv !
capsfilter caps=video/x-raw,format=RGBA ! queue ! tee name=t
t. ! queue ! comp.sink_0
t. ! queue ! videocrop top=315 left=520 right=520 bottom=315 ! videoscale ! comp.sink_1
nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1280 sink_0::height=720 sink_1::xpos=960 sink_1::ypos=0 sink_1::width=320 sink_1::height=180 !
nvvidconv ! videoconvert ! capsfilter caps=“video/x-raw,format=BGRx” ! gtksink name=gtksink
“”"

window = Gtk.Window()
widget = GstVBoxWidget(pipeline_desc)
window.add(widget)
window.connect(‘destroy’, on_destroy)
window.show_all()
Gtk.main()

But i got this output:
0:00:01.659832620 10053 0x201cf0f0 ERROR nvcompositor gstnvcompositor.c:1112:gst_nvcompositor_decide_allocation: not supported out caps
0:00:01.659885954 10053 0x201cf0f0 WARN aggregator gstaggregator.c:984:gst_aggregator_do_allocation: Failed to decide allocation
0:00:01.659929184 10053 0x201cf0f0 WARN aggregator gstaggregator.c:1070:gst_aggregator_update_src_caps: Allocation negotiation failed

Do you guys have any suggestion?

Hello @renan.jorge,

What happens if you run it using gst-launch-1.0 but replacing the gtksink with a fakesink ?

Also, you might be able to remove the videoconvert after the nvvidconv.

If you can, share the output using GST_DEBUG=GST_CAPS:5,GST_PAD:5 gst-launch…

Regards,
Andrew
support@proventusnova.com

Using fakesink i got this output:

it’s seems right to me

(edit: at first i did not see that you aked for GST_DEBUG=GST_CAPS:5,GST_PAD:5)
now with this comand i have the fallowing log:
fakesing_DEBUG_5.txt (433.8 KB)

@renan.jorge,

Thanks for sending back the output.

I see you also removed the nvvidconv, the videoconvert and the capsfilter. Ca you leave those on ?

Regards,
support@proventusnova.com

For sure:

Here image with GST_DEBUG=2

and here the log with GST_DEBUG=GST_CAPS:5,GST_PAD:5
nvvidconv_videoconvert_fakesink.txt (546.7 KB)

Thats interesting,

Might be the caps filter.

And what happens if you replace your entire pipe with somehting like:

videotestsrc ! videoconvert ! gtksink

In your app ?

Regards,
Andrew
support@proventusnova.com

that will possible work:
because this pipeline:

GST_DEBUG=2 gst-launch-1.0 rtspsrc location=rtsp://admin:123456@192.168.1.118:554/streaming/channels/101 latency=0 ! rtph265depay ! h265parse ! nvv4l2decoder ! nvvidconv ! gtksink

work perfectly.
The problems seems the nvcompositor caps do not work with gtksink

Yes, that is what I was thinking.

Whats interesting is that the nvvidconv should decouple the nv elements from regular elements.

I will try running the code on my side just to have some fun.

Regards,
Andrew
support@proventusnova.com