Gstreamer OpenCv Appsink to Appsrc Link Problem

Autonomous Machines Jetson & Embedded Systems Jetson TX2
,
1

Hello; I want to integrate opencv in Gstreamer, to give details, I want to read the data in v4l2src with appsink, do opencv operations and transfer it to a unicast broadcast with appsrc, but when I do this, the pipeline constantly resets itself, I couldn’t figure out why. I would be happy if you could help.

if(gst_element_link_many(GMainElement.v4l2src, GMainElement.source2convertcapsfilter GMainElement.tee, NULL) !=TRUE ||

gst_element_link_many (GUnicastElement.queue_unicast,
GUnicastElement.videoconvert1, GUnicastElement.cairooverlay,
GUnicastElement.videoconvert2,
GUnicastElement.nvvidconv, GUnicastElement.convert2encodercapsfilter, GUnicastElement.nvv4l2h265enc, GUnicastElement.encoder2muxcapsfilter,
GUnicastElement.mpegtsmux, GUnicastElement.mux2payloadcapsfilter, GUnicastElement.rtpmp2tpay, GUnicastElement.payload2udpcapsfilter, GUnicastElement.udpsink, NULL) !=TRUE ||

gst_element_link_many (GMulticastElement.queue_multicast,
GMulticastElement.nvvidconv, GMulticastElement.convert2encodercapsfilter, GMulticastElement.nvv4l2h265enc, GMulticastElement.encoder2muxcapsfilter,
GMulticastElement.mpegtsmux, GMulticastElement.mux2payloadcapsfilter, GMulticastElement.rtpmp2tpay, GMulticastElement.payload2udpcapsfilter, GMulticastElement.udpsink, NULL) !=TRUE ||

gst_element_link_many (GAppsinkElement.queue_appsink,
GAppsinkElement.nvvidconv, GAppsinkElement.convert2encodercapsfilter, GAppsinkElement.nvv4l2h265enc, GAppsinkElement.encoder2muxcapsfilter,
GAppsinkElement.mpegtsmux, GAppsinkElement.mux2payloadcapsfilter, GAppsinkElement.appsinkCV, NULL) != TRUE ||
gst_element_link_many (GAppsinkElement.appsourceCV, GaAppsinkElement.unicast_cv NULL) != TRUE)

tee_unicast_pad = gst_element_get_request_pad(GMainElement.tee, src);
queue_unicast_pad = gst_element_get_static_pad(GUnicastElement.queue_unicast, “sink”);

tee_multicast_pad = gst_element_get_request_pad(GMainElement.tee, src);
queue_multicast_pad = gst_element_get_static_pad(GUnicastElement.queue_multicast, “sink”);

tee_appsink_pad = gst_element_get_request_pad(GMainElement.tee, src);
queue_appsink_pad = gst_element_get_static_pad(GAppsinkElement.queue_appsink, “sink”);

if (gst_pad_link(GUnicastElement.tee_pad, GUnicastElement.queue_pad) != GST_PAD_LINK_OK ||
gst_pad_link(GMulticastElement.tee_pad, GMulticastElement.queue_pad) != GST_PAD_LINK_OK ||
gst_pad_link(GAppsinkElement.tee_pad, GAppsinkElement.queue_pad) != GST_PAD_LINK_OK)

Appsink Method →
void setAppSink(){
_appsinkCV = gst_element_link_factory_make(appsink, “appsink0”);
g_object_set(G_OBJECT(_appsinkCV), emit-signals, TRUE, synchronized, FALSE, NULL);
}

Appsrc Method →
void setAppSource(){
_appsourceCV = gst_element_link_factory_make(appsrc, “appsrc0”);
g_object_set(G_OBJECT(_appsourceCV), caps, “gst_caps_new_simple”(…), NULL);
}

Callback Method →
g_signal_connect(_appsinkCV, “new-sample”, G_CALLBACK(callback_function_opencv), _appsourceCV);

3

Hi,
There are some similar samples:
Displaying to the screen with OpenCV and GStreamer - #9 by DaneLLL
Stream processed video with OpenCV on Jetson TX2 - #5 by DaneLLL
OpenvCV, Gstreamer, Python camera capture/access and streaming to RTP

Please apply your gstreamer pipelines to the samples and see if it works.

4

As far as I understand from these tabs, I need to do the encode and mux operations again after the appsrc, I’m not sure if I understood correctly.

5

Hi,
The examples show encoding to a video file and UDP in cv2.VideoWriter(). You may refer to it and give it a try.

6

Hi;
The problem is not solved, I think I have a problem with the linking, it does not give an error output.

7

Pipeline is constantly resetting itself. In fact, it goes to the last line of the code and starts broadcasting, but then it resets itself. What could be the reason for this? I couldn’t get any output in debug mode. There seems to be no error. @DaneLLL @AlejandroSolis-RidgeRun @michael_gruner @DavidSoto-RidgeRun

I have seen your studies on this subject, maybe you can help me too, thank you for your help.

8

Hi @GarroshHellscream

  1. Please provide a full example. Nothing in the code above seems suspicious to me.

  2. What’s the output of running the app with GST_DEBUG=2 ?

Pipelines can’t restart themselves on their own, so it must be application logic somewhere. The first place I would look is the bus callback where you handle error cases, in most cases there’s where one restarts the pipelines.

9

First of all, thank you very much for your answer @michael_gruner . When I run it in debug mode, it does not give any output. The pipeline seems to be working properly, but it enters a loop that repeats itself. Can we say that it is related to the callback method I use for OpenCV? Is there such a possibility?

If you have an example that you are sure works, I can try it too. An example of a callback function where we receive the image with opencv and push it to appsource with appsink emit signal.