Need help learning GStreamer C API on TX2

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

Folks, I’m hoping someone can help me with my attempts to learn how to use GStreamer on the TX2

To set the scene I’ve little to no experience of C beyond “Hello World”. I’ve a reasonable amount of experience using C# on Windows but I know this is a very different beast.

I’m making my way through the GStreamer tutorials and I’ve gotten stuck on tutorial 2, not a good sign. I’m probably making things more complex than they need to be because the standard steps in the tutorial don’t work on the Jetson but I figure I need to work this out.

The gst-launch pipeline that works on the Jetson is this

gst-launch-1.0 videotestsrc ! vertigotv ! nvvidconv ! 'video/x-raw(memory:NVMM), format=(string)NV12' ! autovideosink

I can get this to run using C if I leave out the ‘vertigotv’ element but if I include it then I need to use ‘nvvidconv’ with parameters and that’s where the wheels start coming off. I can’t work out how to set the parameters on the ‘nvvidconv’ element.

I believe the relevant sections of code are

GstElement *pipeline, *source, *sink, *filter, *converter;
GstCaps *capsnvvidconv;

converter = gst_element_factory_make ("nvvidconv", "converter");

capsnvvidconv = gst_caps_from_string("video/x-raw(memory:NVMM), format=(string)NV12");

gst_bin_add_many (GST_BIN (pipeline), source, filter, converter, sink, NULL);

/* FAILS at the following block */
if (gst_element_link_filtered (filter, converter, capsnvvidconv) != TRUE ) { 
  g_printerr ("Second element link failed.\n"); 
  gst_object_unref (pipeline);
  return -1;
}

I’ve found this forum post noting that using creating CAPS using ‘(memory:NVMM)’ can be tricky so I’m following their suggestions (I think)

https://devtalk.nvidia.com/default/topic/934515/jetson-tk1/using-x-raw-memory-nvmm-in-gstreamer-program/

Full code below, any help greatly appreciated.

#include <gst/gst.h>

int
main (int argc, char *argv[])
{
  GstElement *pipeline, *source, *sink, *filter, *converter;
  GstCaps *capsnvvidconv;
  GstBus *bus;
  GstMessage *msg;
  GstStateChangeReturn ret;

  /* Initialize GStreamer */
  gst_init (&argc, &argv);

  /* Create the elements */
  source = gst_element_factory_make ("videotestsrc", "source");
  filter = gst_element_factory_make ("vertigotv", "filter");
  converter = gst_element_factory_make ("nvvidconv", "converter");
  sink = gst_element_factory_make ("autovideosink", "sink");

  capsnvvidconv = gst_caps_from_string("video/x-raw(memory:NVMM), format=(string)NV12");
  if(!capsnvvidconv) {
    g_printerr ("Caps could not be created.\n");
    return -1;	  
  }
  /* Create the empty pipeline */
  pipeline = gst_pipeline_new ("test-pipeline");

  if (!pipeline || !source || !sink) {
    g_printerr ("Not all elements could be created.\n");
    return -1;
  }

  /* Build the pipeline */
  
  gst_bin_add_many (GST_BIN (pipeline), source, filter, converter, sink, NULL);
  
  if (gst_element_link (source, filter) != TRUE ) {
    g_printerr ("First element link failed.\n");
    gst_object_unref (pipeline);
    return -1;
  }
  
  if (gst_element_link_filtered (filter, converter, capsnvvidconv) != TRUE ) {
    g_printerr ("Second element link failed.\n");
    gst_object_unref (pipeline);
    return -1;
  }

  if (gst_element_link (converter, sink) != TRUE ) {
    g_printerr ("Third element link failed.\n");
    gst_object_unref (pipeline);
    return -1;
  }
  
  /* Modify the source's properties */
  g_object_set (source, "pattern", 0, NULL);

  /* Start playing */
  ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
  if (ret == GST_STATE_CHANGE_FAILURE) {
    g_printerr ("Unable to set the pipeline to the playing state.\n");
    gst_object_unref (pipeline);
    return -1;
  }

/* Remaining error parsing and cleanup code omitted for brevity */
2

As ever, after spending a day or so on this, I worked it out ten minutes after posting the question online. I’ll leave this up in case anyone else encounters a similar problem.

My issue was misunderstanding the syntax of ‘gst_element_link_filtered’, I was assuming it was in reading order i.e. (first element, second element, caps to be applied to the second element).

It’s actually (first element, second element, caps to be applied BETWEEN them). In hindsight this makes more sense and is also explained in the documentation, I just didn’t understand it.

So… old, non-working code

if (gst_element_link_filtered (filter, converter, capsnvvidconv) != TRUE ) {
    g_printerr ("Second element link failed.\n");
    gst_object_unref (pipeline);
    return -1;
  }

  if (gst_element_link (converter, sink) != TRUE ) {
    g_printerr ("Third element link failed.\n");
    gst_object_unref (pipeline);
    return -1;
  }

Proper working code

if (gst_element_link (filter, converter) != TRUE ) {
    g_printerr ("Second element link failed.\n");
    gst_object_unref (pipeline);
    return -1;
  }

  if (gst_element_link_filtered (converter, sink, capsnvvidconv) != TRUE ) {
    g_printerr ("Third element link failed.\n");
    gst_object_unref (pipeline);
    return -1;
  }