How to get request pad on nvstreamdemux in runtime?

Hello,

I start deepstream application and want to add source on runtime. In my pipeline have to use nvstreamdemux. When create pipeline at first time. It’s Ok. But when pipeline started. Add new source. The nvstreamdemux cannot request new source pad? How to do this? Thank you

There is this example Nvidia provides:
https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/tree/master/runtime_source_add_delete

I think you want the stream muxer. It multiplexes multiple sources (or just one) and attaches metadata to the buffer. The demuxer does the opposite.

It was kind of confusing initially to me as well. Usually a muxer goes at the end of the pipeline and the demuxer at the beginning, but not in this case (and for good reason if you read the documentation).

Helpful documentation:
https://docs.nvidia.com/metropolis/deepstream/plugin-manual/index.html

To answer your question directly: you request a new pad from the stream muxer with a pad name and gst_element_get_request_pad since the pads are request pads. A template seems not to be supported (or I am trying something wrong) so you either either have to keep sink count manually (eg. sink_123) or use the numsinkpads member (what my implementation does). Some sort of locking may be needed to avoid requesting identical pads (I had this problem in my implementation, so I put a mutex around the pad request code).

The hard part, what I can’t figure out, is how to modify batch-size in the playback state. I don’t think it’s supported so you have to set batch-size to your expected maximum early on.

Hello mdegans,

Thank you for your reply. I do put a mutex around the pad request code, but it still failed to request new pad for nvstreamdemux when running. Did you successfully request new source pad on demuxer? Can you give more detail about how did you do?

So, to figure out how to link a pad in gstreamer you need to inspect the element to see what pads it supports. Assuming you want nvstreamdemux to demux batched buffers and metadata, you can inspect the element with gst-inspect and look at the kind of pads it has.

$ gst-inspect-1.0 nvstreamdemux
Factory Details:
  Rank                     primary (256)
  Long-name                Stream demultiplexer

<b>...</b>

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                 format: { (string)NV12, (string)RGBA }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
  
[b]  SRC template: 'src_%u'
    Availability: On request[/b]
    Capabilities:
      video/x-raw(memory:NVMM)
                 format: { (string)NV12, (string)RGBA }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
...

The sink pad is “always” there, so you can link to it with gst_element_link or with gst_element_get_static_pad and link the pads. The src pads (buffers come out of them), however, are request pads so you need to request them with gst_element_get_request_pad(pad_name) in C (or elemement_foo.get_request_pad(pad_name) in Python, Vala, or Genie.

Example in C.

Some Nvidia functions to do the job easily can be found in /opt/nvidia/deepstream/deepstream-4.0/apps/apps-common/src/deepstream_common.c

/*
 * Copyright (c) 2018-2019, NVIDIA CORPORATION. All rights reserved.
 *
 * Permission is hereby granted, free of charge, to any person obtaining a
 * copy of this software and associated documentation files (the "Software"),
 * to deal in the Software without restriction, including without limitation
 * the rights to use, copy, modify, merge, publish, distribute, sublicense,
 * and/or sell copies of the Software, and to permit persons to whom the
 * Software is furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
 * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
 * DEALINGS IN THE SOFTWARE.
 */
...
gboolean
link_element_to_<b>streammux</b>_sink_pad (GstElement *streammux, GstElement *elem,
    gint index)
{
  gboolean ret = FALSE;
  GstPad *mux_sink_pad = NULL;
  GstPad *src_pad = NULL;
  gchar pad_name[16];

  if (index >= 0) {
    g_snprintf (pad_name, 16, "sink_%u", index);
    pad_name[15] = '\0';
  } else {
    strcpy (pad_name, "sink_%u");
  }

  mux_sink_pad = gst_element_get_request_pad (streammux, pad_name);
  if (!mux_sink_pad) {
    NVGSTDS_ERR_MSG_V ("Failed to get sink pad from streammux");
    goto done;
  }

  src_pad = gst_element_get_static_pad (elem, "src");
  if (!src_pad) {
    NVGSTDS_ERR_MSG_V ("Failed to get src pad from '%s'",
                        GST_ELEMENT_NAME (elem));
    goto done;
  }

  if (gst_pad_link (src_pad, mux_sink_pad) != GST_PAD_LINK_OK) {
    NVGSTDS_ERR_MSG_V ("Failed to link '%s' and '%s'", GST_ELEMENT_NAME (streammux),
        GST_ELEMENT_NAME (elem));
    goto done;
  }

  ret = TRUE;

done:
  if (mux_sink_pad) {
    gst_object_unref (mux_sink_pad);
  }
  if (src_pad) {
    gst_object_unref (src_pad);
  }
  return ret;
}
...

gboolean
link_element_to_<b>demux</b>_src_pad (GstElement *demux, GstElement *elem, guint index)
{
  gboolean ret = FALSE;
  GstPad *demux_src_pad = NULL;
  GstPad *sink_pad = NULL;
  gchar pad_name[16];

  g_snprintf (pad_name, 16, "src_%u", index);
  pad_name[15] = '\0';

  demux_src_pad = gst_element_get_request_pad (demux, pad_name);
  if (!demux_src_pad) {
    NVGSTDS_ERR_MSG_V ("Failed to get sink pad from demux");
    goto done;
  }

  sink_pad = gst_element_get_static_pad (elem, "sink");
  if (!sink_pad) {
    NVGSTDS_ERR_MSG_V ("Failed to get src pad from '%s'",
                        GST_ELEMENT_NAME (elem));
    goto done;
  }

  if (gst_pad_link (demux_src_pad, sink_pad) != GST_PAD_LINK_OK) {
    NVGSTDS_ERR_MSG_V ("Failed to link '%s' and '%s'", GST_ELEMENT_NAME (demux),
        GST_ELEMENT_NAME (elem));
    goto done;
  }

  ret = TRUE;

done:
  if (demux_src_pad) {
    gst_object_unref (demux_src_pad);
  }
  if (sink_pad) {
    gst_object_unref (sink_pad);
  }
  return ret;
}

There are more useful functions in that file (eg. unlink elements as well). Most of it seems to be pretty much boilerplate Gstreamer. If you haven’t aready done so, i’d recommend going through the Gstreamer tutorials as they explain how to link always, request, and sometimes pads, the last of which you will probably need for linking various kinds of sources that create pads themselves (rather than on request).

Edit: note that in those above functions you still need to manage the index count manually, making sure not to request the same pad twice. If you need to release and “recycle” a pad, you can do so using the unlink_element_from_streammux_sink_pad in the above file or similar gstreamer boilerplate from the tutorials.

Thank you mdegans,

I did all thing as you said before, check available of pad source nvstreamdemux, research on gstreammer tutorial how to link pad on runtime. Used function link and unlink pad in deepstream-common.c, but it is not working.

When pipeline is running, I check gst_element_get_static_pad(demuxer) before gst_element_get_request_pad(demuxer); but twice of it given NULL pad. I pretty sure that index name pad is unique.

So if you’re doing something like …

pad = gst_element_get_request_pad(demuxer, "src_0");

… and that is returning NULL, something went wrong with requesting the pad. To view detailed debug information you can either hook up your app’s arguments to gst_init and specify --gst-debug-level=5 at the command line or set the GST_DEBUG environment variable to an integer. The debug level ranges from 0-9. 9 is not recommended (see links for details). That should give you some clues as to why the pad request is failing.

Otherwise, you can post a snippet of code here, or ask an Nvidia rep to take a look via PM (I do not work for Nvidia, please do not send me your source). It’s hard to tell what’s going wrong exactly without seeing the context.