Change gst-nvinferserver "interval" value after pipeline is running

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Hardware Platform (Jetson / GPU) Jetson Orin Nano
• DeepStream Version 6.2
• Issue Type( questions, new requirements, bugs) Question

Hi, I’m interested in understanding whether it is possible to dynamically change the gst-nvinferserver’s “interval” property value after the pipeline is running.

Things I’ve tried:

  1. Set the property value in a callback function which has the gst-nvinferserver element in context. This does not give an error while setting the property value however this change of value is not reflected either.
    Example:
def callback():
    pgie.set_property("interval", 10)

def main():
    global pgie
    pgie = Gst.ElementFactory.make("nvinferserver", "primary-inference")
    if not pgie:
        sys.stderr.write(" Unable to create pgie \n")

    pgie.set_property('config-file-path', 'config.txt')
  1. Resetting a new config file in the callback function does not help either.
    Example:
def callback():
    pgie.set_property("config-file-path", 'new-config.txt')

def main():
    global pgie
    pgie = Gst.ElementFactory.make("nvinferserver", "primary-inference")
    if not pgie:
        sys.stderr.write(" Unable to create pgie \n")

    pgie.set_property('config-file-path', 'config.txt')

I’ve gone through a few topics related to this but haven’t been able to locate an answer that works.

  1. How to hot reload configuration changes dynamicall while pipeline is running
  2. NVDSAlalytics - change line coordinates on the fly - #5 by bcao
  3. Toggling nvinfer inference processing on/off at runtime - #4 by miguel.taylor

Does gst-nvinferserver support dynamic changes to it’s configurations?

3

You can try to use the gst_nvevent_infer_interval_update in the opt\nvidia\deepstream\deepstream\sources\libs\gstnvdscustomhelper\gst-nvdscustomevent.c to update the interval.

4

Hi yuweiw, thanks for the response. Can you please provide a working example for how to use/invoke the “gst_nvevent_infer_interval_update” function? I don’t see any python bindings for this function so I’m not sure how to invoke it

5

We currently do not have such a Python demo. You can try to bind that by referring to the BINDINGSGUIDE.md yourself.

6

Will check it out. A few observations

  1. With the example below
def callback():
    pgie.set_property("interval", 10)

def main():
    global pgie
    pgie = Gst.ElementFactory.make("nvinferserver", "primary-inference")
    if not pgie:
        sys.stderr.write(" Unable to create pgie \n")

    pgie.set_property('config-file-path', 'config.txt')

I’ve noticed that if I do a pgie.get_property(‘interval’) the next time it enters the callback function - it shows me the updated interval value on the console but it does not affect the number of inference requests being sent to the Triton Server. Is the interval sent as part of the server initialization when the pipeline starts running? I don’t think that should be the case though. Any thoughts on this?

  1. Following up on the previous point, I’ve also tried restarting the pipeline to see if this is able to change the interval value
def callback():
    pipeline.set_state(Gst.State.NULL)
    pgie.set_property("interval", 10)
    time.sleep(5)
    pipeline.set_state(Gst.State.PLAYING)

def main():
    global pgie
    pgie = Gst.ElementFactory.make("nvinferserver", "primary-inference")
    if not pgie:
        sys.stderr.write(" Unable to create pgie \n")

    pgie.set_property('config-file-path', 'config.txt')

However this does not restart the pipeline correctly… It processes two frames and then freezes. What’s the right way to restart the pipeline? I’ve tried this with a file source and a usb-source. It freezes for both (on restart).

  1. I also tried making the interval property a dynamically controllable property as mentioned here (Dynamic Controllable Parameters) and also set a GST_PARAM_CONTROLLABLE flag when the interval property is created in gst_nvinfer_server_class_init() at /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvinferserver/gstnvinferserver.cpp

However, this hasn’t been able to help. Are there any cpp/python examples using dynamic controllable parameters? Would this be the right approach?

7

Through your method, it is possible to dynamically set variables normally to the plugin, but the internal variables of nviferserver updated through events. So you can set the variables dynamically, but it doesn’t actually work. You can refer to our source code opt\nvidia\deepstream\deepstream\sources\gst-plugins\gst-nvinferserver about how this variable work within the plugin.

8

Noted. I’ve made a few changes to use gst_nvevent_infer_interval_update from
opt\nvidia\deepstream\deepstream\sources\libs\gstnvdscustomhelper\gst-nvdscustomevent.c
Following this link I created a function that is callable from Python and within the binding function it creates a custom event followed by which it tries to send the event to the sink element of nvinferserver. However, I’m getting a segmentation fault while trying to send the event.

In my python code, I invoke the bindings function like

def callback():
    interval = 10
    pyds.gst_element_send_nvevent_interval_update(sys.getsizeof(pgie), '0', interval)

def main():
    global pgie
    pgie = Gst.ElementFactory.make("nvinferserver", "primary-inference")
    if not pgie:
        sys.stderr.write(" Unable to create pgie \n")

    pgie.set_property('config-file-path', 'config.txt')

In bindfunctions.cpp, I’ve created

        m.def("gst_element_send_nvevent_interval_update",
              [](size_t gst_element, char* stream_id, int interval) {
                  auto *element = reinterpret_cast<GstElement *>(gst_element);
                  std::cout << "Reached gst_element_send_nvevent_interval_update\n";
                  //Code below seg-faults
                  return gst_element_send_event(element, gst_nvevent_infer_interval_update(stream_id, interval));
              },
              pydsdoc::methodsDoc::gst_element_send_nvevent_interval_update);

I’ve made a few other changes, but the overall compilation works fine and I’m able to reach the binding function. However it results in a seg fault when trying to send the event (I’m guessing due to improper casting). Any thoughts on this?

If the casting works fine then I’m hoping to reach gst_nvinfer_server_sink_event() in opt\nvidia\deepstream\deepstream\sources\gst-plugins\gst-nvinferserver\gstnvinferserver.cpp. Am I correct in assuming this?

BTW I have to also pass stream_id to gst_nvevent_infer_interval_update. How can I get this value in the python workflow?

9

Could you attach the log of the crash info? Or you can just debug that in your enviroment with gdb.

You can use the source_id from the NvDsFrameMeta.

12

hi there, this is a bit irrelevant to the problem you’re asking about, but I was kinda hoping you could help me out.

I’m trying to statically set the interval value to adjust number of batches to skip in my pipeline but for some reason it’s not working properly,

Set the property value in a callback function which has the gst-nvinferserver element in context. This does not give an error while setting the property value however this change of value is not reflected either.

for me i get this same behavior, I tried setting it from either the config file or in the pipeline code itself but no use, as I understood you already managed to do that but can’t change it dynamically so can you share with me exactly how you did it statically?

13

It is different for your scenario. If you want to set the interval statically, basically you can just set the interval parameter in your config file. You can file a new topic and describes your environment and pipeline.