Deepstream osd display lagging like droping frames

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) : RTX 4070
• DeepStream Version : 6.4
• Issue Type( questions, new requirements, bugs): Question
Hi,
I’m trying to run 3 stage detector model. Here’s the latency report. The problem is after displaying on screen(nvosd->sink) for few seconds there is a lag. However in the latency report I don’t see that. I’m using rtmp stream.
Here the code snippet for creating source bin

static GstElement *
create_source_bin(guint index, gchar *uri)
{
    GstElement *bin = NULL, *uri_decode_bin = NULL;
    gchar bin_name[16] = {};

    g_snprintf(bin_name, 15, "source-bin-%02d", index);
    /* Create a source GstBin to abstract this bin's content from the rest of the
     * pipeline */
    bin = gst_bin_new(bin_name);

    /* Source element for reading from the uri.
     * We will use decodebin and let it figure out the container format of the
     * stream and the codec and plug the appropriate demux and decode plugins. */
    if (PERF_MODE)
    {
        uri_decode_bin = gst_element_factory_make("nvurisrcbin", "uri-decode-bin");
        g_object_set(G_OBJECT(uri_decode_bin), "file-loop", TRUE, NULL);
    }
    else
    {
        uri_decode_bin = gst_element_factory_make("uridecodebin", "uri-decode-bin");
    }

    if (!bin || !uri_decode_bin)
    {
        g_printerr("One element in source bin could not be created.\n");
        return NULL;
    }

    /* We set the input uri to the source element */
    g_object_set(G_OBJECT(uri_decode_bin), "uri", uri, NULL);

    /* Connect to the "pad-added" signal of the decodebin which generates a
     * callback once a new pad for raw data has beed created by the decodebin */
    g_signal_connect(G_OBJECT(uri_decode_bin), "pad-added",
                     G_CALLBACK(cb_newpad), bin);
    g_signal_connect(G_OBJECT(uri_decode_bin), "child-added",
                     G_CALLBACK(decodebin_child_added), bin);

    gst_bin_add(GST_BIN(bin), uri_decode_bin);

    /* We need to create a ghost pad for the source bin which will act as a proxy
     * for the video decoder src pad. The ghost pad will not have a target right
     * now. Once the decode bin creates the video decoder and generates the
     * cb_newpad callback, we will set the ghost pad target to the video decoder
     * src pad. */
    if (!gst_element_add_pad(bin, gst_ghost_pad_new_no_target("src", GST_PAD_SRC)))
    {
        g_printerr("Failed to add ghost pad in source bin\n");
        return NULL;
    }

    return bin;
}

Here’s how the pipeline is arranged:

gst_bin_add_many(GST_BIN(pipeline), queue1, pgie, queue2, tracker, queue3, sgie1, queue4,
                         sgie2, queue5, nvdslogger, queue6, nvvidconv, queue7, nvosd, queue8, sink, NULL);
        /* we link the elements together
         * nvstreammux -> nvinfer -> nvdslogger -> nvtiler -> nvvidconv -> nvosd
         * -> video-renderer */
        if (!gst_element_link_many(streammux, queue1, pgie, queue2, tracker, queue3, sgie1, queue4,
                                   sgie2, queue5, nvdslogger, nvvidconv, queue7, nvosd, sink, NULL))
        {
            g_printerr("Elements could not be linked. Exiting.\n");
            return -1;
        }

Any suggestion on this is highly appreciated. @fanzh @Morganh
log_14_rtsp.txt (2.3 MB)

3

How do you get there is a lag and how long is it?

4

Hi @yuweiw
When I’m visualizing the video on screen through osd, I find that there is delay like dropping of frames. This doesn’t happen initially. However after some time this starts to happen. Any help is highly appreciated

5

Could you help narrow down this issue by reducing the plugins in the pipeline like below?

source -> nvstreammux -> sink
6

Hi.
I’ve checked. It’s working fine when there are no pgie or sgie like you have pointed out. Any particular reason why the osd display slows down when pgie, sgie1 and sgie2 are added.

7

Yes. These will increase the loading of your card. It can cause the excessive frames accumulation. You can try to set interval parameter to the nvinfer to improve the performance.

8

Hi,
Thank you for your response and time. However I’m performing speed anpr, where speed violation is an algorithm. If there is a violation then I send the registration plate number. Interval parameter is there in pgie,sgie1(which are detection models). If I set interval==10 in both pgie and sgie1, then will this be synchronized?

9

You can only set the interval to the pgie. If there are no detected object for sgie, it will not do the inference.

10

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

11

@yuweiw Thanks for your time

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.