How to reduce nvds_obj_enc_process latency?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): GeForce 4090
• DeepStream Version: 6.4
• JetPack Version (valid for Jetson only): NA
• TensorRT Version: 8.6
• NVIDIA GPU Driver Version (valid for GPU only): 535
• Issue Type( questions, new requirements, bugs): question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing): NA
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description): NA

I attach a probe to a queue sink in my pipeline to encode detected objects, the probe function is:

GstPadProbeReturn enc_buffer_probe (
    GstPad * pad, 
    GstPadProbeInfo * info, 
    gpointer u_data)
{
    GstBuffer *buf = (GstBuffer *) info->data;

    NvDsObjEncCtxHandle crop_ctx = (NvDsObjEncCtxHandle)u_data;

    GstMapInfo inmap = GST_MAP_INFO_INIT;
    if (!gst_buffer_map (buf, &inmap, GST_MAP_READ)) 
    {
        GST_ERROR ("input buffer mapinfo failed");
        return GST_PAD_PROBE_DROP;
    }
    NvBufSurface *ip_surf = (NvBufSurface *) inmap.data;
    gst_buffer_unmap (buf, &inmap);

    NvDsObjectMeta *obj_meta = NULL;
    NvDsMetaList *l_frame = NULL;
    NvDsMetaList *l_obj = NULL;
    NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta (buf);

    for (l_frame = batch_meta->frame_meta_list; l_frame != NULL; l_frame = l_frame->next) 
    {
        NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) (l_frame->data);

        for (l_obj = frame_meta->obj_meta_list; l_obj != NULL; l_obj = l_obj->next) 
        {
            obj_meta = (NvDsObjectMeta *) (l_obj->data);

            NvDsObjEncUsrArgs userData = { 0 };
            /* Preset */
            userData.isFrame = 0;
            /* To be set by user */
            userData.saveImg = FALSE;
            userData.attachUsrMeta = TRUE;
            /* Set if Image scaling Required */
            userData.scaleImg = FALSE;
            userData.scaledWidth = 0;
            userData.scaledHeight = 0;
            /* Quality */
            userData.quality = 100;
            /*Main Function Call */
            nvds_obj_enc_process (crop_ctx, &userData, ip_surf, obj_meta, frame_meta);
        }
    }
    nvds_obj_enc_finish (crop_ctx);
    return GST_PAD_PROBE_OK;
}

I collected latency using GStreamer tracing subsystem with: GST_TRACERS="latency(flags=pipeline+element)" GST_DEBUG=GST_TRACER:7

sample3
sample31403×360 46.2 KB

sample2
sample21403×360 46.6 KB

sample1
sample11403×360 43.8 KB

The latency of other the queues in pipeline are in the order of nanoseconds, but the latency of the queue element (queue2) whose sink I attach the probe function (enc_buffer_probe) is very high, and often are close to 10 milliseconds. enc_buffer_probe is the only probe attach to queue2. Is this latency expect for nvds_obj_enc_process()? If not, how do I reduce the latency?

3

The probe functions are blocking callbacks which will hold the GstBuffer before the function exit. The JPEG image encoding is time consuming operations. 10 ms looks OK for the whole batch frames encoding.

4

@Fiona.Chen

Thank for the reply.

Just for clarification: by “whole batch frames encoding” do you mean whole frames encoding or objects encoding? I don’t encode frames, scaling and save image are both disabled, I encode all objects in all frames in a batch, is 10ms still in the normal latency range in that case?

5

I mean all the objects in all frames in the batch encoding.

1 Like
6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.