Latency measurement in Deepstream pipeline

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Hardware Platform (GPU)
• DeepStream Version nvidia/deepstream:7.1-gc-triton-devel
• Issue Type(questions, new requirements)

Hey,
We’re working on latency measurement for pipeline elements and ran into a minor issue.
We’re enabling latency measurement by setting NVDS_ENABLE_LATENCY_MEASUREMENT=1.

When calling nvds_measure_buffer_latency, it prints latency details for every frame and source:

************BATCH-NUM = 1220**************
Source id = 0 Frame_num = 1220 Frame latency = 6.22681 (ms)
************BATCH-NUM = 1221**************
Source id = 0 Frame_num = 1221 Frame latency = 7.13281 (ms)

Since these logs aren’t useful to us, we added a new C++ function (bindings) that returns this data instead of printing it.
This allows us to control the data frequency and submit it as metrics, and it’s working well.
However, we still see these logs for each frame:

Encode Latency = 4.392822  
Encode Latency = 3.986084

I couldn’t find the source of these logs in the repo.
Any help in disabling or removing these logs would be greatly appreciated!
In a later phase, I will open a PR to add the option to retrieve latency information, which is very useful for metrics.
Thanks!

3

You need to modify the source code of the deepstream-app and rebuild this sample.

deepstreamsources\apps\sample_apps\deepstream-app\deepstream_app.c
static GstPadProbeReturn
latency_measurement_buf_prob (GstPad * pad, GstPadProbeInfo * info,
    gpointer u_data)
4

I checked the file:
/apps/sample_apps/deepstream-app/deepstream_app.c
but I didn’t find any Encode Latency prints.

To clarify, the log:

************BATCH-NUM = 1220**************
Source id = 0 Frame_num = 1220 Frame latency = 6.22681 (ms)

comes from the C++ file:
/opt/nvidia/deepstream/deepstream/bindings/bindfunctions.cpp
which I was able to modify and rebuild successfully.

However, I couldn’t find any reference to Encode Latency.
I even searched the entire sources folder:
/opt/nvidia/deepstream/deepstream/sources
but found nothing.

5

About the Encode Latency log, this is a log of the encoder which is not open source now. This print is only controlled by the macro NVDS_ENABLE_LATENCY_MEASUREMENT. It can’t be closed if you set that macro to 1 now.

6

Can I control it with environment variable like GST_DEBUG: encoder:2 or something similar?

7

No. Since it is already controlled by the macro, only this macro can turn it on or off.

8

I’ve noticed that setting GST_DEBUG=2 disables this Encode Latency log, but it also disables my callback probes.
I’m using deepstream_python_apps with the C++ bindings.
Are there any plans to introduce a better method for latency measurement?

9

No. You can put forward the points that need to be improved in detail, and we will discuss and confirm that. Thanks

10

Here are my suggestions for improving latency measurement.

Method 1:

In this approach, we introduced a new C++ binding function that returns latency instead of printing it.
While this worked for us, it resulted in an “Encode Latency” log for each frame.
Additionally, this method is not ideal because it measures the latency from the beginning of the pipeline (source/sink latency) rather than the latency of a specific element.

class LatencyMeasurement():
    def add_element(self, element: Gst.Element):
        element_name = element.get_name()
        src_pad = element.get_static_pad("src")
        src_pad.add_probe(Gst.PadProbeType.BUFFER, self.create_callback(element_name), None)

    def create_callback(self, element_name: str):
        def callback(pad: Gst.Pad, probe_info: Gst.PadProbeInfo, user_data) -> Gst.PadProbeReturn:
            gst_buffer = probe_info.get_buffer()
            latency_info = pyds.nvds_get_buffer_latency(hash(gst_buffer))
            for item in latency_info:
                source_id = item.get("source_id")
                frame_num = item.get("frame_num")
                latency = item.get("latency")
                self.metrics.submit(element_name, source_id, frame_num, latency)

            return Gst.PadProbeReturn.OK
        return callback

Method 2: Preferred Approach for Element Latency

This method introduces a new function, nvds_get_element_latency, which directly retrieves the current latency of a specific element.

class LatencyMeasurement():
    def add_element(self, element: Gst.Element):
        element_name = element.get_name()
        latency_info = pyds.nvds_get_element_latency(element)
        for item in latency_info:
            source_id = item.get("source_id")
            frame_num = item.get("frame_num")
            latency = item.get("latency")
            self.metrics.submit(element_name, source_id, frame_num, latency)

Method 3: Preferred Approach for Pipeline Latency

This method introduces nvds_get_pipeline_latency, a function that retrieves the current latency of the entire pipeline.

class LatencyMeasurement():
    def add_pipeline(self, pipeline: Gst.Pipeline):
        latency_info = pyds.nvds_get_pipeline_latency(pipeline)
        self.metrics.submit(pipeline.get_name(), latency_info.latency)

I appreciate your help!!
Thanks

11

Thank you for your suggestion. The perf printing is just a debug requirement, and we have no plans to add control at this time.