How to check inference time for a frame when using deepstream

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU) Jetson AGX
• DeepStream Version 7.0
**• JetPack Version (valid for Jetson only) 6.0 GA
**• TensorRT Version 8.6
**• NVIDIA GPU Driver Version (valid for GPU only) 540.3.0
**• Issue Type( questions, new requirements, bugs) questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

How to check inference time for a frame when using deepstream-test5-app

2

Hi,

We have an open-source tool called gst-shark for GStreamer profiling that you can use to measure interlatency and obtain the nvinfer processing time.

Installation:

sudo apt install -y libgraphviz-dev
git clone git@github.com:RidgeRun/gst-shark.git
cd gst-shark/
meson build --prefix /usr/
ninja -C build
sudo ninja -C build install

To see all the available tracers run:

gst-inspect-1.0 | grep tracers
sharktracers:  buffer (GstTracerFactory)
sharktracers:  bitrate (GstTracerFactory)
sharktracers:  queuelevel (GstTracerFactory)
sharktracers:  framerate (GstTracerFactory)
sharktracers:  scheduletime (GstTracerFactory)
sharktracers:  interlatency (GstTracerFactory)
sharktracers:  proctime (GstTracerFactory)
sharktracers:  graphic (GstTracerFactory)
sharktracers:  cpuusage (GstTracerFactory)
coretracers:  latency (GstTracerFactory)
coretracers:  log (GstTracerFactory)
coretracers:  rusage (GstTracerFactory)
coretracers:  stats (GstTracerFactory)
coretracers:  leaks (GstTracerFactory)

To use it, enable the tracers with the GST_DEBUG variable and then select the desired tracers using GST_TRACERS. Below is an example of how to use gst-shark to measure interlatency in the deepstream-test5-app:

GST_DEBUG=*TRACE*:9 GST_TRACERS="interlatency" deepstream-test5-app ...

4

In DeepStream videos are inferenced in “batch” but not frame by frame. What is your “inference time” refer to? Are the “preprocessing” and “postprocessing” included?

5

not include

6

How to print this message to file?

7

Only batch inferencing time can be got. In gst-nvinfer, there is no log with inferencing time. There are two ways to get the time.

  1. Add latency calculation code in NvDsInferContextImpl::queueInputBatch() function in /opt/nvidia/deepstream/deepstream/sources/libs/nvdsinfer/nvdsinfer_backend.cpp. Rebuild libnvds_infer.so and run your app with the new libnvds_infer.so.
  2. Use Nsight tool to profiling the app. DeepStream SDK FAQ - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums
8

1、There is a topic about the first .
I modify the NvDsInferContextImpl::queueInputBatch() in the nvdsinfer_context_impl.cpp

批注 2024-09-05 155524
批注 2024-09-05 1555241202×472 18.8 KB

and rebuild the so and copy the lib directory

批注 2024-09-05 155902
批注 2024-09-05 155902791×95 3.58 KB

But I don’t find the print

The second in this topic I don’t find any useful infomation

9

The DeepStream directory is /opt/nvidia/deepstream/deepstream/lib

10

With these exports you can enable the output files and set the location:

export GST_SHARK_CTF_DISABLE=TRUE
export GST_SHARK_LOCATION=/tmp/profile
11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.