Back to Back detector performance

Accelerated Computing Intelligent Video Analytics DeepStream SDK
#1

Hi

Is there a way to get higher performance of running back-to-back-detectors with Deepstream5.0 on NX?

We tested back-to-back-detectors with DS5 on NX and got not smooth inference result. The speed might be lower than 15 FPS.
Also the GPU usage was around 99%. (see attached.)
Already set nvpmodel to 2 and enabled jetson-clock.
Moreover, modifying interval did not see any change.

Is it normal?

jtop_ds5_ls_p
jtop_ds5_ls_p1024×640 290 KB

Thank you for any advice,

#3

Hi,

Please try the following command to maximize the performance:

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

If the above cannot meet your requirement, please run the pipeline with FP16 or INT8 mode.
https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html#gst-nvinfer-file-configuration-specifications

[property]
...
network-mode=1  # 0: FP32 1: INT8 2: FP16

Thanks.

#4

Hi AastaLLL,

Thank you for your prompt support.

May I check again?
On NX , should we set nvpmodel to 0 or 2 for max performance?

As network-mode was set to 1 and we got “INT calibration file not specified, TRYING FP16” ( see attached.)
However, using FP 16 seems better than FP32.
Just like to know, is it possible to use INT 8?

b2b_int8-err
b2b_int8-err1366×768 299 KB

Thank you,

#5

Hi,

1.
You can choose nvpmodel based on your use case.

Mode 0 has a higher clock rate, but only 2 CPU enabled.
Mode 2 enables all the CPUs, but the clock rate is lower.

So if your task is not CPU-intensive (parallel to more than 2 CPU), mode 0 should give you a better performance.

2.
INT8 requires GPU architecture > 7.x, and Xavier NX does meet the requirement.
The error indicates you need a calibration table to map a floating number into an integer.

Based on the configure below, it should find a calibration table named cal_trt.bin.
Could you check if the file exists in your environment first?

Thanks.