How to run PeopleNet in FP16 mode in DeepStream 7.0?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

DeepStream Version - 7.0
Docker Image - nvcr.io/nvidia/deepstream:7.0-gc-triton-devel
GPU - NVIDIA A100-SXM4-40GB
NVIDIA GPU Driver - 550.90.07
CUDA Version -12.4
PyTorch Version - 2.4.0

I need to run PeopleNet on my DS-7.0 setup in FP16 mode. Following are some questions I have -

  1. Which model do I need to choose from here - PeopleNet | NVIDIA NGC?
  2. All the models here with ONNX files are quantized to INT8.
  3. If I pick a model with ETLT file, how do I add that in nvinfer config?

Thanks!

3

For DeepStream7.0, only ONNX model is supported.
There is peoplenet model download script deepstream_reference_apps/deepstream_app_tao_configs/peoplenet_test.sh at DS_7.0 · NVIDIA-AI-IOT/deepstream_reference_apps for DeepSTream 7.0. The model nvinfer configuration deepstream_reference_apps/deepstream_app_tao_configs/nvinfer/config_infer_primary_peoplenet.txt at DS_7.0 · NVIDIA-AI-IOT/deepstream_reference_apps

You just need to change the “network-mode=2” in deepstream_reference_apps/deepstream_app_tao_configs/nvinfer/config_infer_primary_peoplenet.txt at DS_7.0 · NVIDIA-AI-IOT/deepstream_reference_apps, then you can run peoplenet FP16 with DeepStream.

1 Like
4

Thanks for pointing out to the correct files!

So, even though the ONNX models have “_int8” suffix, they can still run in FP16 or FP32 mode during nvinfer?

5

It has nothing to do with the suffix of the name. TensorRT will handle it if you set the correct parameter to the interfaces.

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.