INT8 with deepstream

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 6.0
• JetPack Version (valid for Jetson only)
• TensorRT Version 8.0.1
• NVIDIA GPU Driver Version (valid for GPU only) 495.44
• Issue Type( questions, new requirements, bugs) Question

Hi, i am using FP32 Centerface FaceDetector with Deepstream. I want to use it’s INT8 version.

I have 3 doubts-

  1. Does INT8_calib file is mandatory for that? I can see some TRT docs where it says not rewuired.

  2. Can this document “TensorRT/samples/opensource/sampleINT8API at release/6.0 · NVIDIA/TensorRT · GitHub” help me to generate Centerface INT8 model?

  3. What are the steps to produce INT8_calib file? i have 1000 face images but what is next?

Thanks.

  1. Does INT8_calib file is mandatory for that? I can see some TRT docs where it says not rewuired. ==> INT8_calib file is mandatory for INT8 inference
    1. Can this document “TensorRT/samples/opensource/sampleINT8API at release/6.0 · NVIDIA/TensorRT · GitHub” help me to generate Centerface INT8 model? ==> as the readme of this sample mentioned : This sample, sampleINT8API, performs INT8 inference with the dynamic range the user provided per activation tensor instead of using the INT8 calibrator. I recommend to refer to TensorRT/samples/opensource/sampleINT8 at release/6.0 · NVIDIA/TensorRT · GitHub for INT8 calibration.
  2. What are the steps to produce INT8_calib file? i have 1000 face images but what is next? ===> you can refer to above sampleINT8 sample and TRT doc - Developer Guide :: NVIDIA Deep Learning TensorRT Documentation ( 7.1. Introduction to Quantization and started with PTQ calibration solution - 7.3. Post-Training Quantization using Calibration ).

BTW, if you want to save INT8 calibration, running it with FP16 precision could also get much better perf than FP32 since FP16 and INT8 can use Tneosr Core accelerators, but certainly, INT8 can get the best perf.

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.