Which type/format of data is needed for int8 calibration

AI & Data Science Deep Learning (Training & Inference) TensorRT
, ,
1

Hi
Thanks for providing such powerful Tensorrt. In order to maximize the efficiency, we are using the dla with standalone mode, and using Int8 as input/output data type. Also we set the flag to allow all formats of input/output.
But we don’t know which type/format of input data we need to prepare.

1> When all of input is Int8, the input data in calibrator will be fp32 or fp16 or Int8?
2> Will trt reformat the input automatically?

2

Hi, Please refer to the below links to perform inference in INT8

Thanks!

4

@NVES Thanks for your reply.

By referencing the code A
and B, it looks like the input is always of FLOAT & LINEAR. But is this based on the premise of FLOAT & LINEAR network inputs or works on any input type & format?

5

Yes, It need to be float32. Please refer to the following doc for more details.

Thank you.

1 Like
6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

7

Also check out the DLA github page for samples and resources or to report issues: Recipes and tools for running deep learning workloads on NVIDIA DLA cores for inference applications.

We have a FAQ page that addresses some common questions that we see developers run into: Deep-Learning-Accelerator-SW/FAQ

1 Like