NVIDIA Developer Forums

Why is' int8 'not as fast as' fp16'

AI & Data Science Deep Learning (Training & Inference) TensorRT

spolisetty February 1, 2021, 5:13am 3

Could you please provide us more info and reproduce scripts/model, so we can assist better.

Thank you.

Topic		Replies	Views	Activity
YoloV4 slower in INT8 than FP16 TensorRT	5	1545	June 5, 2021
The inference speed of yolov5 tensorrt has little difference between int8 and fp16 TensorRT tensorrt , cuda	1	1555	September 8, 2022
TRT Engin in INT8 is much slower than FP16 TensorRT	4	1992	November 11, 2021
Same inference speed for INT8 and FP16 TensorRT	10	5931	October 12, 2021
Little performance difference between int8 and fp16 on RTX2080 TensorRT	4	2605	July 5, 2021
Int8 is not faster than fp16 on xavier Jetson AGX Xavier tensorrt	5	779	October 18, 2021
QAT int8 TRT engine slower than fp16 TensorRT tensorrt , pytorch , python , onnx	3	2339	January 6, 2022
TensorRT int8 slower than FP16 due to reformat layer TensorRT tensorrt , cudnn	0	115	October 11, 2024
Yolov3 int8 on tensorrt 7.1.0.16 Jetson Xavier NX tensorrt	4	868	October 18, 2021
[Hugging Face transformer models + pytorch_quantization] PTQ quantization int8 is slower than fp16 TensorRT tensorrt , python , onnx , natural-language-processing-nlp	4	3065	January 6, 2022