I convert an unet model into fp16 and int8 using tensorrt 5.1.6. When I measure the latency using trtexec they’re both around 16ms on xavier, could anyone help me to about the reason? The input is 512, 512, 3. Here’s the nvprof output: **int8:** Type Time(%) Time Calls Avg Mi…

Int8 is not faster than fp16 on xavier

AastaLLL August 28, 2020, 7:29am 6

Hi,

Sorry for the late.

Would you mind to attach the complete TensorRT log here?
We can check if you have run the INT8 mode correctly through the output log.

Please remember to add --verbose argument when collecting the log.
Thanks.

Topic		Replies	Views
Same inference speed for INT8 and FP16 TensorRT	10	5881	October 12, 2021
TRT Engin in INT8 is much slower than FP16 TensorRT	4	1974	November 11, 2021
Inference Speed Jetson Xavier NX pytorch	6	904	April 12, 2023
Why is' int8 'not as fast as' fp16' TensorRT tensorrt	1	565	February 1, 2021
Little performance difference between int8 and fp16 on RTX2080 TensorRT	4	2584	July 5, 2021
How to verify if QAT TRT engine is indeed INT8 on Xavier Jetson AGX Xavier tensorrt	16	604	October 5, 2022
Decreased performance from FP16 to INT8 in TF-TRT on Jetson Xavier General	12	2700	October 12, 2021
QAT int8 TRT engine slower than fp16 TensorRT tensorrt , pytorch , python , onnx	3	2310	January 6, 2022
[Hugging Face transformer models + pytorch_quantization] PTQ quantization int8 is slower than fp16 TensorRT tensorrt , python , onnx , natural-language-processing-nlp	4	3046	January 6, 2022
The same performance with int8 and fp16 DeepStream SDK	10	1241	October 12, 2021