How can we know we have convert the onnx to int8trt rather than Float32?

530869411 · June 10, 2021, 8:27am

@spolisetty thank you very much for help me so many times,Now i am trying to do tensorrt QAT,Could you tell me ,where i can find some example?

spolisetty · June 10, 2021, 8:33am

Hope following will help you,

Thank you.

530869411 · June 10, 2021, 8:51am

@spolisetty Does tensor support RT ?or tensorRT only have postQT

spolisetty · June 14, 2021, 10:39am

Looks like we have same query on new thread. Please follow up here

Thank you.

Topic		Replies	Views
How to verify if QAT TRT engine is indeed INT8 on Xavier Jetson AGX Xavier tensorrt	15	819	September 22, 2022
Convert int8-onnx model to trt engine? TensorRT onnx	6	1255	April 29, 2023
[Hugging Face transformer models + pytorch_quantization] PTQ quantization int8 is slower than fp16 TensorRT tensorrt , python , onnx , natural-language-processing-nlp	4	3209	January 6, 2022
QAT int8 TRT engine slower than fp16 TensorRT tensorrt , pytorch , python , onnx	3	2565	January 6, 2022
TensorRT generated QAT engine, why the engine is bigger than pretrained fp16 engine? TensorRT	3	1433	January 4, 2022
Trtexec cannot convert QAT onnx model to trt model Jetson AGX Xavier tensorrt	6	832	August 9, 2022
Post quantization aware training is slower than fp16 and post quantization TensorRT	12	2960	September 25, 2024
Inference Speed Jetson Xavier NX pytorch	5	1116	March 30, 2023
Onnx -> tensorrt fp32 conversion performance degradation different outputs TensorRT tensorrt , pytorch , onnx	4	2319	November 29, 2022
TensorRT the inference is slow for the QAT model comparing to the PTQ case Jetson AGX Xavier tensorrt , nvbugs	18	1893	January 16, 2023