Tensorrt quantization bad performance

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

hello I use int8 calibrator to quantize my yolov5 model,
but get an extrem bad model performance.

mAP reduce to 1 from 50

I’m sure that my preprocess and calibration code are all right.

so Is there any tips to raise my model accuracy?

5

Hi @lc5104hit,
Can you please check the below link, as it may answer your query.
https://on-demand.gputechconf.com/gtc/2017/presentation/s7310-8-bit-inference-with-tensorrt.pdf
If the issue persist, request you to share the calibration cache.

Thanks!

6

hello,
I try using relu instead leakyrelu,
the result imporves.

So why this happens?

7

Maybe modify Leakyrelu to Relu works.