ONNX Model and Tensorrt Engine gives different output for parseq model

keivan.moazami · July 17, 2023, 4:54am

It is a bug in 8.6.1 version of tensorrt. and it will be fixed in the next release.

TensorRt result is different from onnx model in parseq model

opened 04:31PM - 15 Jul 23 UTC

triaged

## Description I converted parseq ocr model from pytorch to onnx and tested it …on onnx model and every thing is ok, but when I convert onnx to fp32 or fp16 tensorrt engine, output of the model is very different from onnx model. I use onnsim to simplify onnx. if i dont use onnxsim all results are nan. model repo : https://github.com/baudm/parseq  ## Environment **TensorRT Version**: TensorRT-8.6.1.6 **NVIDIA GPU**: RTX 3060 **NVIDIA Driver Version**: 531.79 **CUDA Version**: cuda-12.0 **CUDNN Version**:cudnn-8.9.1.23_cuda12 Operating System: Win 10 Python Version: 3.8 PyTorch Version: 1.13 Onnx opset : 14 ## Relevant Files onnx model: https://drive.google.com/file/d/1CRXsD8Zk5Mo50JYCZytrAtBbFm2oOqvc/view?usp=sharing trtexec.exe --onnx=parseq/test.onnx --workspace=10000 --saveEngine=parseq/test_fp32.trs --verbose trt engine fp32: https://drive.google.com/file/d/17eecl4QrRrE1BiLqDE8HJT0wZCVm3BkB/view?usp=sharing trt engine fp32 log: https://drive.google.com/file/d/1i9KkbKainaNIz5QQvolmScIu53DzFHHv/view?usp=sharing trtexec.exe --onnx=parseq/test.onnx --fp16 --workspace=10000 --saveEngine=parseq/test_fp16.trs --verbose trt engine fp16: https://drive.google.com/file/d/1CIzRZ-71a2hXZWnMNtWn7k2tuM3Pi6K_/view?usp=sharing trt engine fp16 log: https://drive.google.com/file/d/15LOBtarM6RZiiyZaz66qt6Z8nu67JyrN/view?usp=sharing ## Steps To Reproduce I wrote a sample code to compare similarity of onnx and trt inference result. when I use real data, mean of similarity is 0.3 and when I use random number it is near 0.85 sample code: https://drive.google.com/file/d/1dLo9iD3ZUPVuvU6LNFnwQSCjcLDTiKQr/view?usp=sharing sample real data: https://drive.google.com/file/d/1VtQgOYw5ZYQSZmUOGyJ7xPKElC7caFMl/view?usp=sharing

Topic		Replies	Views
TensorRT gives diffent results than ONNX and Pytorch TensorRT	8	1569	September 28, 2023
ONNX Model and Tensorrt Engine gives different output TensorRT tensorrt , onnx	13	5402	June 29, 2022
YOLOv4 TensorRT inference results wayy off, but onnxruntime is not TensorRT tensorrt	7	947	June 7, 2022
ONNX Model and Tensorrt Engine gives different output TensorRT tensorrt , onnx	4	731	March 21, 2023
Tensorrt 8.6 GA : C++ Inference gives diffrence results compared to onnx \|\| pt model python inference TensorRT	3	635	September 20, 2023
Two machines with very similar SW stack but different GPUs generate different folded model using the Polygraphy tool on the same model onnx input TensorRT	7	813	June 22, 2022
Inference result gets worse when converting pytorch model to TensorRT model TensorRT pytorch	6	1142	January 19, 2022
Same version TensorRT with two methods to convert onnx model，One used trtexec[FAILED] , the other used python[Success] TensorRT	5	757	October 3, 2023
Keras CRNN model conversion to tensorrt engine error TensorRT tensorrt , tensorflow , onnx	3	958	April 8, 2022
ONNX to TRT Engine conversion Error TensorRT tensorrt	8	3717	May 25, 2022

ONNX Model and Tensorrt Engine gives different output for parseq model

Related topics