I converted parseq ocr model from pytorch to onnx and tested it on onnx model and every thing is ok, but when I convert onnx to fp32 or fp16 tensorrt engine, output of the model is very different from onnx model.
model repo : GitHub - baudm/parseq: Scene Text Recognition with Permuted Autoregressive Sequence Models (ECCV 2022)
TensorRT Version : TensorRT-8.6.1.6GPU Type : RTX 3060Nvidia Driver Version : 531.79CUDA Version : cuda-12.0CUDNN Version : cudnn-8.9.1.23_cuda12Operating System + Version : win 10Python Version : 3.8Onnx opset : 14
onnx model: test.onnx - Google Drive
trtexec.exe --onnx=parseq/test.onnx --workspace=10000 --saveEngine=parseq/test_fp32.trs --verbosetest_fp32.trt - Google Drive test_fp32_log.txt - Google Drive
trtexec.exe --onnx=parseq/test.onnx --fp16 --workspace=10000 --saveEngine=parseq/test_fp16.trs --verbosetest_fp16.trt - Google Drive test_fp16_log.txt - Google Drive
I wrote a sample code to compare similarity of onnx and trt inference result. when I use real data, mean of similarity is 0.3 and when I use random number it is near 0.85
sample code:
sample real data:
I have same problem with VitStr based on timm vision transformer
VitStr:
Vision transformer:
""" Vision Transformer (ViT) in PyTorch
A PyTorch implement of Vision Transformers as described in:
'An Image Is Worth 16 x 16 Words: Transformers for Image Recognition at Scale'
- https://arxiv.org/abs/2010.11929
`How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers`
- https://arxiv.org/abs/2106.10270
`FlexiViT: One Model for All Patch Sizes`
- https://arxiv.org/abs/2212.08013
The official jax code is released and available at
* https://github.com/google-research/vision_transformer
* https://github.com/google-research/big_vision
Acknowledgments:
* The paper authors for releasing code and weights, thanks!
* I fixed my class token impl based on Phil Wang's https://github.com/lucidrains/vit-pytorch
show original
It seems that the problem is vision transformer.
Hi,
validating your model with the below snippet
import sys
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
It is a bug in 8.6.1 version of tensorrt. and it will be fixed in the next release.
opened 04:31PM - 15 Jul 23 UTC
triaged
## Description
I converted parseq ocr model from pytorch to onnx and tested it … on onnx model and every thing is ok, but when I convert onnx to fp32 or fp16 tensorrt engine, output of the model is very different from onnx model.
I use onnsim to simplify onnx. if i dont use onnxsim all results are nan.
model repo : https://github.com/baudm/parseq
<!--
A clear and concise description of the issue.
For example: I tried to run model ABC on GPU, but it fails with the error below (share a 2-3 line error log).
-->
## Environment
**TensorRT Version**: TensorRT-8.6.1.6
**NVIDIA GPU**: RTX 3060
**NVIDIA Driver Version**: 531.79
**CUDA Version**: cuda-12.0
**CUDNN Version**:cudnn-8.9.1.23_cuda12
Operating System: Win 10
Python Version: 3.8
PyTorch Version: 1.13
Onnx opset : 14
## Relevant Files
onnx model: https://drive.google.com/file/d/1CRXsD8Zk5Mo50JYCZytrAtBbFm2oOqvc/view?usp=sharing
trtexec.exe --onnx=parseq/test.onnx --workspace=10000 --saveEngine=parseq/test_fp32.trs --verbose
trt engine fp32: https://drive.google.com/file/d/17eecl4QrRrE1BiLqDE8HJT0wZCVm3BkB/view?usp=sharing
trt engine fp32 log: https://drive.google.com/file/d/1i9KkbKainaNIz5QQvolmScIu53DzFHHv/view?usp=sharing
trtexec.exe --onnx=parseq/test.onnx --fp16 --workspace=10000 --saveEngine=parseq/test_fp16.trs --verbose
trt engine fp16: https://drive.google.com/file/d/1CIzRZ-71a2hXZWnMNtWn7k2tuM3Pi6K_/view?usp=sharing
trt engine fp16 log: https://drive.google.com/file/d/15LOBtarM6RZiiyZaz66qt6Z8nu67JyrN/view?usp=sharing
## Steps To Reproduce
I wrote a sample code to compare similarity of onnx and trt inference result. when I use real data, mean of similarity is 0.3 and when I use random number it is near 0.85
sample code:
https://drive.google.com/file/d/1dLo9iD3ZUPVuvU6LNFnwQSCjcLDTiKQr/view?usp=sharing
sample real data:
https://drive.google.com/file/d/1VtQgOYw5ZYQSZmUOGyJ7xPKElC7caFMl/view?usp=sharing
system
July 31, 2023, 4:55am
6
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.