Description
Hello,
I am trying to convert an .onnx model to a fp16 .engine using the below command:
trtexec --onnx=-beit-base-patch16-224.onnx --fp16 --saveEngine=model.engine --minShapes=\'pixel_values\':1x3x224x224 --optShapes=\'pixel_values\':8x3x224x224 --maxShapes=\'pixel_values\':8x3x224x224 --precisionConstraints=obey --layerPrecisions=/beit/embeddings/patch_embeddings/projection/Conv:fp32
The engine is build as expected when using fp32, however when the --fp16 flag is set, the below warnings appear and cause model outputs to be wrong:
Even when setting the --precisionConstraints=obey
and layerPrecisions=/beit/embeddings/patch_embeddings/projection/Conv:fp32
or layerPrecisions=/beit/embeddings/patch_embeddings/projection/Conv.weight:fp32
the weights are still cast to fp16 causing a bad model conversion.
Am I doing something wrong or is there any way to fix the above issue?
ONNX model link: Download | file.io
(Using the Deepstream 6.1.1 docker image)
Thanks!
Environment
TensorRT Version:
8.4.1
GPU Type:
RTX 3050TI
Nvidia Driver Version:
522.30
CUDA Version:
11.8
CUDNN Version:
Operating System + Version:
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):
Relevant Files
Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)
Steps To Reproduce
Please include:
- Exact steps/commands to build your repro
- Exact steps/commands to run your repro
- Full traceback of errors encountered