TensorRT, result error in fp16

641263629 · October 19, 2021, 2:25am

Description

TensorRT result error, result is related to the previous input in FP16 model. float32 model is OK.

Environment

TensorRT Version: TensorRT-8.2.0.6
GPU Type: Tesla T4
Nvidia Driver Version: 470.57.02
CUDA Version: 11.4
CUDNN Version: cudnn-11.4-linux-x64-v8.2.4.15.tgz
Operating System + Version: ubuntu 18.04
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

Exact steps/commands to build your repro
Exact steps/commands to run your repro
Full traceback of errors encountered

NVES · October 19, 2021, 6:09am

Hi ,
We recommend you to check the supported features from the below link.

You can refer below link for all the supported operators list.
For unsupported operators, you need to create a custom plugin to support the operation

github.com

onnx/onnx-tensorrt/blob/main/docs/operators.md

<!--- SPDX-License-Identifier: Apache-2.0 -->

# Supported ONNX Operators

TensorRT 8.4 supports operators up to Opset 17. Latest information of ONNX operators can be found [here](https://github.com/onnx/onnx/blob/master/docs/Operators.md)

TensorRT supports the following ONNX data types: DOUBLE, FLOAT32, FLOAT16, INT8, and BOOL

> Note: There is limited support for INT32, INT64, and DOUBLE types. TensorRT will attempt to cast down INT64 to INT32 and DOUBLE down to FLOAT, clamping values to `+-INT_MAX` or `+-FLT_MAX` if necessary.

See below for the support matrix of ONNX operators in ONNX-TensorRT.

## Operator Support Matrix

| Operator                  | Supported  | Supported Types | Restrictions                                                                                                           |
|---------------------------|------------|-----------------|------------------------------------------------------------------------------------------------------------------------|
| Abs                       | Y          | FP32, FP16, INT32 |
| Acos                      | Y          | FP32, FP16 |
| Acosh                     | Y          | FP32, FP16 |
| Add                       | Y          | FP32, FP16, INT32 |

This file has been truncated. show original

Thanks!

Topic		Replies	Views
Fp16 engine（generate on windows with TRT861） get stuck on linux (TRT 861) TensorRT	1	382	September 13, 2023
nVidia release versions compatibility TensorRT	4	877	September 20, 2023
nvonnxparser::IParse::parse() fail,and trt report paramenter check fail TensorRT tensorrt	7	1195	July 12, 2021
Does TensorRT support GeForce and Quadro boards TensorRT	1	582	April 6, 2023
Comaptibility Of TensorRT with Windows 11 Professional TensorRT	2	2264	December 6, 2021
Does tensorrt suport op like "expand_as" or "tile" for dynamic shape input? TensorRT	2	873	November 11, 2021
Torch_tensorrt compile doesn't support pretrained torchvision Mask_RCNN model TensorRT tensorrt , pytorch	3	663	August 12, 2022
How to reduce differences in inference output across gpus when using fp16? TensorRT cudnn	0	337	December 22, 2023
TensorRT TensorRT	1	729	September 4, 2021
Engine upgrade TensorRT	2	340	March 27, 2023

TensorRT, result error in fp16

Description

Environment

Relevant Files

Steps To Reproduce

Related topics