Inference TensorRT randomly returns nan

llwxr1995 · April 23, 2023, 9:03am

Description

I use onnx-tensorrt to inference my model.
In my python code, I am calling TensorRT-8.0 api to load and parse the ONNX model. It rans well. But in my C++ code. It randomly returns nan.

Environment

TensorRT Version: 8.4.1
Nvidia Driver Version: 470.182.03
CUDA Version: 11.4
CUDNN Version: 8.4.0
Operating System + Version: linux18.04

Relevant Files

Steps To Reproduce

AakankshaS · April 24, 2023, 5:07am

Hi,
Can you try running your model with trtexec command, and share the “”–verbose"" log in case if the issue persist

You can refer below link for all the supported operators list, in case any operator is not supported you need to create a custom plugin to support that operation

github.com

onnx/onnx-tensorrt/blob/main/docs/operators.md

<!--- SPDX-License-Identifier: Apache-2.0 -->

# Supported ONNX Operators

TensorRT 8.6 supports operators up to Opset 17. Latest information of ONNX operators can be found [here](https://github.com/onnx/onnx/blob/master/docs/Operators.md)

TensorRT supports the following ONNX data types: DOUBLE, FLOAT32, FLOAT16, INT8, and BOOL

> Note: There is limited support for INT32, INT64, and DOUBLE types. TensorRT will attempt to cast down INT64 to INT32 and DOUBLE down to FLOAT, clamping values to `+-INT_MAX` or `+-FLT_MAX` if necessary.

See below for the support matrix of ONNX operators in ONNX-TensorRT.

## Operator Support Matrix

| Operator                  | Supported  | Supported Types | Restrictions                                                                                                           |
|---------------------------|------------|-----------------|------------------------------------------------------------------------------------------------------------------------|
| Abs                       | Y          | FP32, FP16, INT32 |
| Acos                      | Y          | FP32, FP16 |
| Acosh                     | Y          | FP32, FP16 |
| Add                       | Y          | FP32, FP16, INT32 |

This file has been truncated. show original

Also, request you to share your model and script if not shared already so that we can help you better.

Meanwhile, for some common errors and queries please refer to below link:

Thanks!

llwxr1995 · April 27, 2023, 7:11am

verbose from trtexec.odt (74.4 KB)
Hi,
I Have upload my verbose.

Topic		Replies	Views
TensorRT model giving constant output TensorRT deepstream	4	1352	November 30, 2021
Detectron2: faster inferencing TensorRT	2	1401	April 29, 2022
Tensorrt Execution Provider TensorRT tensorrt , cudnn , onnx	1	787	November 27, 2023
All outputs are nan TensorRT	5	2611	September 23, 2022
TensortRT execute with variable batch size gave incorrect results TensorRT	1	424	November 9, 2021
Keras CRNN model conversion to tensorrt engine error TensorRT tensorrt , tensorflow , onnx	3	956	April 8, 2022
Does TensorRT exploit parallelism in a computational graph during inference? TensorRT	2	526	April 18, 2023
Different engines give different inference results when using the same onnx model and giving the same input TensorRT	4	937	December 31, 2023
TensorRT Inference form a .etlt model on Python TAO Toolkit tensorrt	7	1218	November 16, 2021
Tensorrt 8.6 GA : C++ Inference gives diffrence results compared to onnx \|\| pt model python inference TensorRT	3	629	September 20, 2023

Inference TensorRT randomly returns nan

Description

Environment

Relevant Files

Steps To Reproduce

Related topics