Error (Could not find any implementation for node ArgMax_260.)

david9xqqb · April 29, 2022, 1:21pm

Description

Trying to the C++ quickstart example running. following the instructions here, the first step is to get a working tensorRT engine, from an onnx file. Converting the mode with trtexec fails.

I’m using the trtexec created with the build of the repo at GitHub - NVIDIA/TensorRT: NVIDIA® TensorRT™, an SDK for high-performance deep learning inference, includes a deep learning inference optimizer and runtime that delivers low latency and high throughput for inference applications.

Environment

TensorRT Version: 8.2.4-1+cuda11.4
GPU Type: GTX3090
Nvidia Driver Version: 11.6
CUDA Version: 11.4
CUDNN Version: 8.4
Operating System + Version: Ubuntu 20.04
Docker nvcr.io/nvidia/pytorch:20.12-py3

Baremetal or Container (if container which image + tag):

Steps To Reproduce

cloned the repo and built

git clone -b master https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update –init --recursive
cd quickstart
docker run --rm -it --gpus all -p 8888:8888 -v `pwd`:/workspace -w /workspace/SemanticSegmentation nvcr.io/nvidia/pytorch:20.12-py3 bash
python export.py

close the docker

convert the onnx model:

trtexec --onnx=fcn-resnet101.onnx --fp16 --workspace=64 --minShapes=input:1x3x256x256 --optShapes=input:1x3x1026x1282 --maxShapes=input:1x3x1440x2560 --buildOnly --saveEngine=fcn-resnet101.engine

This is the error:

[04/29/2022-15:21:52] [E] Error[10]: [optimizer.cpp::computeCosts::2011] Error Code 10: Internal Error (Could not find any implementation for node ArgMax_260.)
[04/29/2022-15:21:52] [E] Error[2]: [builder.cpp::buildSerializedNetwork::609] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
[04/29/2022-15:21:52] [E] Engine could not be created from network
[04/29/2022-15:21:52] [E] Building engine failed
[04/29/2022-15:21:52] [E] Failed to create engine from model.
[04/29/2022-15:21:52] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8200] # trtexec --onnx=fcn-resnet101.onnx --fp16 --workspace=64 --minShapes=input:1x3x256x256 --optShapes=input:1x3x1026x1282 --maxShapes=input:1x3x1440x2560 --buildOnly --saveEngine=fcn-resnet101.engine
Segmentation fault (core dumped)

NVES · April 29, 2022, 1:37pm

Hi ,
We recommend you to check the supported features from the below link.

You can refer below link for all the supported operators list.
For unsupported operators, you need to create a custom plugin to support the operation

github.com

onnx/onnx-tensorrt/blob/main/docs/operators.md

<!--- SPDX-License-Identifier: Apache-2.0 -->

# Supported ONNX Operators

TensorRT 8.4 supports operators up to Opset 17. Latest information of ONNX operators can be found [here](https://github.com/onnx/onnx/blob/master/docs/Operators.md)

TensorRT supports the following ONNX data types: DOUBLE, FLOAT32, FLOAT16, INT8, and BOOL

> Note: There is limited support for INT32, INT64, and DOUBLE types. TensorRT will attempt to cast down INT64 to INT32 and DOUBLE down to FLOAT, clamping values to `+-INT_MAX` or `+-FLT_MAX` if necessary.

See below for the support matrix of ONNX operators in ONNX-TensorRT.

## Operator Support Matrix

| Operator                  | Supported  | Supported Types | Restrictions                                                                                                           |
|---------------------------|------------|-----------------|------------------------------------------------------------------------------------------------------------------------|
| Abs                       | Y          | FP32, FP16, INT32 |
| Acos                      | Y          | FP32, FP16 |
| Acosh                     | Y          | FP32, FP16 |
| Add                       | Y          | FP32, FP16, INT32 |

This file has been truncated. show original

Thanks!

david9xqqb · April 29, 2022, 2:53pm

I have no idea what you mean by that.

I am trying to use an NVIDIA product out-of-the-box and it doesn’t work. I guess the NVIDIA developer is the one who needs to look at the Supported ONNX Operators not me.

If the model fcn-resnet101.onnx, is not supported by tensorrt, remove it from the quickstart example and provide something that works

Please provide me instructions on how to get the tensorrt quickstart c++ example working.

Many thanks!

spolisetty · May 5, 2022, 2:25pm

Hi,

We recommend you to please try on the latest TensorRT version 8.4 EA.
If you still face this issue please file this issue on Issues · NVIDIA/TensorRT · GitHub to get better help.

Thank you.

david9xqqb · May 6, 2022, 4:07pm

This was bad advise. On the one hand the problem persists (I already posted the problem at the github, with no reply yet…) while on the other hand, all the tensorrt engines created with tao-converter STOPPED WORKING, and there is no version of tao-converter compatible with tensorRT 8.4…

spolisetty · May 6, 2022, 5:18pm

Hi,

Could you please share with us the ONNX model you’re using to try from our end for better debugging.

Thank you.

terpysk · May 10, 2022, 9:51pm

@spolisetty Onnx example given in Nvidia’s offcial tutorial page. You can get the onnx model by running the export.py by yourself (TensorRT/export.py at main · NVIDIA/TensorRT · GitHub)

yinguoliang · May 17, 2022, 3:25am

To fix this problem just add the workspace size with --workspace=4096 option. This because the workspace is not enough for tensorrt 8.X.
Here list a example of the changed cmd:
trtexec --onnx=fcn-resnet101.onnx --fp16 --workspace=4096 --minShapes=input:1x3x256x256 --optShapes=input:1x3x1026x1282 --maxShapes=input:1x3x1440x2560 --buildOnly --saveEngine=fcn-resnet101.engine
Thanks to jasxu-nvidia 's comments
reference from Quick Start, Unable to prepare engine · Issue #1965 · NVIDIA/TensorRT · GitHub

david9xqqb · May 17, 2022, 6:05am

I can confirm that the larger workspace allows the command to execute with no errors.

I did not try the converted model in the quick start code, since I have already implemented my own.

system · May 31, 2022, 6:05am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.