Trying to the C++ quickstart example running. following the instructions
here, the first step is to get a working tensorRT engine, from an onnx file. Converting the mode with trtexec fails.
I’m using the trtexec created with the build of the repo at
GitHub - NVIDIA/TensorRT: TensorRT is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators.
TensorRT Version: 8.2.4-1+cuda11.4
GPU Type: GTX3090
Nvidia Driver Version: 11.6
CUDA Version: 11.4
CUDNN Version: 8.4
Operating System + Version: Ubuntu 20.04
Baremetal or Container (if container which image + tag):
Steps To Reproduce
cloned the repo and built
git clone -b master https://github.com/nvidia/TensorRT TensorRT
git submodule update –init --recursive
docker run --rm -it --gpus all -p 8888:8888 -v `pwd`:/workspace -w /workspace/SemanticSegmentation nvcr.io/nvidia/pytorch:20.12-py3 bash
close the docker
convert the onnx model:
trtexec --onnx=fcn-resnet101.onnx --fp16 --workspace=64 --minShapes=input:1x3x256x256 --optShapes=input:1x3x1026x1282 --maxShapes=input:1x3x1440x2560 --buildOnly --saveEngine=fcn-resnet101.engine
This is the error:
[04/29/2022-15:21:52] [E] Error: [optimizer.cpp::computeCosts::2011]
Error Code 10: Internal Error (Could not find any implementation for node ArgMax_260.)
[04/29/2022-15:21:52] [E] Error: [builder.cpp::buildSerializedNetwork::609] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
[04/29/2022-15:21:52] [E] Engine could not be created from network
[04/29/2022-15:21:52] [E] Building engine failed
[04/29/2022-15:21:52] [E] Failed to create engine from model.
[04/29/2022-15:21:52] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8200] # trtexec --onnx=fcn-resnet101.onnx --fp16 --workspace=64 --minShapes=input:1x3x256x256 --optShapes=input:1x3x1026x1282 --maxShapes=input:1x3x1440x2560 --buildOnly --saveEngine=fcn-resnet101.engine
Segmentation fault (core dumped)
We recommend you to check the supported features from the below link.
These support matrices provide a look into the supported platforms, features, and hardware capabilities of the NVIDIA TensorRT 8.4.0 Early Access (EA) APIs, parsers, and layers.
You can refer below link for all the supported operators list.
For unsupported operators, you need to create a custom plugin to support the operation
This file has been truncated.
<!--- SPDX-License-Identifier: Apache-2.0 -->
# Supported ONNX Operators
TensorRT 8.2 supports operators up to Opset 13. Latest information of ONNX operators can be found [here](https://github.com/onnx/onnx/blob/main/docs/Operators.md)
TensorRT supports the following ONNX data types: DOUBLE, FLOAT32, FLOAT16, INT8, and BOOL
> Note: There is limited support for INT32, INT64, and DOUBLE types. TensorRT will attempt to cast down INT64 to INT32 and DOUBLE down to FLOAT, clamping values to `+-INT_MAX` or `+-FLT_MAX` if necessary.
See below for the support matrix of ONNX operators in ONNX-TensorRT.
## Operator Support Matrix
| Operator | Supported | Supported Types | Restrictions |
| Abs | Y | FP32, FP16, INT32 |
| Acos | Y | FP32, FP16 |
| Acosh | Y | FP32, FP16 |
| Add | Y | FP32, FP16, INT32 |
I have no idea what you mean by that.
I am trying to use an NVIDIA product out-of-the-box and it doesn’t work. I guess the NVIDIA developer is the one who needs to look at the Supported ONNX Operators not me.
If the model
fcn-resnet101.onnx, is not supported by tensorrt, remove it from the quickstart example and provide something that works
Please provide me instructions on how to get the tensorrt quickstart c++ example working.
We recommend you to please try on the latest TensorRT version 8.4 EA.
If you still face this issue please file this issue on Issues · NVIDIA/TensorRT · GitHub to get better help.
This was bad advise. On the one hand the problem persists (I already posted the problem at the github, with no reply yet…) while on the other hand, all the tensorrt engines created with tao-converter STOPPED WORKING, and there is no version of tao-converter compatible with tensorRT 8.4…
Could you please share with us the ONNX model you’re using to try from our end for better debugging.
@spolisetty Onnx example given in Nvidia’s offcial tutorial page. You can get the onnx model by running the export.py by yourself ( TensorRT/export.py at main · NVIDIA/TensorRT · GitHub)
To fix this problem just add the workspace size with --workspace=4096 option. This because the workspace is not enough for tensorrt 8.X.
Here list a example of the changed cmd:
trtexec --onnx=fcn-resnet101.onnx --fp16 --workspace=4096 --minShapes=input:1x3x256x256 --optShapes=input:1x3x1026x1282 --maxShapes=input:1x3x1440x2560 --buildOnly --saveEngine=fcn-resnet101.engine
Thanks to 's comments jasxu-nvidia
reference from Quick Start, Unable to prepare engine · Issue #1965 · NVIDIA/TensorRT · GitHub
I can confirm that the larger workspace allows the command to execute with no errors.
I did not try the converted model in the quick start code, since I have already implemented my own.