Onnx model to TRT conversion error

aaryan · June 8, 2020, 11:47am

Description

I’m trying to convert my onnx model to TensorRT engine model but getting error for one unsupported layer.

[06/06/2020-14:42:34] [W] [TRT] onnx2trt_utils.cpp:217: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[06/06/2020-14:42:34] [W] [TRT] onnx2trt_utils.cpp:243: One or more weights outside the range of INT32 was clamped
[06/06/2020-14:42:34] [W] [TRT] ModelImporter.cpp:135: No importer registered for op: ReverseSequence. Attempting to import as plugin.
[06/06/2020-14:42:34] [I] [TRT] builtin_op_importers.cpp:3556: Searching for plugin: ReverseSequence, plugin_version: 001, plugin_namespace:
[06/06/2020-14:42:34] [E] [TRT] INVALID_ARGUMENT: getPluginCreator could not find plugin ReverseSequence version 001
ERROR: builtin_op_importers.cpp:3558 In function importFallbackPluginImporter:
[8] Assertion failed: creator && “Plugin not found”
[06/06/2020-14:42:34] [E] Failed to parse onnx file
[06/06/2020-14:42:34] [E] Parsing model failed
[06/06/2020-14:42:34] [E] Engine creation failed
[06/06/2020-14:42:34] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # ./trtexec --onnx=/home/Downloads/model.onnx --shapes=input_1:32x3x244x244

Relevant Files

Here is the onnx model file attached:

Can you tell me what’s wrong in this model? What I can do to convert this into TensorRT engine ?

SunilJB · June 8, 2020, 10:33am

ReverseSequence op is currently not supported in TRT. Please refer to below link for more details.

github.com

onnx/onnx-tensorrt/blob/master/operators.md

# Supported ONNX Operators

TensorRT 7.0 supports operators up to Opset 11. Latest information of ONNX operators can be found [here](https://github.com/onnx/onnx/blob/master/docs/Operators.md)

TensorRT supports the following ONNX data types: FLOAT32, FLOAT16, INT8, and BOOL

\*There is limited support for INT32 and INT64 types. TensorRT will attempt to cast down INT64 to INT32 where possible. If not possible, TensorRT will throw an error. See the [TensorRT layer support matrix](https://docs.nvidia.com/deeplearning/sdk/tensorrt-support-matrix/index.html#layers-precision-matrix) for more information on data type support.

## Operator Support Matrix

| Operator              | Supported? | Restrictions                                                                                                                           |
|-----------------------|------------|----------------------------------------------------------------------------------------------------------------------------------------|
| Abs                   | Y          |
| Acos                  | Y          |
| Acosh                 | Y          |
| Add                   | Y          |
| And                   | Y          |
| ArgMax                | Y          |
| ArgMin                | Y          |
| Asin                  | Y          |

This file has been truncated. show original

You need to create a custom plugin for any unsupported layer.

Thanks

aaryan · June 8, 2020, 11:46am

I converted this onnx model to quantized onnx model using quantization tool.
Got this result:

[06/08/2020-15:04:42] [W] [TRT] onnx2trt_utils.cpp:217: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[06/08/2020-15:04:42] [W] [TRT] onnx2trt_utils.cpp:243: One or more weights outside the range of INT32 was clamped
[06/08/2020-15:04:42] [E] [TRT] onnx2trt_utils.cpp:391: Found unsupported datatype (2) when importing initializer: conv2d/Conv2D/ReadVariableOp:0_quantized
ERROR: ModelImporter.cpp:92 In function parseGraph:
[8] Assertion failed: convertOnnxWeights(initializer, &weights, ctx)
[06/08/2020-15:04:42] [E] Failed to parse onnx file
[06/08/2020-15:04:42] [E] Parsing model failed
[06/08/2020-15:04:42] [E] Engine creation failed
[06/08/2020-15:04:42] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # ./trtexec --onnx=/home/Downloads/quantized_model.onnx --shapes=input_1:32x3x244x244

Any inputs to this? Is this the right approach??

aaryan · June 8, 2020, 11:19am

Also, Can you help how can I write a custom plugin for this? I got through some tutorials on this they said to replace that operation with some other. But I don’t know which operation can become alternative for this reverse_sequence. Can you guide me in this?

SunilJB · June 8, 2020, 12:30pm

Please refer to below link:

Thanks