# Failed converting ONNX model to TensorRT model

**URL:** <https://forums.developer.nvidia.com/t/failed-converting-onnx-model-to-tensorrt-model/216981>\
**Category:** TensorRT\
**Created:** [June 8, 2022, 4:58pm UTC](https://forums.developer.nvidia.com/t/failed-converting-onnx-model-to-tensorrt-model/216981 "2022-06-08T16:58:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![user10264](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@user10264](https://forums.developer.nvidia.com/u/user10264)\
**Post date:** [June 8, 2022, 4:58pm UTC](https://forums.developer.nvidia.com/t/failed-converting-onnx-model-to-tensorrt-model/216981/1 "2022-06-08T16:58:55Z")

</div>

## Description

Hi Team,

Looking for some help please.

I have an ONNX model (pytorch). I want to convert the model from ONNX to TensorRT, manually and programmatically.  
I have written some Python code that uses the TensorRT builder API to do the conversion, and i have tested the code on two different machines/environment:

1. Nvidia Tesla K80 (AWS P2.xlarge instance)
2. Nvidia Tesla V100 (AWS P3.xlarge instance)

I can successfully convert the model to TRT on Nvidia V100, programmatically.  
But when i run the same conversion code, it fails on my K80 machine. See error log below.

python dependencies:

```auto
pip freeze | ag nvidia                                                         
nvidia-cublas-cu11==2022.4.8
nvidia-cublas-cu117==11.10.1.25
nvidia-cuda-runtime-cu11==2022.4.25
nvidia-cuda-runtime-cu117==11.7.60
nvidia-cudnn-cu11==2022.5.19
nvidia-cudnn-cu116==8.4.0.27
nvidia-pyindex==1.0.9
nvidia-tensorrt==8.4.0.6

```

Log:

```auto
[06/07/2022-14:22:18] [TRT] [I] [MemUsageChange] Init CUDA: CPU +152, GPU +0, now: CPU 302, GPU 116 (MiB)
[06/07/2022-14:22:20] [TRT] [I] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 322 MiB, GPU 116 MiB
[06/07/2022-14:22:20] [TRT] [I] [MemUsageSnapshot] End constructing builder kernel library: CPU 323 MiB, GPU 116 MiB
[06/07/2022-14:22:20] [TRT] [I] ----------------------------------------------------------------
[06/07/2022-14:22:20] [TRT] [I] Input filename: /tmp/pytest-of-ubuntu/pytest-2/test_mmdetection_model_inferen0/model/saved_model.onnx
[06/07/2022-14:22:20] [TRT] [I] ONNX IR version: 0.0.7
[06/07/2022-14:22:20] [TRT] [I] Opset version: 11
[06/07/2022-14:22:20] [TRT] [I] Producer name: pytorch
[06/07/2022-14:22:20] [TRT] [I] Producer version: 1.10
[06/07/2022-14:22:20] [TRT] [I] Domain:           
[06/07/2022-14:22:20] [TRT] [I] Model version: 0
[06/07/2022-14:22:20] [TRT] [I] Doc string:       
[06/07/2022-14:22:20] [TRT] [I] ----------------------------------------------------------------
[06/07/2022-14:22:20] [TRT] [W] onnx2trt_utils.cpp:365: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[06/07/2022-14:22:20] [TRT] [W] onnx2trt_utils.cpp:391: One or more weights outside the range of INT32 was clamped
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [W] builtin_op_importers.cpp:4777: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [W] builtin_op_importers.cpp:4777: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:20] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:20] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:20] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [W] builtin_op_importers.cpp:4777: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [W] builtin_op_importers.cpp:4777: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [W] builtin_op_importers.cpp:4777: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [W] builtin_op_importers.cpp:4777: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [W] builtin_op_importers.cpp:4777: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [W] builtin_op_importers.cpp:4777: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [W] builtin_op_importers.cpp:4777: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-14:22:21] [TRT] [I] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-14:22:21] [TRT] [I] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:21] [TRT] [W] builtin_op_importers.cpp:4777: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-14:22:21] [TRT] [I] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-14:22:23] [TRT] [W] onnx2trt_utils.cpp:391: One or more weights outside the range of INT32 was clamped
[06/07/2022-14:22:23] [TRT] [W] onnx2trt_utils.cpp:391: One or more weights outside the range of INT32 was clamped
[06/07/2022-14:22:24] [TRT] [W] onnx2trt_utils.cpp:391: One or more weights outside the range of INT32 was clamped
[06/07/2022-14:22:25] [TRT] [W] onnx2trt_utils.cpp:391: One or more weights outside the range of INT32 was clamped
[06/07/2022-14:22:26] [TRT] [W] onnx2trt_utils.cpp:391: One or more weights outside the range of INT32 was clamped
[06/07/2022-14:22:28] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[06/07/2022-14:22:28] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: 
[06/07/2022-14:22:28] [TRT] [I] Successfully created plugin: TRTBatchedNMS
[06/07/2022-14:22:31] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.8.0 but loaded cuBLAS/cuBLAS LT 111.0.1
[06/07/2022-14:22:31] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +191, GPU +74, now: CPU 686, GPU 190 (MiB)
[06/07/2022-14:22:31] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +88, GPU +34, now: CPU 774, GPU 224 (MiB)
[06/07/2022-14:22:31] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[06/07/2022-14:22:33] [TRT] [W] GPU error during getBestTactic: Conv_6 + Relu_7 : an illegal memory access was encountered
[06/07/2022-14:22:33] [TRT] [E] 1: [resizingAllocator.cpp::deallocate::105] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[06/07/2022-14:22:33] [TRT] [E] 1: [virtualMemoryBuffer.cpp::~StdVirtualMemoryBufferImpl::121] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[06/07/2022-14:22:33] [TRT] [E] 10: [optimizer.cpp::computeCosts::2033] Error Code 10: Internal Error (Could not find any implementation for node Conv_6 + Relu_7.)
[06/07/2022-14:22:33] [TRT] [E] 2: [builder.cpp::buildSerializedNetwork::619] Error Code 2: Internal Error (Assertion engine != nullptr failed. )

```

Then i also tried using the `trtexec` to convert it on K80, but it also failed. See error log below.  
I also tried increase the workspace size to 11GB, but also got the same error.

`trtexec` 's verbose log (partial log):

```auto
[06/07/2022-14:44:42] [V] [TRT] Tactic: 10944511 Time: 0.063244
[06/07/2022-14:44:42] [V] [TRT] Fastest Tactic: 2097151 Time: 0.024228
[06/07/2022-14:44:42] [V] [TRT] --------------- Timing Runner: Conv_457 (CudnnConvolution)
[06/07/2022-14:44:42] [V] [TRT] Tactic: 0 Time: 0.064372
[06/07/2022-14:44:42] [V] [TRT] Tactic: 1 skipped. Scratch requested: 1024, available: 0
[06/07/2022-14:44:42] [V] [TRT] Tactic: 2 skipped. Scratch requested: 262144, available: 0
[06/07/2022-14:44:42] [V] [TRT] Tactic: 4 skipped. Scratch requested: 11225088, available: 0
[06/07/2022-14:44:42] [V] [TRT] Tactic: 5 skipped. Scratch requested: 1223168, available: 0
[06/07/2022-14:44:42] [V] [TRT] Fastest Tactic: 0 Time: 0.064372
[06/07/2022-14:44:42] [V] [TRT] Setting workspace to 1024enables more tactics for profiling
[06/07/2022-14:44:42] [V] [TRT] --------------- Timing Runner: Conv_457 (CublasConvolution)
[06/07/2022-14:44:42] [V] [TRT] CublasConvolution has no valid tactics for this config, skipping
[06/07/2022-14:44:42] [V] [TRT] --------------- Timing Runner: Conv_457 (CaskConvolution)
[06/07/2022-14:44:42] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping
[06/07/2022-14:44:42] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: FusedConvActConvolution Tactic: 2097151
[06/07/2022-14:44:42] [V] [TRT] ***************Autotuning format combination: Float(65536,1,4096,256) -> Float(4608,1,288,18)***************
[06/07/2022-14:44:42] [V] [TRT] --------------- Timing Runner: Conv_457 (CudnnConvolution)
[06/07/2022-14:44:42] [V] [TRT] CudnnConvolution has no valid tactics for this config, skipping
[06/07/2022-14:44:42] [V] [TRT] --------------- Timing Runner: Conv_457 (CublasConvolution)
[06/07/2022-14:44:42] [V] [TRT] CublasConvolution has no valid tactics for this config, skipping
[06/07/2022-14:44:42] [V] [TRT] --------------- Timing Runner: Conv_457 (CaskConvolution)
[06/07/2022-14:44:42] [V] [TRT] CaskConvolution has no valid tactics for this config, skipping
[06/07/2022-14:44:42] [V] [TRT] =============== Computing costs for
[06/07/2022-14:44:42] [V] [TRT] ***************Autotuning format combination: Float(1016,1) -> Float(1000,1), Int32(1000,1)***************
[06/07/2022-14:44:42] [V] [TRT] --------------- Timing Runner: TopK_1603 (TopK)
[06/07/2022-14:44:42] [V] [TRT] Tactic: 2 skipped. Scratch requested: 512000, available: 0
[06/07/2022-14:44:42] [V] [TRT] Fastest Tactic: -3360065831133338131 Time: inf
[06/07/2022-14:44:42] [V] [TRT] Deleting timing cache: 481 entries, 1232 hits
[06/07/2022-14:44:42] [E] Error[10]: [optimizer.cpp::computeCosts::2011] Error Code 10: Internal Error (Could not find any implementation for node TopK_1603.)
[06/07/2022-14:44:42] [E] Error[2]: [builder.cpp::buildSerializedNetwork::609] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
[06/07/2022-14:44:42] [E] Engine could not be created from network
[06/07/2022-14:44:42] [E] Building engine failed
[06/07/2022-14:44:42] [E] Failed to create engine from model.
[06/07/2022-14:44:42] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8400] # ./trtexec --onnx=/home/ubuntu/work/work_dir/resnet50_reppoints_reflective_screw_od.onnx --minShapes=input:1x3x244x244 --optShapes=input:1x3x244x244 --maxShapes=input:1x3x244x244 --plugins=/home/ubuntu/work/repos/avi-inference/avi_inference/libs/libmmdeploy_tensorrt_ops.so --workspace=4096 --verbose

```

`trtexec` 's non verbose log (full log):

```auto
./trtexec --onnx=/home/ubuntu/work/work_dir/resnet50_reppoints_reflective_screw_od.onnx --minShapes=input:1x3x244x244 --optShapes=input:1x3x244x244 --maxShapes=input:1x3x244x244 --plugins=/home/ubuntu/work/repos/avi-inference/avi_inference/libs/libmmdeploy_tensorrt_ops.so
&&&& RUNNING TensorRT.trtexec [TensorRT v8400] # ./trtexec --onnx=/home/ubuntu/work/work_dir/resnet50_reppoints_reflective_screw_od.onnx --minShapes=input:1x3x244x244 --optShapes=input:1x3x244x244 --maxShapes=input:1x3x244x244 --plugins=/home/ubuntu/work/repos/avi-inference/avi_inference/libs/libmmdeploy_tensorrt_ops.so
[06/07/2022-18:33:09] [I] === Model Options ===
[06/07/2022-18:33:09] [I] Format: ONNX
[06/07/2022-18:33:09] [I] Model: /home/ubuntu/work/work_dir/resnet50_reppoints_reflective_screw_od.onnx
[06/07/2022-18:33:09] [I] Output:
[06/07/2022-18:33:09] [I] === Build Options ===
[06/07/2022-18:33:09] [I] Max batch: explicit batch
[06/07/2022-18:33:09] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[06/07/2022-18:33:09] [I] minTiming: 1
[06/07/2022-18:33:09] [I] avgTiming: 8
[06/07/2022-18:33:09] [I] Precision: FP32
[06/07/2022-18:33:09] [I] LayerPrecisions:
[06/07/2022-18:33:09] [I] Calibration:
[06/07/2022-18:33:09] [I] Refit: Disabled
[06/07/2022-18:33:09] [I] Sparsity: Disabled
[06/07/2022-18:33:09] [I] Safe mode: Disabled
[06/07/2022-18:33:09] [I] DirectIO mode: Disabled
[06/07/2022-18:33:09] [I] Restricted mode: Disabled
[06/07/2022-18:33:09] [I] Save engine:
[06/07/2022-18:33:09] [I] Load engine:
[06/07/2022-18:33:09] [I] Profiling verbosity: 0
[06/07/2022-18:33:09] [I] Tactic sources: Using default tactic sources
[06/07/2022-18:33:09] [I] timingCacheMode: local
[06/07/2022-18:33:09] [I] timingCacheFile:
[06/07/2022-18:33:09] [I] Input(s)s format: fp32:CHW
[06/07/2022-18:33:09] [I] Output(s)s format: fp32:CHW
[06/07/2022-18:33:09] [I] Input build shape: input=1x3x244x244+1x3x244x244+1x3x244x244
[06/07/2022-18:33:09] [I] Input calibration shapes: model
[06/07/2022-18:33:09] [I] === System Options ===
[06/07/2022-18:33:09] [I] Device: 0
[06/07/2022-18:33:09] [I] DLACore:
[06/07/2022-18:33:09] [I] Plugins: /home/ubuntu/work/repos/avi-inference/avi_inference/libs/libmmdeploy_tensorrt_ops.so
[06/07/2022-18:33:09] [I] === Inference Options ===
[06/07/2022-18:33:09] [I] Batch: Explicit
[06/07/2022-18:33:09] [I] Input inference shape: input=1x3x244x244
[06/07/2022-18:33:09] [I] Iterations: 10
[06/07/2022-18:33:09] [I] Duration: 3s (+ 200ms warm up)
[06/07/2022-18:33:09] [I] Sleep time: 0ms
[06/07/2022-18:33:09] [I] Idle time: 0ms
[06/07/2022-18:33:09] [I] Streams: 1
[06/07/2022-18:33:09] [I] ExposeDMA: Disabled
[06/07/2022-18:33:09] [I] Data transfers: Enabled
[06/07/2022-18:33:09] [I] Spin-wait: Disabled
[06/07/2022-18:33:09] [I] Multithreading: Disabled
[06/07/2022-18:33:09] [I] CUDA Graph: Disabled
[06/07/2022-18:33:09] [I] Separate profiling: Disabled
[06/07/2022-18:33:09] [I] Time Deserialize: Disabled
[06/07/2022-18:33:09] [I] Time Refit: Disabled
[06/07/2022-18:33:09] [I] Skip inference: Disabled
[06/07/2022-18:33:09] [I] Inputs:
[06/07/2022-18:33:09] [I] === Reporting Options ===
[06/07/2022-18:33:09] [I] Verbose: Disabled
[06/07/2022-18:33:09] [I] Averages: 10 inferences
[06/07/2022-18:33:09] [I] Percentile: 99
[06/07/2022-18:33:09] [I] Dump refittable layers:Disabled
[06/07/2022-18:33:09] [I] Dump output: Disabled
[06/07/2022-18:33:09] [I] Profile: Disabled
[06/07/2022-18:33:09] [I] Export timing to JSON file:
[06/07/2022-18:33:09] [I] Export output to JSON file:
[06/07/2022-18:33:09] [I] Export profile to JSON file:
[06/07/2022-18:33:09] [I]
[06/07/2022-18:33:09] [I] === Device Information ===
[06/07/2022-18:33:09] [I] Selected Device: Tesla K80
[06/07/2022-18:33:09] [I] Compute Capability: 3.7
[06/07/2022-18:33:09] [I] SMs: 13
[06/07/2022-18:33:09] [I] Compute Clock Rate: 0.8235 GHz
[06/07/2022-18:33:09] [I] Device Global Memory: 11441 MiB
[06/07/2022-18:33:09] [I] Shared Memory per SM: 112 KiB
[06/07/2022-18:33:09] [I] Memory Bus Width: 384 bits (ECC enabled)
[06/07/2022-18:33:09] [I] Memory Clock Rate: 2.505 GHz
[06/07/2022-18:33:09] [I]
[06/07/2022-18:33:09] [I] TensorRT version: 8.4.0
[06/07/2022-18:33:09] [I] Loading supplied plugin library: /home/ubuntu/work/repos/avi-inference/avi_inference/libs/libmmdeploy_tensorrt_ops.so
[06/07/2022-18:33:09] [I] [TRT] [MemUsageChange] Init CUDA: CPU +156, GPU +0, now: CPU 167, GPU 115 (MiB)
[06/07/2022-18:33:10] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 167 MiB, GPU 115 MiB
[06/07/2022-18:33:10] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 230 MiB, GPU 115 MiB
[06/07/2022-18:33:10] [I] Start parsing network model
[06/07/2022-18:33:10] [I] [TRT] ----------------------------------------------------------------
[06/07/2022-18:33:10] [I] [TRT] Input filename: /home/ubuntu/work/work_dir/resnet50_reppoints_reflective_screw_od.onnx
[06/07/2022-18:33:10] [I] [TRT] ONNX IR version: 0.0.7
[06/07/2022-18:33:10] [I] [TRT] Opset version: 11
[06/07/2022-18:33:10] [I] [TRT] Producer name: pytorch
[06/07/2022-18:33:10] [I] [TRT] Producer version: 1.10
[06/07/2022-18:33:10] [I] [TRT] Domain:
[06/07/2022-18:33:10] [I] [TRT] Model version: 0
[06/07/2022-18:33:10] [I] [TRT] Doc string:
[06/07/2022-18:33:10] [I] [TRT] ----------------------------------------------------------------
[06/07/2022-18:33:10] [W] [TRT] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[06/07/2022-18:33:10] [W] [TRT] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [W] [TRT] builtin_op_importers.cpp:4780: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-18:33:10] [I] [TRT] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-18:33:10] [I] [TRT] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:10] [W] [TRT] builtin_op_importers.cpp:4780: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-18:33:10] [I] [TRT] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [W] [TRT] builtin_op_importers.cpp:4780: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [W] [TRT] builtin_op_importers.cpp:4780: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [W] [TRT] builtin_op_importers.cpp:4780: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-18:33:11] [I] [TRT] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-18:33:11] [I] [TRT] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:11] [W] [TRT] builtin_op_importers.cpp:4780: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-18:33:11] [I] [TRT] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-18:33:12] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:12] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:12] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:12] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:12] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:12] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:12] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:12] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:12] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:12] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:12] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:12] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:12] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:12] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:12] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:12] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:12] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:12] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:12] [I] [TRT] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-18:33:12] [I] [TRT] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:12] [W] [TRT] builtin_op_importers.cpp:4780: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-18:33:12] [I] [TRT] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-18:33:12] [I] [TRT] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-18:33:12] [I] [TRT] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:12] [W] [TRT] builtin_op_importers.cpp:4780: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-18:33:12] [I] [TRT] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-18:33:13] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:13] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:13] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:13] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:13] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:13] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:13] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:13] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:13] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:13] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:13] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:13] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:13] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:13] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:13] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:13] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[06/07/2022-18:33:13] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:13] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[06/07/2022-18:33:13] [I] [TRT] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-18:33:13] [I] [TRT] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:13] [W] [TRT] builtin_op_importers.cpp:4780: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-18:33:13] [I] [TRT] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-18:33:13] [I] [TRT] No importer registered for op: MMCVDeformConv2d. Attempting to import as plugin.
[06/07/2022-18:33:13] [I] [TRT] Searching for plugin: MMCVDeformConv2d, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:13] [W] [TRT] builtin_op_importers.cpp:4780: Attribute deform_groups not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[06/07/2022-18:33:13] [I] [TRT] Successfully created plugin: MMCVDeformConv2d
[06/07/2022-18:33:17] [W] [TRT] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[06/07/2022-18:33:19] [W] [TRT] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[06/07/2022-18:33:22] [W] [TRT] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[06/07/2022-18:33:24] [W] [TRT] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[06/07/2022-18:33:27] [W] [TRT] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped

[06/07/2022-18:33:31] [I] [TRT] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[06/07/2022-18:33:31] [I] [TRT] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace:
[06/07/2022-18:33:31] [I] [TRT] Successfully created plugin: TRTBatchedNMS
[06/07/2022-18:33:31] [W] [TRT] Output type must be INT32 for shape outputs
[06/07/2022-18:33:31] [W] [TRT] Output type must be INT32 for shape outputs
[06/07/2022-18:33:31] [W] [TRT] Output type must be INT32 for shape outputs
[06/07/2022-18:33:31] [W] [TRT] Output type must be INT32 for shape outputs
[06/07/2022-18:33:31] [W] [TRT] Output type must be INT32 for shape outputs
[06/07/2022-18:33:31] [I] Finish parsing network model
[06/07/2022-18:33:33] [W] [TRT] TensorRT was linked against cuBLAS/cuBLASLt 11.6.5 but loaded cuBLAS/cuBLASLt 11.4.1
[06/07/2022-18:33:33] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +123, GPU +49, now: CPU 525, GPU 164 (MiB)
[06/07/2022-18:33:33] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +89, GPU +34, now: CPU 614, GPU 198 (MiB)
[06/07/2022-18:33:33] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[06/07/2022-18:33:42] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[06/07/2022-18:35:20] [E] Error[10]: [optimizer.cpp::computeCosts::2011] Error Code 10: Internal Error (Could not find any implementation for node TopK_1603.)
[06/07/2022-18:35:20] [E] Error[2]: [builder.cpp::buildSerializedNetwork::609] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed. )
[06/07/2022-18:35:20] [E] Engine could not be created from network
[06/07/2022-18:35:20] [E] Building engine failed
[06/07/2022-18:35:20] [E] Failed to create engine from model.
[06/07/2022-18:35:20] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8400] # ./trtexec --onnx=/home/ubuntu/work/work_dir/resnet50_reppoints_reflective_screw_od.onnx --minShapes=input:1x3x244x244 --optShapes=input:1x3x244x244 --maxShapes=input:1x3x244x244 --plugins=/home/ubuntu/work/repos/avi-inference/avi_inference/libs/libmmdeploy_tensorrt_ops.so

```

btw, i also opened an issue in here: [Failed converting ONNX model to TensorRT model · Issue #2039 · NVIDIA/TensorRT · GitHub](https://github.com/NVIDIA/TensorRT/issues/2039)

## Environment

**TensorRT Version** : 8.4.0.6  
**NVIDIA GPU** : K80  
**NVIDIA Driver Version** : 470.129.06  
**CUDA Version** : cuda\_11.2.r11.2/compiler.29618528\_0  
**CUDNN Version** : 8.4.1.50  
**Operating System** : ubuntu 20.04  
**Python Version (if applicable)** : 3.7  
**Tensorflow Version (if applicable)** :  
**PyTorch Version (if applicable)** : 1.10.1+cu113  
**Baremetal or Container (if so, version)** : Baremetal (AWS P2 instance)

## Relevant Files

ONNX model file: [https://drive.google.com/file/d/12LDQNM2Qn8bRH7Ejt3Tjnfj15FbCuH7J/view?usp=sharing](https://drive.google.com/file/d/12LDQNM2Qn8bRH7Ejt3Tjnfj15FbCuH7J/view?usp=sharing)  
Plugin file:  
[https://drive.google.com/file/d/1BkO4oPLBgzhl7SxaUtSRtc\_uJaY-LBxL/view?usp=sharing](https://drive.google.com/file/d/1BkO4oPLBgzhl7SxaUtSRtc_uJaY-LBxL/view?usp=sharing)

## Steps To Reproduce

Followed the instructions [here](https://github.com/NVIDIA/TensorRT/tree/main/samples/trtexec) to build `trtexec` , and then run the following command:

```auto
./trtexec --onnx=/home/ubuntu/work/work_dir/resnet50_reppoints_reflective_screw_od.onnx --minShapes=input:1x3x244x244 --optShapes=input:1x3x244x244 --maxShapes=input:1x3x244x244 --plugins=/home/ubuntu/work/repos/avi-inference/avi_inference/libs/libmmdeploy_tensorrt_ops.so --workspace=4096 --verbose

```

See the ONNX model file and the plugin file in the “Relevant Files” section.

Please include:

- Exact steps/commands to build your repro
- Exact steps/commands to run your repro
- Full traceback of errors encountered

---

<div class="post-metadata">

**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\
**Post date:** [June 8, 2022, 5:37pm UTC](https://forums.developer.nvidia.com/t/failed-converting-onnx-model-to-tensorrt-model/216981/2 "2022-06-08T17:37:53Z")

</div>

Hi,  
Request you to share the ONNX model and the script if not shared already so that we can assist you better.  
Alongside you can try few things:

> **[Quick Start Guide — NVIDIA TensorRT](https://docs.nvidia.com/deeplearning/tensorrt/latest/getting-started/quick-start-guide.html)**

1. validating your model with the below snippet

# check\_model.py

import sys  
import onnx  
filename = yourONNXmodel  
model = onnx.load(filename)  
onnx.checker.check\_model(model).  
2) Try running your model with trtexec command.

> **[TensorRT/samples/trtexec at master · NVIDIA/TensorRT](https://github.com/NVIDIA/TensorRT/tree/master/samples/trtexec)**
>
> NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. - NVIDIA/TensorRT

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging  
Thanks!

---

<div class="post-metadata">

**Author:** ![user10264](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@user10264](https://forums.developer.nvidia.com/u/user10264)\
**Post date:** [June 8, 2022, 5:51pm UTC](https://forums.developer.nvidia.com/t/failed-converting-onnx-model-to-tensorrt-model/216981/5 "2022-06-08T17:51:44Z")

</div>

Thanks @NVES.  
I did both. No errors found in check\_modoel.py.  
I can see errors in the execution of `trtexec`.  
The error is included in the post above.  
The model and plugin file are also included.

---

<div class="post-metadata">

**Author:** ![spolisetty](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/spolisetty/32/14047_2.png) [@spolisetty](https://forums.developer.nvidia.com/u/spolisetty)\
**Post date:** [June 13, 2022, 10:32am UTC](https://forums.developer.nvidia.com/t/failed-converting-onnx-model-to-tensorrt-model/216981/6 "2022-06-13T10:32:12Z")

</div>

Hi,

As suggested in the git issue please try downgrading your CUDA to 10.2.

Thank you.