Onnx to torchrt convertion error

Description

I am trying to convert pytorch model to tensorrt engine.
I had successfully converted pytorch model to onnx but when converting onnx to tensorrt engine it is given below error.

Environment

TensorRT Version: 7.2.1
GPU Type: Jetson tx1
CUDA Version: 10.2

Please include:

[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2590
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Conv_498 [Conv] inputs: [2000 → (1, 1088, 60, 80)], [2589 → (256, 1088, 1, 1)], [2590 → (256)],
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 1088, 60, 80)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Conv_498 for ONNX node: Conv_498
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (1, 1), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 256
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 256, 60, 80)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2588 for ONNX tensor: 2588
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Conv_498 [Conv] outputs: [2588 → (1, 256, 60, 80)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu_499 [Relu]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2588
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Relu_499 [Relu] inputs: [2588 → (1, 256, 60, 80)],
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu_499 for ONNX node: Relu_499
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2003 for ONNX tensor: 2003
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Relu_499 [Relu] outputs: [2003 → (1, 256, 60, 80)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Conv_500 [Conv]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2003
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2592
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2593
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Conv_500 [Conv] inputs: [2003 → (1, 256, 60, 80)], [2592 → (384, 256, 3, 3)], [2593 → (384)],
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 256, 60, 80)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Conv_500 for ONNX node: Conv_500
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (2, 2), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 384
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 384, 29, 39)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2591 for ONNX tensor: 2591
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Conv_500 [Conv] outputs: [2591 → (1, 384, 29, 39)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu_501 [Relu]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2591
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Relu_501 [Relu] inputs: [2591 → (1, 384, 29, 39)],
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu_501 for ONNX node: Relu_501
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2006 for ONNX tensor: 2006
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Relu_501 [Relu] outputs: [2006 → (1, 384, 29, 39)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Conv_502 [Conv]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2000
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2595
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2596
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Conv_502 [Conv] inputs: [2000 → (1, 1088, 60, 80)], [2595 → (256, 1088, 1, 1)], [2596 → (256)],
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 1088, 60, 80)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Conv_502 for ONNX node: Conv_502
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (1, 1), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 256
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 256, 60, 80)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2594 for ONNX tensor: 2594
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Conv_502 [Conv] outputs: [2594 → (1, 256, 60, 80)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu_503 [Relu]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2594
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Relu_503 [Relu] inputs: [2594 → (1, 256, 60, 80)],
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu_503 for ONNX node: Relu_503
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2009 for ONNX tensor: 2009
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Relu_503 [Relu] outputs: [2009 → (1, 256, 60, 80)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Conv_504 [Conv]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2009
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2598
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2599
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Conv_504 [Conv] inputs: [2009 → (1, 256, 60, 80)], [2598 → (288, 256, 3, 3)], [2599 → (288)],
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 256, 60, 80)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Conv_504 for ONNX node: Conv_504
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (2, 2), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 288
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 288, 29, 39)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2597 for ONNX tensor: 2597
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Conv_504 [Conv] outputs: [2597 → (1, 288, 29, 39)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu_505 [Relu]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2597
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Relu_505 [Relu] inputs: [2597 → (1, 288, 29, 39)],
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu_505 for ONNX node: Relu_505
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2012 for ONNX tensor: 2012
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Relu_505 [Relu] outputs: [2012 → (1, 288, 29, 39)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Conv_506 [Conv]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2000
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2601
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2602
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Conv_506 [Conv] inputs: [2000 → (1, 1088, 60, 80)], [2601 → (256, 1088, 1, 1)], [2602 → (256)],
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 1088, 60, 80)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Conv_506 for ONNX node: Conv_506
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (1, 1), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 256
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 256, 60, 80)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2600 for ONNX tensor: 2600
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Conv_506 [Conv] outputs: [2600 → (1, 256, 60, 80)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu_507 [Relu]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2600
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Relu_507 [Relu] inputs: [2600 → (1, 256, 60, 80)],
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu_507 for ONNX node: Relu_507
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2015 for ONNX tensor: 2015
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Relu_507 [Relu] outputs: [2015 → (1, 256, 60, 80)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Conv_508 [Conv]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2015
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2604
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2605
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Conv_508 [Conv] inputs: [2015 → (1, 256, 60, 80)], [2604 → (288, 256, 3, 3)], [2605 → (288)],
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 256, 60, 80)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Conv_508 for ONNX node: Conv_508
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (1, 1), prepadding: (1, 1), postpadding: (1, 1), dilations: (1, 1), numOutputs: 288
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 288, 60, 80)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2603 for ONNX tensor: 2603
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Conv_508 [Conv] outputs: [2603 → (1, 288, 60, 80)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu_509 [Relu]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2603
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Relu_509 [Relu] inputs: [2603 → (1, 288, 60, 80)],
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu_509 for ONNX node: Relu_509
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2018 for ONNX tensor: 2018
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Relu_509 [Relu] outputs: [2018 → (1, 288, 60, 80)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Conv_510 [Conv]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2018
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2607
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2608
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Conv_510 [Conv] inputs: [2018 → (1, 288, 60, 80)], [2607 → (320, 288, 3, 3)], [2608 → (320)],
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 288, 60, 80)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Conv_510 for ONNX node: Conv_510
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (3, 3), strides: (2, 2), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 320
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 320, 29, 39)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2606 for ONNX tensor: 2606
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Conv_510 [Conv] outputs: [2606 → (1, 320, 29, 39)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Relu_511 [Relu]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2606
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Relu_511 [Relu] inputs: [2606 → (1, 320, 29, 39)],
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Relu_511 for ONNX node: Relu_511
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2021 for ONNX tensor: 2021
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Relu_511 [Relu] outputs: [2021 → (1, 320, 29, 39)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: MaxPool_512 [MaxPool]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2000
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: MaxPool_512 [MaxPool] inputs: [2000 → (1, 1088, 60, 80)],
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: MaxPool_512 for ONNX node: MaxPool_512
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2022 for ONNX tensor: 2022
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: MaxPool_512 [MaxPool] outputs: [2022 → (1, 1088, 29, 39)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Concat_513 [Concat]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2006
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2012
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2021
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2022
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Concat_513 [Concat] inputs: [2006 → (1, 384, 29, 39)], [2012 → (1, 288, 29, 39)], [2021 → (1, 320, 29, 39)], [2022 → (1, 1088, 29, 39)],
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Concat_513 for ONNX node: Concat_513
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2023 for ONNX tensor: 2023
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Concat_513 [Concat] outputs: [2023 → (1, 2080, 29, 39)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Conv_514 [Conv]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2023
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: fpn.lateral4.weight
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Conv_514 [Conv] inputs: [2023 → (1, 2080, 29, 39)], [fpn.lateral4.weight → (256, 2080, 1, 1)],
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:450: Convolution input dimensions: (1, 2080, 29, 39)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:141: Registering layer: Conv_514 for ONNX node: Conv_514
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:533: Using kernel: (1, 1), strides: (1, 1), prepadding: (0, 0), postpadding: (0, 0), dilations: (1, 1), numOutputs: 256
[02/06/2022-19:02:25] [V] [TRT] builtin_op_importers.cpp:534: Convolution output dimensions: (1, 256, 29, 39)
[02/06/2022-19:02:25] [V] [TRT] ImporterContext.hpp:116: Registering tensor: 2024 for ONNX tensor: 2024
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:179: Conv_514 [Conv] outputs: [2024 → (1, 256, 29, 39)],
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:103: Parsing node: Pad_515 [Pad]
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:119: Searching for input: 2024
[02/06/2022-19:02:25] [V] [TRT] ModelImporter.cpp:125: Pad_515 [Pad] inputs: [2024 → (1, 256, 29, 39)],
ERROR: builtin_op_importers.cpp:2191 In function importPad:
[8] Assertion failed: mode == “constant” && value == 0.f && “This version of TensorRT only supports constant 0 padding!”
[02/06/2022-19:02:25] [E] Failed to parse onnx file
[02/06/2022-19:02:25] [E] Parsing model failed
[02/06/2022-19:02:25] [E] Engine creation failed
[02/06/2022-19:02:25] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --onnx=deblur.onnx --verbose

Hi,

Looks like you’re using an old version of TensorRT. I believe this issue has been fixed in the new release. We recommend you to please try on latest TensorRT verison 8.2
Please refer following similar issue for more details.

Thank you.

I can not use latest version due to business reason. Can I do any other workaround with model or add that layer in trt 7.

Hi,

Unfortunately there is no workaround, if we are using non-0 or non-constant padding TRT 7 cannot handle it.

Thank you.