(Unnamed Layer* 60) [ElementWise]: elementwise inputs must have same dimensions or follow broadcast rules (input dimensions were [2] and [3])

Hi Nvidia Team,

I have converted the Pytorch Model(trained on custom data) to ONNX.
And While I am converting the ONNX to TRT I am getting the issue as attached below:

[06/16/2021-10:25:53] [V] [TRT] Parsing node: Concat_32 [Concat]
[06/16/2021-10:25:53] [V] [TRT] Searching for input: 433
[06/16/2021-10:25:53] [V] [TRT] Searching for input: 14507
[06/16/2021-10:25:53] [V] [TRT] Searching for input: 14508
[06/16/2021-10:25:53] [V] [TRT] Concat_32 [Concat] inputs: [433 -> (1)], [14507 -> (1)], [14508 -> (1)], 
[06/16/2021-10:25:53] [V] [TRT] Registering constant layer: 14507 for ONNX initializer: 14507
[06/16/2021-10:25:53] [V] [TRT] Registering constant layer: 14508 for ONNX initializer: 14508
[06/16/2021-10:25:53] [V] [TRT] Registering layer: Concat_32 for ONNX node: Concat_32
[06/16/2021-10:25:53] [V] [TRT] Registering tensor: 436 for ONNX tensor: 436
[06/16/2021-10:25:53] [V] [TRT] Concat_32 [Concat] outputs: [436 -> (3)], 
[06/16/2021-10:25:53] [V] [TRT] Parsing node: Tile_33 [Tile]
[06/16/2021-10:25:53] [V] [TRT] Searching for input: Transformation.GridGenerator.inv_delta_C
[06/16/2021-10:25:53] [V] [TRT] Searching for input: 436
[06/16/2021-10:25:53] [V] [TRT] Tile_33 [Tile] inputs: [Transformation.GridGenerator.inv_delta_C -> (23, 23)], [436 -> (3)], 
[06/16/2021-10:25:53] [V] [TRT] Registering constant layer: Transformation.GridGenerator.inv_delta_C for ONNX initializer: Transformation.GridGenerator.inv_delta_C
[06/16/2021-10:25:53] [E] [TRT] (Unnamed Layer* 60) [ElementWise]: elementwise inputs must have same dimensions or follow broadcast rules (input dimensions were [2] and [3]).
[06/16/2021-10:25:53] [V] [TRT] Registering layer: Tile_33 for ONNX node: Tile_33
[06/16/2021-10:25:53] [E] [TRT] (Unnamed Layer* 60) [ElementWise]: elementwise inputs must have same dimensions or follow broadcast rules (input dimensions were [2] and [3]).
[06/16/2021-10:25:53] [V] [TRT] Registering tensor: 437 for ONNX tensor: 437
[06/16/2021-10:25:53] [V] [TRT] Tile_33 [Tile] outputs: [437 -> ()], 
[06/16/2021-10:25:53] [V] [TRT] Parsing node: Unsqueeze_34 [Unsqueeze]
[06/16/2021-10:25:53] [V] [TRT] Searching for input: 430
[06/16/2021-10:25:53] [V] [TRT] Unsqueeze_34 [Unsqueeze] inputs: [430 -> ()], 
[06/16/2021-10:25:53] [V] [TRT] Original shape: (), unsqueezing to: (1,)
[06/16/2021-10:25:53] [V] [TRT] Registering layer: Unsqueeze_34 for ONNX node: Unsqueeze_34
[06/16/2021-10:25:53] [E] [TRT] (Unnamed Layer* 60) [ElementWise]: elementwise inputs must have same dimensions or follow broadcast rules (input dimensions were [2] and [3]).
[06/16/2021-10:25:53] [V] [TRT] Registering tensor: 440 for ONNX tensor: 440
[06/16/2021-10:25:53] [V] [TRT] Unsqueeze_34 [Unsqueeze] outputs: [440 -> ()], 
[06/16/2021-10:25:53] [V] [TRT] Parsing node: Concat_35 [Concat]
[06/16/2021-10:25:53] [V] [TRT] Searching for input: 440
[06/16/2021-10:25:53] [E] [TRT] (Unnamed Layer* 60) [ElementWise]: elementwise inputs must have same dimensions or follow broadcast rules (input dimensions were [2] and [3]).
[06/16/2021-10:25:53] [V] [TRT] Searching for input: 14509
[06/16/2021-10:25:53] [V] [TRT] Searching for input: 14510
[06/16/2021-10:25:53] [V] [TRT] Concat_35 [Concat] inputs: [440 -> ()], [14509 -> (1)], [14510 -> (1)], 
[06/16/2021-10:25:53] [V] [TRT] Registering constant layer: 14509 for ONNX initializer: 14509
[06/16/2021-10:25:53] [V] [TRT] Registering constant layer: 14510 for ONNX initializer: 14510
[06/16/2021-10:25:53] [E] [TRT] (Unnamed Layer* 60) [ElementWise]: elementwise inputs must have same dimensions or follow broadcast rules (input dimensions were [2] and [3]).
[06/16/2021-10:25:53] [E] [TRT] /home/darshan/onnx-tensorrt/ModelImporter.cpp:703: While parsing node number 35 [Concat -> "443"]:
[06/16/2021-10:25:53] [E] [TRT] /home/darshan/onnx-tensorrt/ModelImporter.cpp:704: --- Begin node ---
[06/16/2021-10:25:53] [E] [TRT] /home/darshan/onnx-tensorrt/ModelImporter.cpp:705: input: "440"
input: "14509"
input: "14510"
output: "443"
name: "Concat_35"
op_type: "Concat"
attribute {
  name: "axis"
  i: 0
  type: INT
}

[06/16/2021-10:25:53] [E] [TRT] /home/darshan/onnx-tensorrt/ModelImporter.cpp:706: --- End node ---
[06/16/2021-10:25:53] [E] [TRT] /home/darshan/onnx-tensorrt/ModelImporter.cpp:709: ERROR: /home/darshan/onnx-tensorrt/onnx2trt_utils.cpp:196 In function convertAxis:
[8] Assertion failed: axis >= 0 && axis < nbDims
[06/16/2021-10:25:53] [E] Failed to parse onnx file
[06/16/2021-10:25:53] [E] Parsing model failed
[06/16/2021-10:25:53] [E] Engine creation failed
[06/16/2021-10:25:53] [E] Engine set up failed

One issue is:(Unnamed Layer* 60) [ElementWise]: elementwise inputs must have same dimensions or follow broadcast rules (input dimensions were [2] and [3]).

Other is: [8] Assertion failed: axis >= 0 && axis < nbDims

Please help me to resolve this issue.

Looking forward to your reply

Thanks,
Darshan

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

Can I get the better answer? I will get the same message when I file some conversion Issue.

Hi @darshancganji12,

Please let us know which TensorRT version are you using. We recommend you to try on latest TensorRT 8.0 version. And if you still face this issue, we recommend you to please share complete verbose logs and if possible issue reproducible ONNX model(you may DM us also) for better debugging.

Thank you.

Hi @spolisetty,

Thank you so much for your kind reply as always.
As requested I am attaching the Log and the ONNX model below:

I used torch 1.8.1 with Opset 12 for converting to ONNX.

Error_log.txt (218.1 KB)

Here is the ONNX model: Text_recognition_1.8_12.onnx - Google Drive

Thanks,
Darshan C G

@darshancganji12,

Looks like you missed to share these details. Could you please let us know have you tried on TensorRT 8.0 version and which TensorRT version are you currently using.

Thank you.

Hi @spolisetty,

I am using TRT version: 7.1.3.4. Can I get the Docker image for TRT 8.0?

Thanks,
Darshan

Docker image for TRT 8.0 not yet available. you can download latest NGC container and setup TRT 8.0 by following below doc.
https://ngc.nvidia.com/containers/nvidia:tensorrt
https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-debian

Thank you.