Failure creating a trt engine file with batch > 1 from a dynamic onnx file caused by Where operator error

[b_tata1]

Description

Using trtexec.exe to create a trt.engine file from the attached dynamic onnx file with batch > 1 fails with the following error:

ERROR:
[E] Error[4]: [graphShapeAnalyzer.cpp::nvinfer1::builder::`anonymous-namespace’::ShapeNodeRemover::analyzeShapes::1872] Error Code 4: Miscellaneous (IISelectLayer /Where: broadcast dimensions must be conformable)

Environment

GPU Type: NVIDIA RTX A3000 Laptop GPU
Nvidia Driver Version: 511.65
CUDA Version: 11.6, V11.6.124
CUDNN Version: 8.4.1
Operating System + Version: Windows 10
Python Version (if applicable): 3.8.8
TensorRT Version (if applicable): 8.5.1.7
PyTorch Version (if applicable): 1.13.1+cu117
Baremetal or Container (if container which image + tag): Baremetal

Relevant Files - You will find the onnx file, the plugin dll and the c++ code that was used to build the dll in the following link:

OnnxAndPlugin.7z

Steps To Reproduce

1. I ran trtexec.exe with the onnx model to create a trt.engine file whith batch=1, and the trt.engine file was created successfully.
   This is the command I used:
   trtexec.exe --onnx=model.onnx --saveEngine=model.trt.engine --plugins=Plugin.dll --minShapes=input:1x3x1024x1024 --optShapes=input:1x3x1024x1024 --maxShapes=input:1x3x1024x1024 –verbose

2. I replaced the batch number 1 with 10, and ran the following command:

trtexec.exe --onnx=model.onnx --saveEngine=model.trt.engine --plugins=Plugin.dll --minShapes=input:10x3x1024x1024 --optShapes=input:10x3x1024x1024 --maxShapes=input:10x3x1024x1024 –verbose

The creation of trt.engine file failed with the following error:

[E] Error[4]: [graphShapeAnalyzer.cpp::nvinfer1::builder::`anonymous-namespace’::ShapeNodeRemover::analyzeShapes::1872] Error Code 4: Miscellaneous (IISelectLayer /Where: broadcast dimensions must be conformable).

Thank you!

Tanya Z.

@orong13

[b_tata1]

Hi,
Thank you for replying.
I did use onnx and also attached the onnx that I used to the topic.
Thank you.

[spolisetty]

Hi,

[E] Error[4]: [graphShapeAnalyzer.cpp::nvinfer1::builder::`anonymous-namespace’::ShapeNodeRemover::analyzeShapes::1872] Error Code 4: Miscellaneous (IISelectLayer /Where: broadcast dimensions must be conformable)

As the log indicated, the /Where operation sees unconformable broadcast dims. For the Where operation, the condition should have a different rank from either X or Y , which is unsupported by TRT.
Could you please change keepdims to 1 in the ReduceMax operation shown below and let us know is you still face the issue?

Thank you.

[b_tata1]

Hi,

This worked, this time the trt.engine file was created successfully!

I appreciate the quick reply, thank you so much!

Tanya