Assertion failed: “If condition must be a initializer!” on efficientdet pretrained model

Relevant Files

Tensorflow model zoo

Steps To Reproduce

I downloaded the pre-trained model from Tensorflow model zoo. After that I tried running tf2onnx command:
python3 -m tf2onnx.convert --saved-model models --output tf_model_op11.onnx --opset 11
This gave an onnx file with uint8 type. So I used Onnx graph surgeon tool to change the type:
graph = gs.import_onnx(onnx.load("tf_model_op11.onnx"))
for inp in graph.inputs:
inp.dtype = np.float32

After getting the onnx file, I ran the command:
trtexec --onnx=model_op11.onnx --fp16=enable --workspace=5500 --batch=1 --saveEngine=model_op11.trt --verbose
It had dynamic input so I checked this command as well to see if I get different error
trtexec --explicitBatch --optShapesCalib=:1x1024x1024x3 --optShapes=:1x1024x1024x3 --onnx=model_op11_v2.onnx --saveEngine=model_op11.trt --shapes=\'input_tensor:0\':1x1024x1024x3 --fp16=enable --workspace=5500 --useCudaGraph=enable --exportTimes=times.json --verbose
This gave me the same error as above. The suggested solution of downgrading torch won’t work as it wasn’t involved.

[06/11/2021-05:02:47] [I] === Model Options ===
[06/11/2021-05:02:47] [I] Format: ONNX
[06/11/2021-05:02:47] [I] Model: model_op11.onnx
[06/11/2021-05:02:47] [I] Output:
[06/11/2021-05:02:47] [I] === Build Options ===
[06/11/2021-05:02:47] [I] Max batch: 1
[06/11/2021-05:02:47] [I] Workspace: 5500 MB
[06/11/2021-05:02:47] [I] minTiming: 1
[06/11/2021-05:02:47] [I] avgTiming: 8
[06/11/2021-05:02:47] [I] Precision: FP32+FP16
[06/11/2021-05:02:47] [I] Calibration:
[06/11/2021-05:02:47] [I] Safe mode: Disabled
[06/11/2021-05:02:47] [I] Save engine: model_op11.trt
[06/11/2021-05:02:47] [I] Load engine:
[06/11/2021-05:02:47] [I] Builder Cache: Enabled
[06/11/2021-05:02:47] [I] NVTX verbosity: 0
[06/11/2021-05:02:47] [I] Inputs format: fp32:CHW
[06/11/2021-05:02:47] [I] Outputs format: fp32:CHW
[06/11/2021-05:02:47] [I] Input build shapes: model
[06/11/2021-05:02:47] [I] Input calibration shapes: model
[06/11/2021-05:02:47] [I] === System Options ===
[06/11/2021-05:02:47] [I] Device: 0
[06/11/2021-05:02:47] [I] DLACore:
[06/11/2021-05:02:47] [I] Plugins:
[06/11/2021-05:02:47] [I] === Inference Options ===
[06/11/2021-05:02:47] [I] Batch: 1
[06/11/2021-05:02:47] [I] Input inference shapes: model
[06/11/2021-05:02:47] [I] Iterations: 10
[06/11/2021-05:02:47] [I] Duration: 3s (+ 200ms warm up)
[06/11/2021-05:02:47] [I] Sleep time: 0ms
[06/11/2021-05:02:47] [I] Streams: 1
[06/11/2021-05:02:47] [I] ExposeDMA: Disabled
[06/11/2021-05:02:47] [I] Spin-wait: Disabled
[06/11/2021-05:02:47] [I] Multithreading: Disabled
[06/11/2021-05:02:47] [I] CUDA Graph: Disabled
[06/11/2021-05:02:47] [I] Skip inference: Disabled
[06/11/2021-05:02:47] [I] Inputs:
[06/11/2021-05:02:47] [I] === Reporting Options ===
[06/11/2021-05:02:47] [I] Verbose: Enabled
[06/11/2021-05:02:47] [I] Averages: 10 inferences
[06/11/2021-05:02:47] [I] Percentile: 99
[06/11/2021-05:02:47] [I] Dump output: Disabled
[06/11/2021-05:02:47] [I] Profile: Disabled
[06/11/2021-05:02:47] [I] Export timing to JSON file:
[06/11/2021-05:02:47] [I] Export output to JSON file:
[06/11/2021-05:02:47] [I] Export profile to JSON file:

Input filename: model_op11.onnx
ONNX IR version: 0.0.7
Opset version: 11
Producer name:
Producer version:
Domain:
Model version: 0
Doc string:

[06/11/2021-05:02:49] [V] [TRT] ModelImporter.cpp:103: Parsing node: StatefulPartitionedCall/Preprocessor/ResizeToRange/cond [If]
[06/11/2021-05:02:49] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/Preprocessor/ResizeToRange/Less:0
[06/11/2021-05:02:49] [V] [TRT] ModelImporter.cpp:125: StatefulPartitionedCall/Preprocessor/ResizeToRange/cond [If] inputs: [StatefulPartitionedCall/Preprocessor/ResizeToRange/Less:0 → ()],
ERROR: builtin_op_importers.cpp:1554 In function importIf:
[8] Assertion failed: cond.is_weights() && cond.weights().count() == 1 && “If condition must be a initializer!”
[06/11/2021-05:02:49] [E] Failed to parse onnx file
[06/11/2021-05:02:49] [E] Parsing model failed
[06/11/2021-05:02:49] [E] Engine creation failed
[06/11/2021-05:02:49] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # trtexec --onnx=model_op11.onnx --fp16=enable --workspace=5500 --batch=1 --saveEngine=model_op11.trt --verbose

The whole verbose output

TensorRT Version: 7.1.3-1+cuda10.2
NVIDIA GPU: NVIDIA Tegra Xavier (nvgpu)/integrated
NVIDIA Driver Version: 32.4.4 Release Build
CUDA Version: 10.2
Operating System: Ubuntu 18.04
Python Version (if applicable): 3.6.9
Tensorflow Version (if applicable): 2.4
Graphics : NVIDIA Tegra Xavier (nvgpu)/integrated
Processor : ARMv8 Processor rev 0 (v8l) × 6
onnx=1.9.0
tf2onnx=1.8.5/50049d**:

In another post, the suggestion was to downgrade pytorch version but I’m not using pytorch anywhere.

Hi @romilaggarwal611,

Please share ONNX file with us to try from our end for better assistance.

Thank you.

Hi, it seems that this post was made twice by mistake. Sorry.

Here’s the link to another thread of the same post. Another moderator is present on that thread so this one can be closed(or should i delete it?).

Hi @romilaggarwal611,

Sorry for the delayed response, based on the verbose logs looks like you’re using unsupported operation ‘Round’ by TensorRT. We recommend you to implement custom plugin for this.
For your reference,
https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#add_custom_layer_python

you can check supported operators here onnx-tensorrt/operators.md at master · onnx/onnx-tensorrt · GitHub

Please refer to below links related custom plugin implementation and sample:

Thank you.