ONNX -> TensorRT convertAxis assertion failed

WildChlamydia · March 3, 2020, 3:08pm

Hello. I’m trying to convert default faster rcnn inception from TF OD API to TensorRT.

I’m sucessfully converted model to ONNX format and tryin to convert it now to trt using trtexec or C++ parser code, but Im faced this error:

While parsing node number 265 [Concat -> "GridAnchorGenerator/Meshgrid_1/ExpandedShape/concat:0"]:
--- Begin node ---
input: "GridAnchorGenerator/Meshgrid_1/ExpandedShape/Slice:0"
input: "const_fold_opt__2097"
input: "GridAnchorGenerator/Meshgrid_1/ExpandedShape/Slice_1:0"
output: "GridAnchorGenerator/Meshgrid_1/ExpandedShape/concat:0"
name: "GridAnchorGenerator/Meshgrid_1/ExpandedShape/concat"
op_type: "Concat"
attribute {
  name: "axis"
  i: 0
  type: INT
}

--- End node ---
ERROR: /mnt/Data2/reps/TensorRT/parsers/onnx/onnx2trt_utils.cpp:203 In function convertAxis:
[8] Assertion failed: axis >= 0 && axis < nbDims
[03/03/2020-18:00:41] [E] Failed to parse onnx file

Any chance to fix it or workaround?
TensorRT and onnx-tensorrt are the latest built from masters.

SunilJB · March 4, 2020, 5:10am

Hi,

Can you try few things:

Check ONNX model using checker function and see if it passes?

import onnx
model = onnx.load("model.onnx")
onnx.checker.check_model(model)

If (1) passes, maybe try onnx-simplifier on it.
If (2) doesn’t work, could you try to see if anything looks off in Netron when viewing the failing nodes

Please refer below link, in case it helps:
https://github.com/onnx/onnx-tensorrt/issues/125#issuecomment-589932695

Thanks

WildChlamydia · March 10, 2020, 2:38pm

SunilJB:

Hi,

Can you try few things:

Check ONNX model using checker function and see if it passes?
import onnx
model = onnx.load("model.onnx")
onnx.checker.check_model(model)
If (1) passes, maybe try onnx-simplifier on it.

If (2) doesn’t work, could you try to see if anything looks off in Netron when viewing the failing nodes

Please refer below link, in case it helps:
https://github.com/onnx/onnx-tensorrt/issues/125#issuecomment-589932695

Thanks

Thank you for answer.

Yes, I tried, no errors:

>>> model = onnx.load('model.onnx')
>>> onnx.checker.check_model(model)
>>>

I get this error with simplifier:
python3 -m onnxsim /home/undead/faces_rcnn.onnx /home/undead/faces_rcnn_simpl.on - Pastebin.com

WildChlamydia · March 11, 2020, 10:34am

I succeeded to use onnx-simplifier with flag --skip-optimization.
But for RCNN model it’s still failed, this time because of super-big size:

ValueError: Message ONNX_REL_1_6.ModelProto exceeds maximum protobuf size of 2GB: 5451307912

I tried mobilenet from TF OD API, and simplifier with --skip-optimization worked. But still not loads in C++:

--- Begin node ---
output: "detection_classes:0"
name: "node_detection_classes:0"
op_type: "Constant"
attribute {
  name: "value"
  t {
    dims: 1
    dims: 50
    data_type: 1
    name: "detection_classes:0"
    raw_data: "..."
  }
  type: TENSOR
}

--- End node ---
ERROR: /mnt/Data2/reps/TensorRT/parsers/onnx/ModelImporter.cpp:493 In function importModel:
[7] Assertion failed: _importer_ctx.tensors().at(output.name()).is_tensor()
Status of load: Can't parse model file

I’m also succeed to simplify normally without --skip-optimization mobilenet, taken other output node.
In Python simplified model loads without errors and pass check. But error is the same as above…

I’m also succeed to convert this model with different output to .onnx with opset 9 (before it did not work, however opset 9 is less error prone than 11). But still error when I try to load in +C+, this time different:

ONNX IR version:  0.0.6
Opset version:    9
Producer name:    tf2onnx
Producer version: 1.6.0
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
[03/11/2020-13:50:11] [E] [TRT] Parameter check failed at: ../builder/Network.cpp::addInput::962, condition: inName != knownInputs->name
ERROR: /mnt/Data2/reps/TensorRT/parsers/onnx/ModelImporter.cpp:493 In function importModel:
[7] Assertion failed: _importer_ctx.tensors().at(output.name()).is_tensor()
ERROR: image_tensor:0:206 In function importInput:
[8] Assertion failed: *tensor = ctx->network()->addInput(input.name().c_str(), trtDtype, trt_dims)
Status of load: Can't parse model file

Looks like it’s close to impossible to load something from TF to TRT via any of parsers.
UFF even less chances than ONNX.

Possible to convert to cuda engine in python, but cuda engine is depending on architecture of particular card and doesnt work when deploy.

SunilJB · March 12, 2020, 4:39am

Hi,

Could you please share the model file so we can help better?
Meanwhile you can try to use trtexec command line tool in --verbose mode.

https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec

Thanks

WildChlamydia · March 12, 2020, 12:56pm

Sure, thanks.

Here are two models: https://drive.google.com/drive/folders/1fcxDDo0rcfBvdA-xZqXyZQe6PtMlXvgR?usp=sharing
After export to ONNX and after simplifier.

Here are outputs with verbose for regular and simplified:

SunilJB · March 13, 2020, 1:35am

Thanks for sharing the model files, we will look into it and update you.

SunilJB · March 13, 2020, 1:39am

Could you please provide details on the platforms you are using so we can better help:
o Linux distro and version
o GPU type
o Nvidia driver version
o CUDA version
o CUDNN version
o Python version [if using python]
o Tensorflow and PyTorch version
o TensorRT version

Thanks

WildChlamydia · March 13, 2020, 9:50am

Of course:

Ubuntu 18.04.4 LTS - Linux 5.3.0-40-generic
1060 Ti
nvidia-driver-440
Cuda 10.2
Cudnn 7.6.5
Python 3.6.9 but problem in C++ (or trtexec or whatever its the same library code)
TF 1.15.2, not using pytorch
TensorRT 7.0.0.11 but also tried to build latest master from Github

Also I had on start problem with unrecognized type UINT8 but I edited code manually and cast UINT8 to INT8 (not safe but obviously problem here not with it).

WildChlamydia · March 17, 2020, 5:00pm

@SunilJB Hello, any news? Sorry seems I uploaded incorrect models (not good output). I updated drive with new one (bigger one) exported with usual outputs (detection_boxes:0,detection_scores:0,detection_classes:0).

SunilJB · March 18, 2020, 3:43am

Hi @WildChlamydia,
Sorry for the delay.
We are looking into it. As soon as we have any updates will share it on forum.

Thanks

WildChlamydia · March 18, 2020, 9:24am

Thank you very much for your work

WildChlamydia · March 18, 2020, 11:42am

Also, if I export without any tricks, just take network, run tf2onnx and try to load in trtexec:

ERROR: TensorRT/parsers/onnx/ModelImporter.cpp:134 In function parseGraph:
[8] No importer registered for op: NonZero

I think the easiest way is to beat this? But how to avoid it if it is not implemented?
Here is model with NonZero error: https://drive.google.com/file/d/1ZEj2iUTOjJHVcgu8wLFdGl0O3-ctFP6m/view?usp=sharing

P.S. Can you rename topic to something like “Load TF OD API models via ONNX into TensorRT?”

WildChlamydia · April 5, 2020, 3:46pm

Up.
Any updates? I am sorry, maybe something else I can try? I want very much to load at least something from OD without cuda engine binary file. Because it’s not normal solution for deployment. But after a month for trying I still can not load anything via ONNX or UFF (this one is hopeless).

WildChlamydia · May 18, 2020, 4:51pm

Up, still no progress. Many other developers faced the same in this thread: No importer registered for op: NonZero · Issue #401 · onnx/onnx-tensorrt · GitHub

P.S.
URL to ONNX mode died, here is new: https://drive.google.com/open?id=1Iz9jcemvZdrtzbZlzRht1uARrroP5ulP

SunilJB · May 18, 2020, 6:33pm

Looks like the model inputs are in uint8_t, which is unsupported

TensorRT 7.0 does not support NonZero op. Please refer to below supported op link:
https://github.com/onnx/onnx-tensorrt/blob/master/operators.md

Thanks

Topic		Replies	Views
Problem converting TensorFlow 2-> ONNX model to TensorRT Engine (efficientdet_d0) TensorRT	8	1371	November 17, 2022
Unsupported ONNX data type: UINT8 (2) TensorRT	24	8826	May 6, 2021
Cannot convert SSD ONNX model to TensorRT TensorRT tensorrt	15	2319	November 23, 2022
Converting onnx to trt: [8] No importer registered for op: OneHot TensorRT tensorrt	3	2867	February 2, 2021
[TensorRT] ERROR: Network must have at least one output TensorRT tensorrt	29	2272	September 30, 2021
Mod operator unsupported in TensorRT 8.4.1 (included w/ Jetpack 5.0.2) TensorRT jetpack , tensorrt , cuda , jetson-inference , onnx	5	1505	January 2, 2023
ONNX parse problem with TopK TensorRT	13	4008	July 30, 2021
Running a pytorch network converted to ONNX with TensorRT on the TX2 Jetson TX2	24	8780	October 18, 2021
Problem converting ONNX model to TensorRT Engine for SSD Mobilenet V2 Jetson Nano tensorrt , nvbugs , ssd , onnx	38	8687	October 18, 2021
TensorRT's OnnxParser problem TensorRT tensorrt	6	2287	October 12, 2021

ONNX -> TensorRT convertAxis assertion failed

I’m also succeed to simplify normally without --skip-optimization mobilenet, taken other output node. In Python simplified model loads without errors and pass check. But error is the same as above…

Related Topics

I’m also succeed to simplify normally without --skip-optimization mobilenet, taken other output node.
In Python simplified model loads without errors and pass check. But error is the same as above…