I’m trying to use a pre-trained object detection model from TF2 Model Zoo with TensorRT, but I’m stucked due to errors in engine building.
In particular I would like to convert TF2 saved_model.pb in ONNX format, optimize it with TensorRT and perfom inference with TensorRT engine.
Convert saved_model.pb to ONNX whit this command: python3 -m tf2onnx.convert --saved-model resnet/saved_model/ --opset 13 --output resnet.onnx
Check ONNX file with this code: import onnx try: onnx_model = onnx.load(model_path) onnx.checker.check_model(onnx_model) print("Model is valid") except Exception as e: print(e)
Use buildEngine.py for building TRT engine. I used code posted here NVIDIA Tutorial (Tensorflow 2 code example).
I run the command: python3 buildEngine.py --onnx_file resnet.onnx
At first run I got this error: Unsupported ONNX data type: UINT8
So i run again: python3 buildEngine.py --onnx_file resnet_f32.onnx
and now I have this error:
[TensorRT] WARNING: onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
Traceback (most recent call last):
File “buildEngine.py”, line 31, in
main(args)
File “buildEngine.py”, line 22, in main
engine = eng.build_engine(onnx_path, shape=shape)
File “engine.py”, line 16, in build_engine
parser.parse(model.read())
IndexError: Attribute not found: axes
So my questions:
How can I solve this?
It is possible to use TF2 object detection pre-trained models with TensorRT?
We could reproduce the same error. Looks like you are using the opset 13 version, which is currently unsupported for TRT 7.2.2.3. Can you try exporting your model to a lower opset (i.e. opset 11).
We also recommend you to checkout onnx-simplifier.
I tried to export the model with opset 11 and now I get these errors when running buildEngine.py:
[TensorRT] WARNING: onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] ERROR: Network must have at least one output
[TensorRT] ERROR: Network validation failed.
I tried also onnx-semplifier, but also there I got an error:
onnx.onnx_cpp2py_export.checker.ValidationError: Nodes in a graph must be topologically sorted, however input ‘__inference_Preprocessor_ResizeToRange_cond_false_13101_567_const_zero__42:0’ of node:
name: OpType: Slice is not output of any previous nodes.
Sorry for the delayed response, we recommend you to try Polygraphy. We also recommend you to use latest TensorRT version 8.0 EA.
Please run polygraphy surgeon sanitize --fold-constants model.onnx -o folded.onnx before running trtexec.
Let us know if you still face this issue.