ONNX to TRT Serialization Error

Loading TensorRT Engine from File Failed

I have a neural network that originally wrote in Keras
I converted it to Onnx, and then to TensorRT

######################################################

FIRST - Keras to Onnx conversion with -> https://github.com/onnx/keras-onnx
Installed with pip, the conversion succeeded.

SECOND - Onnx to trt conversion with -> https://github.com/onnx/onnx-tensorrt
Cloned the repo
To create the engine:

onnx2trt my_model.onnx -o my_engine.trt

To create the txt file: (link to it attached at the end)

onnx2trt my_model.onnx -t my_engine.trt

When trying to load the trt engine I get:

[E] [TRT] (31) - Serialization Error in verifyHeader: 0 (Magic tag does not match)
[E] [TRT] INVALID_STATE: std::exception[E] [TRT] INVALID_CONFIG: Deserialize the cuda engine failed.

The code to load the trt engine is attached below (link to drive)

######################################################
I updated the keras to onnx converter - installed it from source

pip install -U git+https://github.com/microsoft/onnxconverter-common
pip install -U git+https://github.com/onnx/keras-onnx

FIRST: Convert the Keras model to onnx successfully.

SECOND: Convert Onnx to TRT (engine txt file attached in the link below)
The conversion failed with output:

Input filename: /home/shira/Documents/Projects/EmotionsTrtInfer/models/emotion_model_new.onnx
ONNX IR version: 0.0.6
Opset version: 11
Producer name: keras2onnx
Producer version: 1.7.0
Domain: onnxmltools
Model version: 0
Doc string:

WARNING: ONNX model has a newer ir_version (0.0.6) than this parser was built against (0.0.3).
Writing ONNX model (without weights) as text to /home/shira/Documents/Projects/EmotionsTrtInfer/models/emotion_model_new.txt
Parsing model
While parsing node number 1 [Conv -> “convolution_output22”]:
ERROR: /home/shira/Documents/Git/onnx-tensorrt/ModelImporter.cpp:537 In function importModel:
[5] Assertion failed: tensors.count(input_name)

######################################################

Environment

TensorRT Version 7.0.0:
GPU Type GeForce RTX 2080 Ti:
Nvidia Driver Version 440.33.01:
CUDA Version 10.2:
CUDNN Version 7:
Linux 18.04:

Link contain:

  1. Load_trt_engine.txt -> C++ code to load the engine
  2. emotion_model_old.txt -> engine from first onnx to trt conversion
  3. emotion_model_new.txt -> engine from second onnx to trt conversion
    https://drive.google.com/drive/folders/1mLFmsjTXq2fblImAAPty4eggbumrw7zl?usp=sharing

Can you try few things:

  1. Check ONNX model using checker function and see if it passes?
    import onnx

    model = onnx.load(“model.onnx”)

    onnx.checker.check_model(model)

  2. Please try trtexec commands to generate TRT model
    https://github.com/NVIDIA/TensorRT/blob/master/samples/opensource/trtexec/README.md
    If issue persist, could you please share the ONNX model so we can better help.

Thanks

Thank you for answer.
I checked the ONNX model, also did inference with it and it is ok.

Used trtexec and it worked! thanks a lot!

I got same problem.
My scenario is that.

A: PyTorch Model → B: Onnx Model → C: Tensorrt .TRT file

Python Tensorrt inference with B is done successfully.
onnx checking is passed.

but with C is failed with exactly same error
and TRTEXEC also failed with same error

It’s not because of version-matches. because All these evens were made in the single environment.

I uses TensorRT 7.0.0