What is the difference between building an engine and serializing a model?

Hi

I am learning to use TensorRT.
So far, I have done the following:

  1. Convert TensorFlow model to .onnx
  2. Convert .onnx file to .engine using “trtexec”

Which means that I have still not serialized the model, right?
When serializing…what will be the new extension of the file?
I have read that serializing the model is to store it for later use, but the .engine file is already stored for later use ( ex. I created the .engine file yesterday and I am using it today without using tf2onnx nor trtexec). I do not understand the problem.

Thank you

Hi,

The .engine file generated by trtexec is the serialized engine.
It dump all the requirement for TensorRT inference into a binary file.
This allow you to restore the TensorRT inference simply with the .engine file directly.

The file is a serialized bitstream, it doesn’t have a fixed file extension.
Here are some common usage for your reference: .engine, .plan or .trt.

Thanks.

1 Like