TensorRT : saving models as portable PLAN files


Can we save the model as PLAN files using C++ API ?
Form below reference we understood that TensorRT models can be saved as PLAN files using Python API only.


and https://devblogs.nvidia.com/tensorrt-3-faster-tensorflow-inference/


Hi njs, the PLAN file that the document is referred to is simply the serialized TensorRT engine (which contains the parameters of the optimized model). The serialization can be done using the C++ API nvinfer1::ICudaEngine::serialize() function, please see these code snippets:


Note that the TensorRT Python API is currently unsupported/unavailable on Jetson/arm64, so this C++ API is the primary way.

Also note that you can copy the PLAN between other Jetson TX1’s, however if you change to a Jetson TX2 or other GPU system, you should re-generate the PLAN file because TensorRT performs device-specific profiling and optimizations to determine the best path for the particular hardware.