TensorRT : saving models as portable PLAN files

Hello,

Can we save the model as PLAN files using C++ API ?
Form below reference we understood that TensorRT models can be saved as PLAN files using Python API only.

https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/topics/topics/workflows/caffe_to_tensorrt.html

and https://devblogs.nvidia.com/tensorrt-3-faster-tensorflow-inference/

Regards,
njs

Hi njs, the PLAN file that the document is referred to is simply the serialized TensorRT engine (which contains the parameters of the optimized model). The serialization can be done using the C++ API nvinfer1::ICudaEngine::serialize() function, please see these code snippets:

https://github.com/dusty-nv/jetson-inference/blob/8ed492bfdc9e1b98f7711cd5ae224ce588635656/tensorNet.cpp#L192
https://github.com/dusty-nv/jetson-inference/blob/8ed492bfdc9e1b98f7711cd5ae224ce588635656/tensorNet.cpp#L245

Note that the TensorRT Python API is currently unsupported/unavailable on Jetson/arm64, so this C++ API is the primary way.

Also note that you can copy the PLAN between other Jetson TX1’s, however if you change to a Jetson TX2 or other GPU system, you should re-generate the PLAN file because TensorRT performs device-specific profiling and optimizations to determine the best path for the particular hardware.