TensorRT : saving models as portable PLAN files

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hello,

Can we save the model as PLAN files using C++ API ?
Form below reference we understood that TensorRT models can be saved as PLAN files using Python API only.

and https://devblogs.nvidia.com/tensorrt-3-faster-tensorflow-inference/

Regards,
njs

2

Hi njs, the PLAN file that the document is referred to is simply the serialized TensorRT engine (which contains the parameters of the optimized model). The serialization can be done using the C++ API nvinfer1::ICudaEngine::serialize() function, please see these code snippets:

[url]https://github.com/dusty-nv/jetson-inference/blob/8ed492bfdc9e1b98f7711cd5ae224ce588635656/tensorNet.cpp#L192[/url]
[url]https://github.com/dusty-nv/jetson-inference/blob/8ed492bfdc9e1b98f7711cd5ae224ce588635656/tensorNet.cpp#L245[/url]

Note that the TensorRT Python API is currently unsupported/unavailable on Jetson/arm64, so this C++ API is the primary way.

Also note that you can copy the PLAN between other Jetson TX1’s, however if you change to a Jetson TX2 or other GPU system, you should re-generate the PLAN file because TensorRT performs device-specific profiling and optimizations to determine the best path for the particular hardware.