How to solve "buildCudaEngine" cost long time

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi,i use TensorRT API to create my net and when the program run to the line “engine_ = builder_->buildCudaEngine(*network_);”, it cost very long time, I want to know why and how to shorten the time ?

thanks!

2

Hi,

buildCudaEngine takes time since TensorRT optimizes the kernel with the information of model and GPU architecture.

A typical workflow is to serialize compiled engine(called PLAN) and relaunch TensorRT with PLAN to save building time.
[url]Developer Guide :: NVIDIA Deep Learning TensorRT Documentation

Here is a sample code of PLAN serialize and de-serialize for your reference:
[url]https://github.com/dusty-nv/jetson-inference/blob/master/tensorNet.cpp#L213[/url]

Thanks.

3

thanks for your reply.

I saved my PLAN to a file and then de-seialize from the PLAN file for my program, but next the line “nvinfer1::ICudaEngine* engine = runtime->deserializeCudaEngine(modelMen, modelsize, &pluginFactory);” failed!

I guess the custom layer’s parameter is not saved in the PLAN file.can you give me some advise?

Is there any some demo for the PLAN file and contain custom layer ?

4

Hi,

You can find here for the plugin sample:

Thanks.