Description
I want to build engine(mBuilder->buildEngineWithConfig(*mNetwork, *mConfig)) with onnx from the network backbone of yolov4.It works and runs successfully.
But it takes too much time(49mins) to finish the buildEngineWithConfig if I set Flag as FP16(mConfig->setFlag(nvinfer1::BuilderFlag::kFP16)).
Is there a way to speed up?
Environment
TensorRT Version: 7.2.2.3
GPU Type: 3060
Nvidia Driver Version: 471.41
CUDA Version: 11.1
CUDNN Version: 8.0.4.30
Operating System + Version: Windows 10 21H1
Python Version (if applicable): None
TensorFlow Version (if applicable): None
PyTorch Version (if applicable): None
Baremetal or Container (if container which image + tag): None
Relevant Files
if (mRunMode == 1)
{
spdlog::info(“setFp16Mode”);
if (!mBuilder->platformHasFastFp16()) {
spdlog::warn(“the platform do not has fast for fp16”);
}
mBuilder->setFp16Mode(true);
mConfig->setFlag(nvinfer1::BuilderFlag::kFP16);
}
mBuilder->setMaxBatchSize(mBatchSize);
// set the maximum GPU temporary memory which the engine can use at execution time.
mConfig->setMaxWorkspaceSize(10 << 20);
spdlog::info(“fp16 support: {}”,mBuilder->platformHasFastFp16 ());
spdlog::info(“int8 support: {}”,mBuilder->platformHasFastInt8 ());
spdlog::info(“Max batchsize: {}”,mBuilder->getMaxBatchSize());
spdlog::info(“Max workspace size: {}”,mConfig->getMaxWorkspaceSize());
spdlog::info(“Number of DLA core: {}”,mBuilder->getNbDLACores());
spdlog::info(“Max DLA batchsize: {}”,mBuilder->getMaxDLABatchSize());
spdlog::info(“Current use DLA core: {}”,mConfig->getDLACore()); // TODO: set DLA core
spdlog::info(“build engine…”);
mEngine = mBuilder->buildEngineWithConfig(*mNetwork, *mConfig);