Description
I decided to upgrade to tensorrt 8 and when i tried to run the app i was working on i encountered errors when loading the onnx network or serialized engine.
I checked the documentation and i saw that the function buildEngineWithConfig was deprecated so i changed the code to use buildSerializedNetwork but both functions are failing to return anything except a null pointer. I set the logger severity to display all information available and i could see that the builder is trying to optimize the network but exits the function too early.
I tried to use the serialized model generated with TensorRT 7 but the runtime object fails to return an engine object just like the builder.
Environment
TensorRT Version: 8.0.1.1
GPU Type: GTX 1660
Nvidia Driver Version: 440.82
CUDA Version: 10.2
CUDNN Version:
Operating System + Version: Ubuntu 18.04
Python Version (if applicable): 3.6.9
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):
nvinfer1::IBuilder* builder = nvinfer1::createInferBuilder(gLogger);
nvinfer1::INetworkDefinition* network = builder->createNetworkV2(1u << (int)nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_BATCH);
nvonnxparser::IParser* parser = nvonnxparser::createParser(network, gLogger);
// parse ONNX
if (!parser->parseFromFile(“densenet.onnx”, static_cast< int >(nvinfer1::ILogger::Severity::kVERBOSE)))
{
std::cerr << “ERROR: could not parse the model.\n”;
return;
}
nvinfer1::IBuilderConfig config = builder->createBuilderConfig();
config->setMaxWorkspaceSize(1ULL << 25);
// use FP16 mode if possible
if (builder->platformHasFastFp16())
{
config->setFlag(nvinfer1::BuilderFlag::kFP16);
}
// we have only one image in batch
builder->setMaxBatchSize(1);
auto profile = builder->createOptimizationProfile();
profile->setDimensions(“input”, nvinfer1::OptProfileSelector::kMIN, nvinfer1::Dims4(1,3,256,256));
profile->setDimensions(“input”, nvinfer1::OptProfileSelector::kOPT, nvinfer1::Dims4(1,3,256,256));
profile->setDimensions(“input”, nvinfer1::OptProfileSelector::kMAX, nvinfer1::Dims4(1,3,256,256));
config->addOptimizationProfile(profile);
auto data = builder->buildSerializedNetwork(*network, config);
std::ofstream out(“densenet.engine”, std::ios::binary);
out.write(static_cast<char>(data->data()), data->size());
out.close();