Hello,
The Tensorflow pb (which was converted to the uff file) was generated under the following platform:
PC#1:
Linux distro and version - Linux-x86_64, Ubuntu, 16.04
GPU type – GTX-1070TI
nvidia driver version – 384.130
CUDA version – 8.0.44
CUDNN version – 6.0.21
Python version – 3.5.2
Tensorflow version – 1.4.1
TensorRT version – Not used
The TensorRT uff was generated under the following platform:
PC#2:
Linux distro and version - Linux-x86_64, Ubuntu, 16.04
GPU type - GeForce GTX 1080
nvidia driver version - 396.26
CUDA version - Release 9.0, V9.0.252
CUDNN version - 7.1.4
Python version – 3.5.2
Tensorflow version – 1.8
TensorRT version – 4.0.1.6
The TensorRT CUDA engine serialized file was generated under the following platforms:
Only C++ - No Python usage at all.
Only TensorRT - No Tensorflow usage at all
PC#2:
Linux distro and version - Linux-x86_64, Ubuntu, 16.04
GPU type - GeForce GTX 1080
nvidia driver version - 396.26
CUDA version - Release 9.0, V9.0.252
CUDNN version - 7.1.4
Python version – not used
Tensorflow version – Not used
TensorRT version – 4.0.1.6
Jetson TX2 developer kit board:
Linux distro and version –
Ubuntu 16.04.5 LTS (Xenial Xersus)
L4T -
#R28 (release), REVISION 2.1, GCID: 11272647, BOARD: t186ref, EABI: aarch64, DATE: Thu May 17 07:29:06 UTC 2018
GPU type -
As part of the Jetson TX2 developer kit board
JetPack –
3.2.1 (But TensorRT and CUDNN were updated according to JetPack 3.3 versions)
nvidia driver version - As part of the JetPack
CUDA version - Release 9.0, V9.0.252
CUDNN version - 7.1.5
Python version – Not used
Tensorflow version – Not used
TensorRT version – 4.0.1.6
The TensorRT uff was generated using these Python commands:
Import uff
Uff_model = uff.from_tensorflow_frozen_model(“My pb file path”, “List of my graph outputs nodes names”, text = “My uff text file path”, list_modes=False, output_filename=”My uff binary file path”)
The TensorRT CUDA engine serialized file was generated using these C++ commands:
/*Objects and types declarations:*/
nvuffparser::IUffParser *m_parser;
nvinfer1::IBuilder *m_builder;
nvinfer1::INetworkDefinition *m_network;
nvinfer1::ICudaEngine *m_engine;
nvinfer1::IExecutionContext *m_context;
m_parser = nvuffparser::createUffParser();
for (auto& tensorIntput : m_tensorsInputs)
{
m_parser->registerInput(tensorIntput->m_name.c_str(),
nvinfer1::DimsCHW(tensorIntput->m_dimC,
tensorIntput->m_dimH,
tensorIntput->m_dimW),
nvuffparser::UffInputOrder::kNCHW);
}
for (auto& tensorOutput : m_tensorOutputsNames)
{
m_parser->registerOutput(tensorOutput.c_str());
}
m_builder = nvinfer1::createInferBuilder(m_tRTLogger);
m_builder->setMaxBatchSize(m_maxBatchSize);
m_builder->setMaxWorkspaceSize(MAX_WORKSPACE);
m_network = m_builder->createNetwork();
if (!m_parser->parse(m_uffFilePath.c_str(), *m_network, nvinfer1::DataType::kFLOAT))
{
m_tRTLogger.log(nvinfer1::ILogger::Severity::kERROR, std::string("Fail to parse").c_str());
throw std::runtime_error(std::string("CUDA engine serialization operation error!!!"));
}
/*After the network has been defined, build the engine by configuring the builder*/
m_engine = m_builder->buildCudaEngine(*m_network);
/* we can clean the network and the parser */
m_network->destroy();
m_builder->destroy();
nvinfer1::IHostMemory *engineSerialized = m_engine->serialize();
std::ofstream engineSerializedFile;
/*Open a new file for holding the serialized engine data*/
engineSerializedFile.open(engineSerializedFilePath.string(), std::ios::out | std::ios::binary);
if (engineSerializedFile.is_open() && engineSerializedFile.good() && !engineSerializedFile.fail())
{
/*Save the serialized engine data into the file*/
engineSerializedFile.write(reinterpret_cast<const char *>(engineSerialized->data()), engineSerialized->size());
/*Close the file*/
engineSerializedFile.close();
}
Thanks,