Hi,
I also run into the same failure when I parse the onnx model using the cpp api.
Here is my cpp code:
nvinfer1::INetworkDefinition* network = builder->createNetworkV2(1U << static_cast<uint32_t>(NetworkDefinitionCreationFlag::kEXPLICIT_BATCH));
nvonnxparser::IParser* parser = nvonnxparser::createParser(*network, gLogger.getTRTLogger());
parser->parseFromFile("/mnt/test/model1.onnx",2);
auto preprocessorConfig = builder->createBuilderConfig();
// Create an optimization profile so that we can specify a range of input dimensions.
auto profile = builder->createOptimizationProfile();
// This profile will be valid for all images whose size falls in the range of [(1, 1, 1, 1), (1, 1, 56, 56)]
// but TensorRT will optimize for (1, 1, 28, 28)
profile->setDimensions("segment_ids_1:0", OptProfileSelector::kMIN, Dims2{ 766, 64 });
profile->setDimensions("segment_ids_1:0", OptProfileSelector::kOPT, Dims2{ 766, 64 });
profile->setDimensions("segment_ids_1:0", OptProfileSelector::kMAX, Dims2{ 766, 64 });
profile->setDimensions("input_mask_1:0", OptProfileSelector::kMIN, Dims2{ 778, 64 });
profile->setDimensions("input_mask_1:0", OptProfileSelector::kOPT, Dims2{ 778, 64 });
profile->setDimensions("input_mask_1:0", OptProfileSelector::kMAX, Dims2{ 778, 64 });
profile->setDimensions("input_ids_1:0", OptProfileSelector::kMIN, Dims2{ 779, 64 });
profile->setDimensions("input_ids_1:0", OptProfileSelector::kOPT, Dims2{ 779, 64 });
profile->setDimensions("input_ids_1:0", OptProfileSelector::kMAX, Dims2{ 779, 64 });
preprocessorConfig->addOptimizationProfile(profile);
//int flag = 1 << BuilderFlag.kFP16;
preprocessorConfig->setFlags(1);
// Build the engine
auto engine = builder->buildEngineWithConfig(*network, *preprocessorConfig);
The error message is :
[W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[W] [TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[W] [TRT] Tensor DataType is determined at build time for tensors not marked as input or output.
[F] [TRT] Assertion failed: mg.nodes[mg.regionIndices[outputRegion]].size == mg.nodes[mg.regionIndices[inputRegion]].size
…/builder/cudnnBuilderBlockChooser.cpp:127
Aborting…
[E] [TRT] …/builder/cudnnBuilderBlockChooser.cpp (127) - Assertion Error in buildMemGraph: 0 (mg.nodes[mg.regionIndices[outputRegion]].size == mg.nodes[mg.regionIndices[inputRegion]].size)
dlis_run: /mnt/xuewei/IBert/bertInference.cpp:270: nvinfer1::ICudaEngine* bert::BertInference::APIToModel(int, int, int): Assertion `engine != nullptr’ failed.
Aborted (core dumped)
I used:
tf2onnx.convert --saved-model
converting the tensorflow model to onnx model.
I ran onnx.check. and the check passed.
What can I do next to fix this issue?
Thanks
Environment
TensorRT Version : 7.0.0.11
GPU Type : v100
Nvidia Driver Version :
CUDA Version :10.2
CUDNN Version :7.6
Operating System + Version : 18.04.1-Ubuntu
Python Version (if applicable) :
TensorFlow Version (if applicable) :
PyTorch Version (if applicable) :
Baremetal or Container (if container which image + tag) :