Tensorrt ONNX Inference result mismatch with tensorflow 2.0 Inference

Hi,
I have converted the Tensorflow 2.0 model to onnx model using onnxmltools.
onnx_model = onnxmltools.convert_keras(loaded_model)
onnxmltools.utils.save_model(onnx_model, ModelSrcFile)

onnx model opset is 11

Tried to create the TensorRT engine using the following code
auto builder = createInferBuilder(gLogger);
const auto explicitBatch = 1U << static_cast<uint32_t> NetworkDefinitionCreationFlag::kEXPLICIT_BATCH);
auto network = builder->createNetworkV2(explicitBatch);

auto config = builder->createBuilderConfig();

auto parser = nvonnxparser::createParser(*network, gLogger);




std::string onnxFileName = ModelPath + "model.onnx";

int verbosity = 1;

auto parsed = parser->parseFromFile(
	onnxFileName.c_str(), verbosity);

builder->setMaxBatchSize(ibatchsize);

config->setMaxWorkspaceSize(MAX_WORKSPACE);



network->markOutput(*network->getOutput(0));



ICudaEngine* engine_ToSave = nullptr;
engine_ToSave = builder->buildEngineWithConfig(*network, *config);

if (engine_ToSave == nullptr)
{
	if (network != nullptr)
		network->destroy();

	if (builder != nullptr)
		builder->destroy();

	return false;
}

assert(network->getNbInputs() == 1);
nvinfer1::Dims mInputDims = network->getInput(0)->getDimensions();

assert(mInputDims.nbDims == 4);

assert(network->getNbOutputs() == 1);
nvinfer1::Dims mOutputDims = network->getOutput(0)->getDimensions();
assert(mOutputDims.nbDims == 2);

nvinfer1::IHostMemory* serMem = nullptr;
serMem = engine_ToSave->serialize();

if (serMem == nullptr)
{
	if (network != nullptr)
		network->destroy();

	if (builder != nullptr)
		builder->destroy();

	if (engine_ToSave != nullptr)
		engine_ToSave->destroy();

	return false;
}
std::string tensorRTEngineFile1 = ModelPath + "model.engine";
std::ofstream outfile(tensorRTEngineFile1, std::ios::out | std::ios::binary);
outfile.write(reinterpret_cast<const char*>(serMem->data()), serMem->size());
outfile.close();

Tensorrt Engine creation failed with the current ONNX model. The batch size in input and output layer in ONNX model is “N”. I have changed the batch size in ONNX model to respective batch size. With that change I am able to create the TensorRT engine. Once I did inference with the TensorRT engine, there is a mismatch with the TensorRT inference. The system has the following environment

OS: Windows 10
Framework: Tensorflow 2.0 (with gpu support)
NVIDIA Driver version: 441.41
TensorRT 7.0
CUDA 10.0

Hi,

Can you try using the optimization profiles? Please refer below link for more details:
https://docs.nvidia.com/deeplearning/sdk/tensorrt-archived/tensorrt-700/tensorrt-developer-guide/index.html#opt_profiles

Also, please share the model file and sample repro script so we can help better?

Thanks