I am new to TensorRT, but I encounter this problem with TensorRT 7.0
(my rag: cuDNN 7.6.5/CUDA 10.2/Windows 10 x64, with Xeon v4 CPU and several Titan V GPUs).
In my case: the size of the input tensor of the ONNX model is 256(H)*1(W)*6©
Since in TensorRT 7.x, only dynamic shape mode is supported for ONNX networks, so I added an input layer according to the user guider with dynamic tensor definition:
int BatchSize=256; network->addInput("foo", DataType::kFLOAT, Dims4(BatchSize, 6, -1, -1));
And adding optimized profile:
Dims dim; dim.d=BatchSize; dim.d=6; ... profile->setDimensions("foo", OptProfileSelector::kMIN, dim); ...
But whenever the engine is built, and at runtime it is called, the input dimension’s index batchsize is always 1, therefore the profile dimension does not match it, and the program will return the following debug information:
Parameter check failed at: engine>cpp::nvinfer1::rt:ExecutionContext::setBindingDimensions::948, condition:
The only case it worked is that, at runtime, I have to force to set the batch size to 1 and thus call the engine context to compute the input one at a time, and of cause the performance will be unacceptable.
Any idea on how to solve this problem?