Now I can change dynamic batch size in inference for TensorRT.
Batch size can be varied between min to max according to optimization profile.
But the issue is at plugin.
Even though, I use IPluginV2DynamicExt. Input/Output dimension are fixed.
For example, plugin input dimension is batchsize(lets say 10) x 20 x 40.
batchsize is fixed as 10 in plugin creation time.
When I change batch size at input of network dynamically (i.e. every iteration of inference, batch size is changed), the plugin has fixed input size as 10x20x40 and fixed output size, always 10.
It has never changed with different batch size.
Is it possible to change the plugin’s batch size dynamically?
I have this error when I set 1 to min. input:0: kMIN dimensions in profile 0 are [1,24,94,3] but input has static dimensions [10,24,94,3]. Your test didn’t have this error?
I don’t understand. What to adjust? I set onnx model dimensions as -1,24,94,3. Then your tensorrt does not accept -1. I have error. I have already discussed in earlier discussion. Your tensorrt doesn’t accept -1 input. To be dynamic, if you don’t accept -1, what should I put? I can’t find now. In earlier discussion, dimension needs to be positive. Positive dimension is fixed dimension.