I got this error when I use TensorRT inference Resnet50 with dynamic batch.
I set multiple dynamic batch profiles : [[1, 1, 1], [2, 2, 2], [4, 4, 4], [6, 6, 6], [8, 8, 8], [10, 10, 10], [16, 16, 16], [32, 32, 32], [1, 64, 128]] , each is [min, opt, max] batch pairs for dynamic shape profiles. Exept the last profile, others are only optimized for a specific batch.
When I infer dynamic batch input, I will switch between different profiles. However, when I switch to the last profile from other profiles at the second time, I get this error and thus switch profile failed. I can switch between profiles except the last without fail.
A clear and concise description of the bug or issue.
TensorRT Version: both 7.0 and 8.0
GPU Type: v100
Nvidia Driver Version: cuda11.0 driver
CUDA Version: 10.2
CUDNN Version: 8.2
Operating System + Version: ubuntu18.04
Python Version (if applicable): 3.7.5
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):
Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)
- Exact steps/commands to build your repro
- Exact steps/commands to run your repro
- Full traceback of errors encountered