We are just migrating our system from TensorRT5 to TensorRT7, to get the benefits of dynamic shapes.
Then we just realized the following statement in the documentation:
If the engine supports dynamic shapes, each execution context in concurrent use must use a separate optimization profile. (nvinfer1::IExecutionContext Class Reference)
The statement leads to the fact, that we have to know the number of IExecutionContexts at ICudaEngine creation time. So no real dynamic scaling can be done with IExecutionContext at runtime in TensorRT7. So this is some kind of feature loss, I think…
So my question are :
Are you planing to (re)support this feature in the near future?
What is the reason to force unique usage of OptimizationProfiles?
GPU Type: All
Nvidia Driver Version: 450.xx
CUDA Version: 10.2
Operating System + Version: Ubuntu 18.04