Description
We have a model with two tensors inputs.
Each tensor input has its own optional dimensions sizes.
-
Tensor#0 has two options:
Dimenstions#0 - {N = 1, C = 1, H = 320, W = 640}
Dimenstions#1 - {N =1, C = 1, H = 256, W =512}
Each element type is fp 32 bits -
Tensor#1 has three options:
Dimenstions#0 - { N =1, C = 1024}
Dimenstions#1 - { N =1, C = 512}
Dimenstions#2 - { N =1, C = 800}
Each element type is Int 32 bits
I think that each tensor shall has its own profile range:
-
Profile#0:
Min - Dims4(1, 1, 256, 512)
Max - Dims4(1, 1, 320, 640)
Opt - Dims4(1, 1,300, 568) //Our most commonly used runtime values -
Profile#1:
Min - Dims4(1, 512)
Max - Dims4(1, 1024)
Opt - Dims4(1, 768) //Our most commonly used runtime values
I’m adding two profiles to my builder config using the method addOptimizationProfile not before I’m building them using the setDimensions method to declare their kMIN, kOPT and kMAX.
I understand that I shall set the actual dimensions of each tensor input using the method setBindingDimensions of the context before I activate its enqueueV2 method.
These are my questions:
I cannot understand the relations between the tensors and profiles?
Shall I move between the two profiles in order to set the two tensors inputs actual dimensions? If so, how shall I do it?
Or
Since in setOptimizationProfile API description it is mentioned
“Select an optimization profile for the current context…” ,
shall I understand that one profile shall reply all tensor inputs dimensions??
Can you provide a code example how to select profile, set the actual tensor input dimension and then activate the inference process?
Environment
TensorRT Version: 7.2.3.4
GPU Type: Quadro M2000M
Nvidia Driver Version: R451.77
CUDA Version: 11.0
CUDNN Version: 8.0.1
Operating System + Version: Microsoft WIndows 10 Enterprise 2016
Python Version (if applicable): 3.6.8
TensorFlow Version (if applicable): NA
PyTorch Version (if applicable): NA
Baremetal or Container (if container which image + tag): Baremetal
Relevant Files
NA
Steps To Reproduce
Suppose I want Dimenstions#0 for tensor input#0 and Dimenstions#2 for tensor input#1,shall I do it in this way (pseudo code)?
context->setOptimizationProfile(Profile#0);
context->setBindingDimensions(Tensor#0, Dimenstions#0);context->setOptimizationProfile(Profile#1);
context->setBindingDimensions(Tensor#1, Dimenstions#2);if true == context->allInputDimensionsSpecified()
context->enqueueV2((void**)&memory, *stream, nullptr))
If not, please provide an example
Thanks,