Hi,
I looked into this issue, the problem here is that you’re setting the bindingDimension for bindingIndex = 0 for both iterations, however, each profile get’s it’s own version of each binding. You can see this by printing out all of the binding names. So for Profile 0, the inputBindingIndex is 0, but for Profile 1, the inputBindingIndex is 9, that’s why you’re getting that not allInputDimensions are specified, because you didn’t specify them for bindingIndex 9.
NbOptimizationProfiles: 2
NbEngineBindings: 18
Binding Names:
---
Binding 0: fts_input_images:0
Binding 1: fts_output_prior_layer_2:0
Binding 2: fts_output_prior_layer_3:0
Binding 3: fts_output_prior_layer_1:0
Binding 4: fts_output_prior_layer_0:0
Binding 5: fts_output_softmax_layer_0:0
Binding 6: fts_output_softmax_layer_1:0
Binding 7: fts_output_softmax_layer_3:0
Binding 8: fts_output_softmax_layer_2:0
Binding 9: fts_input_images:0 [profile 1]
Binding 10: fts_output_prior_layer_2:0 [profile 1]
Binding 11: fts_output_prior_layer_3:0 [profile 1]
Binding 12: fts_output_prior_layer_1:0 [profile 1]
Binding 13: fts_output_prior_layer_0:0 [profile 1]
Binding 14: fts_output_softmax_layer_0:0 [profile 1]
Binding 15: fts_output_softmax_layer_1:0 [profile 1]
Binding 16: fts_output_softmax_layer_3:0 [profile 1]
Binding 17: fts_output_softmax_layer_2:0 [profile 1]
Iteration: 0
Profile before setting: 0
[01/14/2020-22:18:56] [W] [TRT] Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles
Profile after setting: 0
[fts_input_images:0] Input Binding Index for Profile0: 0
before setBindingDimensions -- allInputDimensionsSpecified: 0
before setBindingDimensions -- getBindingDimensions: (1, -1, -1, 1)
after setBindingDimensions -- allInputDimensionsSpecified: 1
after setBindingDimensions -- getBindingDimensions: (1, 100, 100, 1)
Iteration: 1
[01/14/2020-22:18:57] [W] [TRT] Could not set default profile 0 for execution context. Profile index must be set explicitly.
Profile before setting: -1
Profile after setting: 1
[fts_input_images:0] Input Binding Index for Profile1: 9
before setBindingDimensions -- allInputDimensionsSpecified: 0
before setBindingDimensions -- getBindingDimensions: (1, -1, -1, 1)
after setBindingDimensions -- allInputDimensionsSpecified: 1
after setBindingDimensions -- getBindingDimensions: (1, 100, 100, 1)
You can the binding index for a given profile N > 0 by adding “[profile N]” to the name of the binding:
int profile0_binding_index = engine->getBindingIndex("fts_input_images:0");
int profile1_binding_index = engine->getBindingIndex("fts_input_images:0 [profile 1]");
This was missed in the documentation and will be added in the future, I just discovered this by looking through all of the bindings, because I noticed this line in the documentation (TensorRT: nvinfer1::ICudaEngine Class Reference): “If the engine has been built for K profiles, the first getNbBindings() / K bindings are used by profile number 0, the following getNbBindings() / K bindings are used by profile number 1 etc.”
std::cout << "NbOptimizationProfiles: "<< engine->getNbOptimizationProfiles() << std::endl;
std::cout << "NbEngineBindings: " << engine->getNbBindings() << std::endl;
std::cout << "[" << input_name << "] Input Binding Index: " << engine->getBindingIndex(input_name.c_str()) << std::endl;
for (int binding=0; binding < engine->getNbBindings(); binding++) {
std::cout << "Binding " << binding << ": " << engine->getBindingName(binding) << std::endl;
}