context->getOptimizationProfile() returns ( -1 )
engine->.getNbOptimizationProfiles() returns ( 1 )
Error thrown when calling enqueueV2():
[executionContext.cpp::setBindingDimensions::924] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::setBindingDimensions::924, condition: mOptimizationProfile >= 0 && mOptimizationProfile < mEngine.getNbOptimizationProfiles()
TensorRt engine was generated using trtexec using an onnx file, having the min/opt/max the same exact shapes . (&&&& PASSED)
Using trtexec to load the engine works perfectly [ trtexec --loadEngine model.engine --fp16 ] (&&&& PASSED)
[graphContext.h::MyelinGraphContext::24] Error Code 1: Myelin (Compiled against cuDNN 126.96.36.199 but running against cuDNN 188.8.131.52.)
TensorRT Version: 8.2.1
Nvidia Driver Version: 470.103.01
CUDA Version: 11.4
CUDNN Version: 8
Operating System + Version: Ubuntu 18.04.1
PyTorch Version (if applicable): 1.9
Could you please try the latest TensorRT version 8.5.2 (if possible, increase some workspace) and let us know if the problem persists?
Please share with us a issue repro ONNX model and script/steps to try from our end.
Yes, I can try to install TensorRT version 8.5.2 and try but I do not think it is a version-related. With TensorRT version 8.2.1 I can successfully load the engine and run ~trtexec~ with inference mode.
for (int32_t s = 0; s < inference.streams; ++s)
auto ec = iEnv.engine->createExecutionContext();
std::cout << "\t&&&&&&&&&& getOptimizationProfile "
<< ec->getOptimizationProfile() << "\n\n";
- [ ec->getOptimizationProfile() ] returns 0 which is the correct profile index. The code for loading the TensorRT engine, deserialization and creating context is very similar.
- Only when I run my application I am getting a bad context with having getOptimizationProfile() returning [ -1 ].
- Does including torch libraries [ for example; #include <torch/torch.h> ] affects the TensorRT execution? And does it create a new context and that a creates some kind of inconsistency?
We don’t think it was due to that.
Could you please make sure for context you’re setting the input shape before cuda memcopy.