We just upgraded to JP 4.6 and TRT 8. We have been running ONNX models on TRT 7 successfully. We were allocating 2g for our engine and noticed on TRT 8 we had insufficient memory for a resources and reducing the memory to 1g resolved the issue and we are able to build our engine and run the inference. We also were encapsulating the i in a unique pointer with a custom deleter calling the class destroy method, which seems to have been deprecated in TRT8 and according to the release notes the class destructors are no longer protected, and are now public, so we just allow the unique pointer to go out of scope without a custom deleter. The problem that we see is when the pointer goes out of scope we see :
pure virtual method called
terminate called without an active exception
Which seem to indicate a joinable thread is going out of scope. We see this behavior only when we switch to TRT 8. I found this thread
Is this now an issue with TRT8 or related to workspace size?
Thank you so much for the response. We are discussing this in our team and will soon get back to you on this. We need to make sure we can provide you with what we have AFA our ONNX model and we are doing a bit more due diligence to make sure this is not on our side, although it all runs on TRT7. Again thanks for the quick response and as soon as I have something I will post it here.
Sorry for the delay to get back to this. We did some more investigation and this is what we have found. It appears that the problem is deleting nvinfer1::IExecutionContext. If this is wrapped in a unique pointer which is then deleted when pointer goes out of context , we see the “pure virtual method” error. If we use new() or define a dummy deleter that does not delete this object , it exits normally with no errors. Also , has nvinfer1::IBuilder::buildEngineWithConfig been deprecated as well? TensorRT: nvinfer1::IBuilder Class Reference
Hi Sorry again for the delay response. I am trying to arrange here in our team as to how to share an application that shows the problem. We have found a work around however. We have an execution context as a class member and it is a unique_ptr. We also have a vector of this class declared and a std::vector of unique_ptr to the class. However if we create a vector of regular pointers and a release each element in a destructor we get a clean exit !