There are some custom operators in my onnx model, and I implemented their plugins on tensorrt. When I use tensorrt to read this onnx model and run it multiple times for the same input, different engines will be given. Some engines give correct inference results, while some engines give incorrect inference results.
This onnx file is exported with pytorch.
In this case, which aspect of the problem do I need to locate first? Thanks a lot.
TensorRT Version: 8.2 GA GPU Type: rtx 3060 Nvidia Driver Version: 126.96.36.1992 CUDA Version: 11.4 CUDNN Version: 11.4 Operating System + Version: Windows 11 PyTorch Version (if applicable): 1.9.0+cu111
I have an onnx model with a total of three operators, among which operator 1 and operator 2 are my custom plug-ins. When parsing in tensorrt, the values of output1 and output3 are both correct, but output2 is incorrect. But I copied the output pointer from the video memory with cudaMemcpy in enqueue() in Operator 2, and the printed value is correct.
How can I debug this problem? Thanks.