I was converting my onnx file which contains custom operation to tensorRT engine using trtexec. It seems like even though there is some bugs in my enqueue function trtexec generated the .engine file. When I use this for inference using TensorRT python API, the bug in my enqueue function appears and complains about cuDA illegal memory access. How can I debug my custom shared plugin library (to see where exactly the illegal access occurs) if I am running inference through python API?
Environment
TensorRT Version: 8.2 GPU Type: RTX 3080 Laptop Nvidia Driver Version: 530.41.03 CUDA Version: 12.1 CUDNN Version: Operating System + Version: Ubuntu 20.04 Python Version (if applicable): 3.8 TensorFlow Version (if applicable): NA PyTorch Version (if applicable): 1.12 Baremetal or Container (if container which image + tag): Container - nvcr.io/nvidia/pytorch:22.02-py3
Hi,
Please refer to below links related custom plugin implementation and sample:
While IPluginV2 and IPluginV2Ext interfaces are still supported for backward compatibility with TensorRT 5.1 and 6.0.x respectively, however, we recommend that you write new plugins or refactor existing ones to target the IPluginV2DynamicExt or IPluginV2IOExt interfaces instead.
Thank you for the quick reply. I wrote my Plugin already based on IPluginV2DynamicExt using these 2 links, which were mentioned in many posts. I followed the steps mentioned in
to create a standalone shared library of the TensorRT Custom Plugin and linked it to trtexec using --plugin flag during conversion. During inference I load it in python using dll open command. The problem is I am not able to test my Enqueue function whether it is producing the expected result. Is there some way to test my plugin ?
We recommend you to please try on the latest TensorRT version 8.6 (nvcr.io/nvidia/tensorrt:23.06-py3), if you still face the same issue, please share with us the minimal issue repro model, scripts, and complete error logs for better debugging?