INMSLayer: CUDA graph invalidation (DevicetoShapeHostCopy)

AI & Data Science Deep Learning (Training & Inference) TensorRT
6

Hello, did you find any solution to this, i am experiencing the same issue using the onnx-tensorrt parser with this yolov8 repository:

The model “blocks” all other parallel processing during NMS execution because of the Memory Copy.

Output-tensor-meta Access RAW model output with batch dimension
Output-tensor-meta Access RAW model output with batch dimension
Poor performance due to INMSLayers followup nms_layer_output(1)[DevicetoShapeHostCopy] and trainstation