Plugin V3 Error when using quantized weights as plugin inputs

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I try to use Plugin V3 with quantized input and weights. But, the following error was encountered:

Error[2]: [qdqGraphOptimizer.cpp::matchQuantizedConstantPluginOrDQ::4004] Error Code 2: Internal Error (L__self___conv1.weight: Quantized constant is only allowed before DQ or PLUGIN_V2 node)

It seems that the quantized constant should be allowed before the PLUGIN_V3 node, but it was missed when PLUGIN_V3 was updated in TensorRT 10.0.

There is no issue when changing Plugin V3 to Plugin V2. It needs to add Plugin V3 to list of nodes that can receive quantized weights as input.

Environment

TensorRT Version: TensorRT 10.0.1
GPU Type: RTX 4000 Ada
Nvidia Driver Version: 535.104.12
CUDA Version: 12.4
CUDNN Version: n/a
Operating System + Version: ubuntu 22.04
Python Version (if applicable): n/a
TensorFlow Version (if applicable): n/a
PyTorch Version (if applicable): n/a
Baremetal or Container (if container which image + tag): n/a

2

Can you please share your model with us?

3

It was fixed in TRT 10.1 version.

4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.