Hi, instead of pytorch-quantization, is it possible to use PyTorch’s own quantization libraries, export to ONNX->TensorRT? I’ve read somewhere that TensorRT does not support QDQ params with scales != 0? Is this still the case since PyTorch’s own quantization libraries will produce QDQ with scales!=0?
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:
- validating your model with the below snippet
filename = yourONNXmodel
model = onnx.load(filename)
2) Try running your model with trtexec command.
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging