Quantization of D-FINE in tensorrt 10.8 fails

AI & Data Science Deep Learning (Training & Inference) TensorRT
, ,
1

Description

A clear and concise description of the bug or issue.

Environment

TensorRT Version: 10.8
GPU Type: L4
Nvidia Driver Version:
CUDA Version: 12.2
CUDNN Version:
Operating System + Version: Ubuntu 22.04
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Steps To Reproduce

convert a D-FINE ONNX model to a quantized int8 engine

/usr/src/tensorrt/bin/trtexec --onnx=d_fine.onnx --saveEngine=dfine_int8_b1.engine --calib=calib.cache --int8

fails with:

 [TRT] [E] IBuilder::buildSerializedNetwork: Error Code 10: Internal Error (Could not find any implementation for node /model/encoder/fpn_blocks.1/cv4/conv_bn_fused/Conv + PWN(PWN(/model/encoder/fpn_blocks.1/cv4/act/Sigmoid), PWN(/model/encoder/fpn_blocks.1/cv4/act/Mul)).)

I can provide the calibration file & the onnx for this, if needed. I have successfully run this conversion with tensorrt 8.6, so I think there is nothing particular about the model.

2

anything on this? I tried downgrading to tensorrt 10.3, but this also doesnt work , causing the exact same error. So with DS 7.1 we cant use D-FINE it seems?

4

Hi @fdarvas ,
Considering the known issues and their fixes, can you pls confirm if you have already tried

  • Using FP32 scales for Q/DQ operations.
  • Enabling StronglyTyped mode.
  • Adjusting builder optimization levels.
    Please let me know if this helps, in case if the issue persist, pls share your model and repro scripts with us.

Thanks

6

can you point me to a description of the known issues?

when enabling “StronglyTyped” mode, then I no longer can set trt.BuilderFlag.INT8, which defeats the purpose? I want to create specifically an int8 engine. Essentially I just want recreate the same engine I could build with tensorrt 8.6.

How do I enable FP32 scales for Q/DQ operations?

I did adjust the builder optimization levels, basically I tried levels 4 and 5, but this did not help.
Where can I share the model & script? Or can I just attach to a post here?