Trtexec convert dla model fail

[parkerpang1997]

Jetson Agx Orin 32G
JetPack 6.0
TensorRT 8.6.2.2

I use trtexec tool to convert onnx model to dla engine. Here is the input of my model:

The command is

/usr/src/tensorrt/bin/trtexec --onnx=model_fp_input/at_simplified_noqdq.onnx --calib=model_fp_input/qat_simplified_precision_config_calib.cache --useDLACore=0 --int8 --fp16 --saveEngine=model_fp_input/qat_simplified_standalone.dla --precisionConstraints=prefer --layerPrecisions=/normalize_input/Conv:fp16 --inputIOFormats=fp16:dla_hwc4 --outputIOFormats=fp16:chw16 --buildDLAStandalone

It returns fail with info as follow:

I want to know the reason why it failed and how should I modified my model if I want to using fp16:dla_hwc4 as model input since I can only offer fp16 and nhw4 data in my project and I don’t want to use preprocessing outside the model. Besides, uint8 and nhw4 input data is also available, but I think it can’t be passed to dla directly.

I have tried to remove the red box and using int8:dla_hwc4 input and trtexec worked successfully. But this model was not suitable for my project.

[AastaLLL]

Hi,

DLA doesn’t support quantization layers. Do you need it in the model?
Usually, users adopt the PTQ strategy on Jetson rather than QAT.

Below is the DLA support matrix for your reference:

Developer Guide :: NVIDIA Deep Learning TensorRT Documentation

Thanks.

[parkerpang1997]

Hi,

The Onnx model with QDQ node is translalted to PTQ model with calibration cache using qdq_translator tools.

Here is the attributes of the first convolution layer and input shape is 8x3x352x640 (NCHW). I want to use fp16:dla_hwc4 input format, pass input to first fp16 conv layer and then quantize to int8. Thus the input for dla engine can be 8x352x640x4 (fp16).

I think my model and the config meet the input format requirements, but conversion failed.
Could you help me analyze the reasons for this failure?

[parkerpang1997]

Is the group for the first convolutional layer required to be 1?

[AastaLLL]

Hi,

Does the model work when adding the GPU fallback?
Based on the log, it requires a reformat layer to convert the data type which usually is done by GPU.

Thanks.