How Tensorrt's horizontal layer fusion output data works?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

When i build a tensorrt engine from inceptionResnetv2 onnx model in int8 precision;There is so called horizontal layer fusion in convolutions with same input tensor and weight kernel size;(The red rectangles below)

1701243628204_5922B2E2-2D69-4358-AD95-5750A6DA6996
1701243628204_5922B2E2-2D69-4358-AD95-5750A6DA6996614×755 35 KB

1701243588800_E86ACE2F-4E1B-4311-9F1A-6EE5417287A9
1701243588800_E86ACE2F-4E1B-4311-9F1A-6EE5417287A91059×1081 69.4 KB

It seems that fused convolution output as [64, 208, 35, 35], and deliver to the next convolutions directly; So these convolutions’ input channels were changed to 208 ? or there is data offset in the convolution input data?

Environment

TensorRT Version: 8.2
GPU Type: A10
CUDA Version: 11.2

2

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!