Why is IBuilder saying the engine has more output tensors?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I have my model successfully built into an TRT engine. However, when comparing the INetworkDefinition with IBuilder logs, I found something interesting:

What INetworkDefinition has logged:

Network Inputs: 2
Network input 0: input_2:0
Network input 1: input_1:0
Network Outputs: 1
Network output 0: output/concat:0

But the IBuilder has logged:

Detected 2 inputs and 3 output network tensors.

Can anyone explain why there is difference in # of outputs?

Environment

TensorRT Version: 7.0.0
GPU Type: 2080Ti
Nvidia Driver Version: 440
CUDA Version: 10.2
CUDNN Version:
Operating System + Version: Ubuntu 16.04
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered
2

Hi @golden0080gba
Apologies for delayed response
It seems concat layer is the output of network, we might remove concat layer and write previous layer’s output directly into network output. Then there will be more than one output tensor .
However if you can provide your model, we can help you better.
Thanks!

3

Thank you @AakankshaS, I think the explanation makes sense. Unfortunately it’s proprietary model and with some experiments, the model during inference outputs correct number of tensors.