[TensorRT] ERROR: Network must have at least one output

Description

I convert my TensorFlow model to onnx. It has three outputs. However, when I’m trying to build a TensorRt engine, it gives me:

[TensorRT] ERROR: Network must have at least one output

Some people said there is a workaround: network.mark_output(network.get_layer(network.num_layers - 1).get_output(0)). This doesn’t work for me since my network has three outputs.

Environment

TensorRT Version: 7.0.0.11
GPU Type: RTX 2080ti
Nvidia Driver Version: 460.73.01
CUDA Version: 10.0
CUDNN Version: 7.6.4
Operating System + Version: Ubuntu 16.04
Python Version: 3.6.13
TensorFlow Version: 1.15

Relevant Files

My onnx file can be downloaded here.

Steps To Reproduce

I saved my TensorFlow model using:

	frozen_graph = tf.get_default_graph().as_graph_def()
	frozen_graph = tf.graph_util.convert_variables_to_constants(sess=sess, input_graph_def=frozen_graph, output_node_names=outputnodes)
	with tf.gfile.FastGFile(os.path.join(model_dir, 'saved_model.pb'), "wb") as f:
		f.write(frozen_graph.SerializeToString())

Then, I convert the pb file to onnx file using command line:

python -m tf2onnx.convert --input .../saved_model.pb --output .../saved_model.onnx --inputs source:0[-1,720,1296,10] --outputs autoencoder/denoiser/final_stage/relu/relu:0,autoencoder/denoiser/final_stage/relu_1/relu:0,autoencoder/denoiser/final_stage/relu_2/relu:0 --target tensorrt

I build a tensorrt engine using:

def build_engine():
    with trt.Builder(TRT_LOGGER) as builder, builder.create_network(1) as network, trt.OnnxParser(network, TRT_LOGGER) as parser:
        builder.max_workspace_size = rtutils.GiB(1)
        with open(MODEL_FILE, 'rb') as model:
            parser.parse(model.read())
        network.get_input(0).shape = INPUT_SHAPE
        if PRECISION == 'float32':
            pass
        elif PRECISION == 'float16':
            builder.fp16_mode = True
            builder.strict_type_constraints = True
        elif PRECISION == 'int8':
            builder.int8_mode = True
            builder.strict_type_constraints = True
        else:
            raise ValueError("Precision error. Expected: 'float32', 'float16', 'int8'.")
        return builder.build_cuda_engine(network)

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

Hi @fusiyuan2000,

Hope following will help you. Also we recommend you to try on latest TRT version.
Please let us know if you still face the issue.

Thank you.

Hi. Thanks for replying. The check_model.py has no information printed, so I suppose that my model is all good.

Using trtexec partly solved my problem. It successfully gave me the average inference time, which is slightly smaller than the nvprof results of my python script.

However, when I saved the trtexec engine using --saveEngine and run the engine in my python script, pycuda._driver.MemoryError: cuMemHostAlloc failed: out of memory happened.

I can run the inference using TensorFlow, nevertheless. So this is not really a problem for me as I only want to measure the time improvement by tensorrt.

Hi @fusiyuan2000,

Please make sure enough memory is available. This error looks like due to insufficient memory. Please try --workspace option in trtexec to increase the workspace size.

Thank you.