TypeError: build_cuda_engine(): incompatible function arguments

I’m trying to make an introduction to tensorrt and my first goal is to import an ONNX model and save the engine. However I ran into the below problem:

File "onnx2_tensorrt.py", line 13, in <module>
    with builder.create_builder_config() as config, builder.build_cuda_engine(network,config) as engine:
TypeError: build_cuda_engine(): incompatible function arguments. The following argument types are supported:
    1. (self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition) -> tensorrt.tensorrt.ICudaEngine

Invoked with: <tensorrt.tensorrt.Builder object at 0x7f79ecd90bc8>, <tensorrt.tensorrt.INetworkDefinition object at 0x7f79ecd90c00>, <tensorrt.tensorrt.IBuilderConfig object at 0x7f79ecdb37a0>

My code is like:

import tensorrt as trt

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
model_path = 'export.onnx'
max_batch_size = 32
builder = trt.Builder(TRT_LOGGER)
with builder.create_network() as network, trt.OnnxParser(network, TRT_LOGGER) as parser:
    with open(model_path, 'rb') as model:
        parser.parse(model.read())

builder.max_batch_size = max_batch_size
builder.max_workspace_size = 1 << 50 # This determines the amount of memory available to the builder when building an optimized engine and should generally be set as high as possible.
with builder.create_builder_config() as config, builder.build_cuda_engine(network,config) as engine:
    with open('sample.engine', 'wb') as f:
        f.write(engine.serialize())

What’s the point I’m missing?

BTW, my system is
-Ubuntu 18.04
-CUDA 10.0 /CuDNN 7.4.2
-TensorRT 6.0
-Python 3.6.8
-GTX1050 Max-Q

I changed the code to

import tensorrt as trt

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
model_path = 'export.onnx'
max_batch_size = 32
builder = trt.Builder(TRT_LOGGER)
with builder.create_network() as network, trt.OnnxParser(network, TRT_LOGGER) as parser:
    with open(model_path, 'rb') as model:
        parser.parse(model.read())

builder.max_batch_size = max_batch_size
builder.max_workspace_size = 1 << 1000000 # This determines the amount of memory available to the builder when building an optimized engine and should generally be set as high as possible.
with builder.create_builder_config() as config, builder.build_cuda_engine(network=network) as engine:
    with open('/home/orcun/PyProjects/yolov3_onnx/sample.engine', 'wb') as f:
        f.write(engine.serialize())

and received bunch of random number as output. But nothing is saved.

...490096333728093987864424739961214369146538584913869616680734636354746057521140975088754537092838323054266272763418957019571498048132599127985829099404369406911488277389085644444576508361767996760585107705827172005558682862431655730624490530379257204517931805139644085985468776476258649568643161005510957517689374385985897170817517572370376739307861090839245647948933289876546907767674649469447266024935617675083187281063140584277923561591655499355261518392995848751915058096695781494437362921595282004690039241689685546086872285924168521145763878635415977812062437373089686577947815086211989963465539489146135836643244658952124903528102811185626429125415058241996246026272186486799030748030005647179560890126783772373408030316632051171051132006312417933120882546436762314897792026174008534292793911394374990859090109317708540397986337305723121804622819478163489902541927602500524886436108940743049062144510115133014081687467186866272139054564990567968648163397464618889527226553324686635145258688765752358232285666531684459259203433112319924000805211701071343546592971495675078532803664047289361339755125629037531796760910552296424123669791572530120259366033073362600495785009452832231209771651949930078381172819232959576492598854637123171148720654845427865171709018704436470288331142807637404470838721862775946702594333232370930293593888820015505512186840199087378463823648294836014728724026926173484210843819335665118804267738590388961723310435946436176254215109329949293796628246696029621753349348315534599715106075238114432824357958756046801829936663346927498252184113139026154190762101570232777464912829167554131237002272757628421649388240673741858577571237989895175986725568600702493511694682921171113852563711376533487112022676879462444600041932855783251617241085645011001036578193204466256314580375135866175958334510775841705204022698031471290583891295636747619296417826487441672855818541664120161336264596269328635225914285144229375864163429343101487229024976509832906753689215241457079293625964237491439317020877311008625357775069148569092075651782488478598420407064596334261847462588170731310996021657706771510081099833511546595845746196783679831720727072683207591757100428209192420228872667749052301871236104888403162747109376

Hi orcdnz,

Can you try something like this instead?

This is taken from the tensorrt samples - /usr/src/tensorrt/samples/python/

def build_engine():
        """Takes an ONNX file and creates a TensorRT engine to run inference with"""
        with trt.Builder(TRT_LOGGER) as builder, builder.create_network() as network, trt.OnnxParser(network, TRT_LOGGER) as parser:
            builder.max_workspace_size = 1 << 28 # 256MiB
            builder.max_batch_size = 1
            # Parse model file
            if not os.path.exists(onnx_file_path):
                print('ONNX file {} not found, please run yolov3_to_onnx.py first to generate it.'.format(onnx_file_path))
                sys.exit(0)
            print('Loading ONNX file from path {}...'.format(onnx_file_path))
            with open(onnx_file_path, 'rb') as model:
                print('Beginning ONNX file parsing')
                parser.parse(model.read())
            print('Completed parsing of ONNX file')
            print('Building an engine from file {}; this may take a while...'.format(onnx_file_path))
            engine = builder.build_cuda_engine(network)
            print("Completed creating Engine")
            with open(engine_file_path, "wb") as f:
                f.write(engine.serialize())
            return engine

You can get an overview of more samples at this page: https://docs.nvidia.com/deeplearning/sdk/tensorrt-sample-support-guide/index.html#introductory_parser_samples

Thanks,
NVIDIA Enterprise Support

1 Like

I just tried your method with my model and got

Loading ONNX file from path /home/orcun/PyProjects/yolov3_onnx/my_yolo.onnx...
Beginning ONNX file parsing
Completed parsing of ONNX file
Building an engine from file /home/orcun/PyProjects/yolov3_onnx/my_yolo.onnx; this may take a while...
[TensorRT] ERROR: Network must have at least one output
Completed creating Engine
Traceback (most recent call last):
  File "onnx2_tensorrt.py", line 29, in <module>
    build_engine('/home/orcun/PyProjects/yolov3_onnx/my_yolo.onnx', '/home/orcun/PyProjects/yolov3_onnx/sample.engine')
  File "onnx2_tensorrt.py", line 26, in build_engine
    f.write(engine.serialize())
AttributeError: 'NoneType' object has no attribute 'serialize'

Why doesnt it find the outputs, which causes it to fail the writing part? How to solve this?

Hi,

Your issue may vary depending on the model, but adding this may solve your problem.

...
network.mark_output(network.get_layer(network.num_layers - 1).get_output(0))
engine = builder.build_cuda_engine(network)
...

See this thread for more details: https://devtalk.nvidia.com/default/topic/1045064/tensorrt/onnx-and-tensorrt-error-network-must-have-at-least-one-output/

1 Like

At least created the engine now. Still dont know what will happen in the inference part but thank you!

in some cases add:

EXPLICIT_BATCH = 1 << (int)(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH)
builder.create_network(EXPLICIT_BATCH)