TensorRT 5.0 Converting to and from UFF

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hey all, I’m having a problem converting a pretrained keras model of vgg16 to a .uff file and then converting that to a .trt file for inference on the Xavier. This is the code I’m using for the conversion:

model = ka.vgg16.VGG16(include_top=False, weights='imagenet')

    output_names = [node.op.name for node in model.outputs]

    frozen_graph_filename = 'keras_vgg16_frozen_graph.pb'
    uff_model_name = "keras_vgg16_model.uff"
    sess = k.get_session()

    # freeze graph and remove training nodes
    graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, output_names)
    graph_def = tf.graph_util.remove_training_nodes(graph_def)

    # write frozen graph to file
    with open(frozen_graph_filename, 'wb') as f:
        f.write(graph_def.SerializeToString())
    f.close()

    # convert frozen graph to uff
    uff_model = uff.from_tensorflow_frozen_model(frozen_graph_filename, output_names)
    with open(uff_model_name, 'wb') as f:
        f.write(uff_model)
    f.close()

This appears works fine but when I try to then convert the output .uff file to a trt file I get this error:

ERROR: Parameter check failed at: ../builder/Network.cpp::addInput::406, condition: isValidDims(dims)
ERROR: UFFParser: Failed to parseInput for node input_1
ERROR: UFFParser: Parser error: input_1: Failed to parse node - Invalid Tensor found at node input_1

My code to do the conversion is:

import tensorrt as trt


TRT_LOGGER = trt.Logger(trt.Logger.WARNING)

with trt.Builder(TRT_LOGGER) as builder, builder.create_network() as network, trt.UffParser() as parser:
    parser.register_input("Placeholder", [3, 224, 224], order=trt.UffInputOrder.NCHW)
    parser.register_output("output")

    parser.parse("keras_vgg16_model.uff", network, weights_type=trt.DataType.FLOAT)

I feel like I’m running around in cirlces trying to get this work so any help would be appreciated. For understanding I’m doing this all on a host machine as PoC before hopefully trying it out on the Xavier in c++

Thanks

2

Hi Sam,

The input and output names in the parser.register_input() and parser.register_output() need to match those from the Keras model.

To get the correct names, I ran these lines:

>>> input_names = [node.op.name for node in model.inputs]
>>> output_names = [node.op.name for node in model.outputs]
>>> print(input_names)
['input_1']
>>> print(output_names)
['block5_pool/MaxPool']

Then I put these names into the code for creating the TRT Engines:

import tensorrt as trt

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)

with trt.Builder(TRT_LOGGER) as builder, builder.create_network() as network, trt.UffParser() as parser:
    parser.register_input("input_1", [3, 224, 224], order=trt.UffInputOrder.NCHW)
    parser.register_output("block5_pool/MaxPool")
    parser.parse("keras_vgg16_model.uff", network, weights_type=trt.DataType.FLOAT)

Thanks,
NVIDIA Enterprise Support

3

Thank you, didn’t realize I had the wrong names. You saved me quite the debugging hassle.

4

No problem!