[TensorRT] ERROR: UFFParser: Unsupported number of graph 0

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I am using TRT to create a engine following the https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#import_tf_python.Here is a test case of my codes.I am creating my own model using tensorflow and try to parse it.

CODES:

import tensorflow as tf
import tensorrt as trt
from tensorflow.examples.tutorials.mnist import input_data
import time
import numpy as np

with tf.variable_scope(“scope”):
a = tf.placeholder(dtype=tf.float32, shape=[3, 3], name=“a”)
b = tf.Variable(tf.random_normal([3, 3,], mean=0, stddev=1, name=“b”))
c = tf.matmul(a, b, name=“c”)

with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
o = sess.run(c, feed_dict={a: np.ones([3, 3])})
print(sess.run(b))
print(o)

graphdef = tf.get_default_graph().as_graph_def()
frozen_graph = tf.graph_util.convert_variables_to_constants(sess,
                                                            graphdef,
                                                            ["scope/c"])
frozen_model = tf.graph_util.remove_training_nodes(frozen_graph)
with open("test_uff", "wb") as ofile:
    ofile.write(frozen_model.SerializeToString())

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
model_file = “test_uff”

with trt.Builder(TRT_LOGGER) as builder, builder.create_network() as network, trt.UffParser() as parser:

parser.register_input("scope/a", (3, 3))
parser.register_output("scope/c")
parser.parse(model_file, network)

It report an error like:
[TensorRT] ERROR: UFFParser: Unsupported number of graph 0

I see others solve the register_input problem by fixing the input graph name.But it doesnt work.
Does someone know how to resolve it?

Ubuntun:16.04
TensorRT:5.0.0.10
Cuda:9.0
Cudnn:7.1.2
Tensorflow:1.11.0

2

Hello,

With your repro, I’m getting another error. Can you verify?

import tensorflow as tf
import tensorrt as trt
from tensorflow.examples.tutorials.mnist import input_data
import time
import numpy as np

with tf.variable_scope("scope"):
    a = tf.placeholder(dtype=tf.float32, shape=[3, 3], name="a")
    b = tf.Variable(tf.random_normal([3, 3,], mean=0, stddev=1, name="b"))
    c = tf.matmul(a, b, name="c")

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    o = sess.run(c, feed_dict={a: np.ones([3, 3])})
    print(sess.run(b))
    print(o)

graphdef = tf.get_default_graph().as_graph_def()
frozen_graph = tf.graph_util.convert_variables_to_constants(sess, graphdef,["scope/c"])
frozen_model = tf.graph_util.remove_training_nodes(frozen_graph)
with open("test_uff", "wb") as ofile:
    ofile.write(frozen_model.SerializeToString())

    TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
    model_file = "test_uff"

with trt.Builder(TRT_LOGGER) as builder, builder.create_network() as network, trt.UffParser() as parser:

    parser.register_input("scope/a", (3, 3))
    parser.register_output("scope/c")
    parser.parse(model_file, network)

Getting following error:

root@5134c3cead20:/home/scratch.zhenyih_sw/reproduce.2424914# python test.py
2018-10-22 19:00:44.372710: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
[[-0.00848615 -0.19294259 -1.6195415 ]
 [ 0.23671253 -0.46649984  1.0571886 ]
 [ 0.54299146  1.31197     0.7096284 ]]
[[0.7712178 0.6525276 0.1472755]
 [0.7712178 0.6525276 0.1472755]
 [0.7712178 0.6525276 0.1472755]]
Traceback (most recent call last):
  File "test.py", line 19, in <module>
    frozen_graph = tf.graph_util.convert_variables_to_constants(sess, graphdef,["scope/c"])
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/graph_util_impl.py", line 251, in convert_variables_to_constants
    returned_variables = sess.run(variable_names)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 887, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1033, in _run
    raise RuntimeError('Attempted to use a closed Session.')
RuntimeError: Attempted to use a closed Session.
3

Thanks NVES for reply.The issue you are suffering from is because of bad indent.The “graph” should be in the sess.When I copy the codes from IDE there might be something wrong.

And the issue I was asking is due to “convert-to-uff” problem.I get same name of uff files and when I change the code it didn’t produce a new file.

So basically

[TensorRT] ERROR: UFFParser: Unsupported number of graph 0

may produced by fail to read the uff graph.Hope that helps.

4

ofile.write(frozen_model.SerializeToString()) does not save in the UFF format.