TensorRT issues

wuxiekeji · March 26, 2018, 12:30am

Hi all,

I’m getting started with TensorRT 3.0 here and having a couple of issues. This is all happening on an i5 + NVIDIA GPU + CUDA 9.0 + Tensorflow 1.6 + TensorRT 3.0 using the Python TensorRT API with the intent to deploy to TX2 later (I wish there was a Python API for TX2 …)

So I have a model I’m trying to convert to TensorRT and I get the following error:

input: "Squeeze_2"
attr {
  key: "T"
  value {
    type: DT_INT64
  }
}
attr {
  key: "_output_shapes"
  value {
    list {
      shape {
        dim {
          size: 1
        }
        dim {
          size: -1
        }
        dim {
          size: -1
        }
      }
    }
  }
}

Traceback (most recent call last):
  File "./deeplab-tf2trt.py", line 23, in <module>
    uff_model = uff.from_tensorflow_frozen_model(GRAPH_FILENAME, ["SemanticPredictions"])
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/conversion_helpers.py", line 103, in from_tensorflow_frozen_model
    return from_tensorflow(graphdef, output_nodes, **kwargs)
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/conversion_helpers.py", line 75, in from_tensorflow
    name="main")
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py", line 64, in convert_tf2uff_graph
    uff_graph, input_replacements)
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py", line 51, in convert_tf2uff_node
    op, name, tf_node, inputs, uff_graph, tf_nodes=tf_nodes)
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py", line 28, in convert_layer
    fields = cls.parse_tf_attrs(tf_node.attr)
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py", line 177, in parse_tf_attrs
    for key, val in attrs.items()}
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py", line 177, in <dictcomp>
    for key, val in attrs.items()}
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py", line 172, in parse_tf_attr_value
    return cls.convert_tf2uff_field(code, val)
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py", line 157, in convert_tf2uff_field
    'type': 'dtype', 'list': 'list'}[code]
KeyError: 'shape'

I then modify /usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py as follows:

:) diff converter.bac.py converter.py
155a156,159
>                 if code == 'shape':
>                     val = field_value[0]
>                     shp = val.dim
>                     return uff.List('i', [dim.size for dim in shp])

Following which I get this error:

Traceback (most recent call last):
  File "./deeplab-tf2trt.py", line 23, in <module>
    uff_model = uff.from_tensorflow_frozen_model(GRAPH_FILENAME, ["SemanticPredictions"])
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/conversion_helpers.py", line 103, in from_tensorflow_frozen_model
    return from_tensorflow(graphdef, output_nodes, **kwargs)
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/conversion_helpers.py", line 77, in from_tensorflow
    uff_metagraph_proto = uff_metagraph.to_uff()
  File "/usr/lib/python2.7/dist-packages/uff/model/meta_graph.py", line 39, in to_uff
    graphs=[graph.to_uff(debug) for graph in self.graphs],
  File "/usr/lib/python2.7/dist-packages/uff/model/graph.py", line 26, in to_uff
    graph = uff_pb.Graph(id=self.name, nodes=self._check_graph_and_get_nodes())
  File "/usr/lib/python2.7/dist-packages/uff/model/graph.py", line 46, in _check_graph_and_get_nodes
    raise extend_with_original_traceback(e, node._trace)
ValueError: Field name must be a string

Originally defined at:
  File "./deeplab-tf2trt.py", line 23, in <module>
    uff_model = uff.from_tensorflow_frozen_model(GRAPH_FILENAME, ["SemanticPredictions"])
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/conversion_helpers.py", line 103, in from_tensorflow_frozen_model
    return from_tensorflow(graphdef, output_nodes, **kwargs)
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/conversion_helpers.py", line 75, in from_tensorflow
    name="main")
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py", line 64, in convert_tf2uff_graph
    uff_graph, input_replacements)
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py", line 51, in convert_tf2uff_node
    op, name, tf_node, inputs, uff_graph, tf_nodes=tf_nodes)
  File "/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py", line 29, in convert_layer
    uff_graph.custom_node(op, inputs, name, fields)
  File "/usr/lib/python2.7/dist-packages/uff/model/graph.py", line 233, in custom_node
    return self._add_node(op, name, inputs=inputs, fields=fields, extra_fields=extra_fields)
  File "/usr/lib/python2.7/dist-packages/uff/model/graph.py", line 65, in _add_node
    node = Node(self, op, name, inputs, fields, extra_fields)

Can anyone shed any light on this error?

AastaLLL · March 26, 2018, 6:12am

Hi,

Not all TensorFlow operation is supported by TensorRT.
You can find the detail supported layers in our document:
[url]Developer Guide :: NVIDIA Deep Learning TensorRT Documentation

If a layer is not supported by TensorRT, it’s not recommended to modify our parser.
It may cause some unexpected error when launching TensorRT engine.

For your use case, flatten layer is supported by our latest TensorRT v3.0.4 package.
But limit to the flatten layer placed in front of fully connected layer.

Thanks.

wuxiekeji · April 3, 2018, 8:16am

Thanks for the help. Do you know if TensorRT 4 is intended to support these layers? I just installed TensorRT 4 and getting a different error than before:

/usr/local/lib/python3.5/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using output node SemanticPredictions
Converting to UFF graph
Warning: No conversion function registered for layer: Identity yet.
Converting as custom op Identity SemanticPredictions
name: "SemanticPredictions"
op: "Identity"
input: "Squeeze_2"
attr {
  key: "T"
  value {
    type: DT_INT64
  }
}
attr {
  key: "_output_shapes"
  value {
    list {
      shape {
        dim {
          size: 1
        }
        dim {
          size: -1
        }
        dim {
          size: -1
        }
      }
    }
  }
}

Traceback (most recent call last):
  File "/usr/lib/python3.5/dist-packages/uff/converters/tensorflow/converter.py", line 82, in convert_tf2numpy_dtype
    return dtype.as_numpy_dtype
AttributeError: 'AttrValue' object has no attribute 'as_numpy_dtype'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./tf2trt.py", line 17, in <module>
    uff_model = uff.from_tensorflow_frozen_model(GRAPH_FILENAME, ["SemanticPredictions"])
  File "/usr/lib/python3.5/dist-packages/uff/converters/tensorflow/conversion_helpers.py", line 113, in from_tensorflow_frozen_model
    return from_tensorflow(tf_graphdef, output_nodes, **kwargs)
  File "/usr/lib/python3.5/dist-packages/uff/converters/tensorflow/conversion_helpers.py", line 77, in from_tensorflow
    name="main")
  File "/usr/lib/python3.5/dist-packages/uff/converters/tensorflow/converter.py", line 74, in convert_tf2uff_graph
    uff_graph, input_replacements)
  File "/usr/lib/python3.5/dist-packages/uff/converters/tensorflow/converter.py", line 61, in convert_tf2uff_node
    op, name, tf_node, inputs, uff_graph, tf_nodes=tf_nodes)
  File "/usr/lib/python3.5/dist-packages/uff/converters/tensorflow/converter.py", line 31, in convert_layer
    fields = cls.parse_tf_attrs(tf_node.attr)
  File "/usr/lib/python3.5/dist-packages/uff/converters/tensorflow/converter.py", line 201, in parse_tf_attrs
    for key, val in attrs.items()}
  File "/usr/lib/python3.5/dist-packages/uff/converters/tensorflow/converter.py", line 201, in <dictcomp>
    for key, val in attrs.items()}
  File "/usr/lib/python3.5/dist-packages/uff/converters/tensorflow/converter.py", line 196, in parse_tf_attr_value
    return cls.convert_tf2uff_field(code, val)
  File "/usr/lib/python3.5/dist-packages/uff/converters/tensorflow/converter.py", line 170, in convert_tf2uff_field
    return TensorFlowToUFFConverter.convert_tf2numpy_dtype(val)
  File "/usr/lib/python3.5/dist-packages/uff/converters/tensorflow/converter.py", line 87, in convert_tf2numpy_dtype
    return np.dtype(dt[dtype])
TypeError: list indices must be integers or slices, not AttrValue

AastaLLL · April 10, 2018, 3:32am

Hi,

Supported layer of TensorRT 4 can be found here:
[url]Developer Guide :: NVIDIA Deep Learning TensorRT Documentation

Thanks.