TensorRT uff_to_trt_engine fails

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hello,

I am trying to create an engine from frozen tensorflow graph with the following code:

uff_model = uff.from_tensorflow_frozen_model("models/graph.pb", output_layers, output_filename="graph.uff")
parser = uffparser.create_uff_parser()
parser.register_input(input_layers, (1, 416, 416), 0)
parser.register_output(output_layers)

engine = trt.utils.uff_to_trt_engine(G_LOGGER, uff_model, parser, 1, 1 << 20)

But it keeps giving error in the engine generation part. The output of the console is following:

Using output node detector/yolo-v3/Reshape
Using output node detector/yolo-v3/Reshape_5
Using output node detector/yolo-v3/Reshape_10
Converting to UFF graph
Warning: No conversion function registered for layer: ResizeNearestNeighbor yet.
Converting as custom op ResizeNearestNeighbor detector/yolo-v3/ResizeNearestNeighbor_1
name: "detector/yolo-v3/ResizeNearestNeighbor_1"
op: "ResizeNearestNeighbor"
input: "detector/yolo-v3/transpose_4"
input: "detector/yolo-v3/ResizeNearestNeighbor_1/size"
attr {
  key: "T"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "align_corners"
  value {
    b: false
  }
}

Warning: No conversion function registered for layer: ResizeNearestNeighbor yet.
Converting as custom op ResizeNearestNeighbor detector/yolo-v3/ResizeNearestNeighbor
name: "detector/yolo-v3/ResizeNearestNeighbor"
op: "ResizeNearestNeighbor"
input: "detector/yolo-v3/transpose_1"
input: "detector/yolo-v3/ResizeNearestNeighbor/size"
attr {
  key: "T"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "align_corners"
  value {
    b: false
  }
}

No. nodes: 778
UFF Output written to masa2.uff
/usr/lib/python2.7/dist-packages/tensorrt/utils/_utils.py:237: RuntimeWarning: tp_compare didn't return -1 or -2 for exception
  if datatype == infer.DataType.HALF and not builder.platform_has_fast_fp16():
Traceback (most recent call last):
  File "/home/user/PycharmProjects/TensorOptimize/optimize_yolo.py", line 54, in <module>
    generate_engine(output_layers=["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_5", "detector/yolo-v3/Reshape_10"], input_layers=["Placeholder"])
  File "/home/user/PycharmProjects/TensorOptimize/optimize_yolo.py", line 31, in generate_engine
    engine = trt.utils.uff_to_trt_engine(G_LOGGER, uff_model, parser, 1, 1 << 20,datatype=trt.infer.DataType.HALF)
  File "/usr/lib/python2.7/dist-packages/tensorrt/utils/_utils.py", line 237, in uff_to_trt_engine
    if datatype == infer.DataType.HALF and not builder.platform_has_fast_fp16():

TypeError: Failed to decode Unicode string

Is there anybody with a solution to this ? Thanks

2

Hello, can you provide details on the platforms you are using?

Linux distro and version
GPU type
nvidia driver version
CUDA version
CUDNN version
Python version [if using python]
Tensorflow version
TensorRT version

3

Thanks for the answer @NVES. Here are the details:

Linux Distro: Ubuntu 18.04
GPU: GT940MX
Driver Version: 390.77
CUDA Version: 9.0.176
CUDNN Version: 7.2.1
Python: 2.7
Tensorflow-gpu: 1.10
TensorRT: 4.0.1.6

4

hello,

can you provide the pb/uff ?

5

Sure, here is a link to the pb file:
https://drive.google.com/file/d/1bQhodrIRkUi3CJPZ-ikDWRhY-HdsNsmv/view?usp=sharing

6

Hello,

We convert the pb and get it to the parser; see output like this:

Warning: No conversion function registered for layer: Range yet.
Converting as custom op Range detector/yolo-v3/range
name: “detector/yolo-v3/range”
op: “Range”
input: “detector/yolo-v3/range/start”
input: “detector/yolo-v3/range/limit”
input: “detector/yolo-v3/range/delta”
attr {
key: “Tidx”
value {
type: DT_FLOAT
}
}

No. nodes: 939
[TensorRT] ERROR: UFFParser: Validator error: detector/yolo-v3/Tile_4: Unsupported operation _Tile
[TensorRT] ERROR: Network must have at least one output

which is expected.

There are several unsupported operations in the network (ex: ResizeNearestNeighbor), so, unfortunately, it will not work.

Can you give me a list of ops you are using? It will help us with future version feature planning.

7

I see. Here is the list of operations if I am not missing anything:

conv2d
concat
pad
split
reshape
meshgrid
transpose
tile
exp
sigmoid
identity
image.resize_nearest_neighbor

I am using the code in GitHub - mystic123/tensorflow-yolo-v3: Implementation of YOLO v3 object detector in Tensorflow (TF-Slim) so you can check the yolo_v3.py in the repo if I am mistaken with the list. Is there anything I can do about unsupported operations or should I wait for an update ?

8

Hello,

Please reference the TensorRT Development Guide for a list of supported operations: https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#support_op

We are always welcome community/customer feedback and improve future versions.