Convert tensorflow frozen model to UFF

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I’m converting a tensorflow frozen model to uff. Uff model was converted but failed in converting to TRTEngine.
The message came out in UFF conversion is as follows.

UFF Version 0.6.5
=== Automatically deduced input nodes ===
[name: "input"
op: "Placeholder"
attr {
  key: "dtype"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "shape"
  value {
    shape {
      dim {
        size: -1
      }
      dim {
        size: 24
      }
      dim {
        size: 94
      }
      dim {
        size: 3
      }
    }
  }
}
]
=========================================

=== Automatically deduced output nodes ===
[name: "d_predictions"
op: "SparseToDense"
input: "CTCGreedyDecoder"
input: "d_predictions/output_shape"
input: "ToInt32"
input: "d_predictions/default_value"
attr {
  key: "T"
  value {
    type: DT_INT32
  }
}
attr {
  key: "Tindices"
  value {
    type: DT_INT64
  }
}
attr {
  key: "validate_indices"
  value {
    b: true
  }
}
]
==========================================

Using output node d_predictions
Converting to UFF graph
Warning: No conversion function registered for layer: SparseToDense yet.
Converting d_predictions as custom op: SparseToDense
W0727 17:56:18.319413 140001436956160 deprecation_wrapper.py:119] From /home/itc/venv/lib/python3.6/site-packages/uff/converters/tensorflow/converter.py:179: The name tf.AttrValue is deprecated. Please use tf.compat.v1.AttrValue instead.

Warning: No conversion function registered for layer: Cast yet.
Converting ToInt32 as custom op: Cast
Warning: No conversion function registered for layer: CTCGreedyDecoder yet.
Converting CTCGreedyDecoder as custom op: CTCGreedyDecoder
Warning: No conversion function registered for layer: Fill yet.
Converting Fill as custom op: Fill
Warning: No conversion function registered for layer: Tile yet.
Converting Tile as custom op: Tile
DEBUG: convert reshape to flatten node
DEBUG [/home/itc/venv/lib/python3.6/site-packages/uff/converters/tensorflow/converter.py:96] Marking ['d_predictions'] as outputs
No. nodes: 172
UFF Output written to graph.pb.uff

That means SparseToDense, Cast, Fill, Tile have no equivalent TRT api, so I need to create plugins manually.
Is that true?

2

Hi @edit_or,
UFF conversion has been deprecated from TRT>=7.
The recommended way is to use
TF << ONNX << TRT
or
use TF-TRT

Please check the below links for reference.
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleOnnxMNIST

Thanks!

3

My main purpose is to use in Deepstream. If I use TF-TRT, the produced engine can be used in Deepstream?

4

Hi @edit_or,
Deepstream only supports pure TensorRT model rather than TF-TRT.
For further Deepstream related queries , respective forum will be able to assist you better.
Thanks!