TensorFlow C++ workflow - ONNX vs UFF?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi,

Very quick question:

What’s the preferred way of deploying TensorFlow models: ONNX or UFF? What are the pros and cons? Which one supports more layers? Which one yields a better performing (i.e. accuracy and latency/throughput) model when deployed?

Thanks!

2

Hi,

Please refer below link for ways of deploying TensorFlow models:
https://docs.nvidia.com/deeplearning/sdk/tensorrt-archived/tensorrt-601/tensorrt-developer-guide/index.html#working_tf

Thanks

3

Hi,

I have read the guide, but that doesn’t answer my question.

TensorFlow models can be deployed in two ways:

-Converting to UFF using Nvidia’s proprietary tools and passing that to uffParser.
-Converting to ONNX using OpenSource tools and passing that to onnxParser.

I would like to know which way is preferred? Which way supports more layers?

Thanks!

4

From the description it is obvious that ONNX supports much more layers. I havent tried ONNX and with UFF I struggled because of no many supported layers, but it works at least :-)

5

Hey guys,

I have found a basic article on how to build a basic model in Tensorflow.

https://appdividend.com/2019/01/30/how-to-build-simple-model-in-tensorflow/

It is an open-source machine learning framework for everyone. TensorFlow is an open-source library for high-performance numerical computation. Tensorflow has a flexible architecture that allows easy deployment of calculation across a variety of platforms like CPUs, GPUs, TPUs, and desktops to clusters of servers to mobile and edge devices.

6

Hi,
We are investigating, and please stay tuned when we have updated releases. In the meantime, we do support both ONNX and UFF, please feel free to try any of the approaches.

Thanks

7

Thanks for the replies!

@filip.krolupper That’s a good point, I realized they do support many more layers than UFF. I guess however that TensorRT itself doesn’t support all of them.

I’ll try both approaches and see which one is easier to use/offers best performance and portability.