# TensorFlow to C++ **URL:** \ **Category:** Jetson AGX Xavier\ **Tags:** tensorflow\ **Created:** [April 18, 2020, 9:54am UTC](https://forums.developer.nvidia.com/t/tensorflow-to-c/120229 "2020-04-18T09:54:59Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![eyalhir74](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@eyalhir74](https://forums.developer.nvidia.com/u/eyalhir74)\ **Post date:** [April 18, 2020, 9:54am UTC](https://forums.developer.nvidia.com/t/tensorflow-to-c/120229/1 "2020-04-18T09:54:59Z")
Hi, I’ve been working with CUDA for the last couple of year on desktops/servers. I’ve looked for quite some time now, for a TensorFlow to C++ demo/tutorial - taking a tensor flow module and running the inference in C++ app, optimize it etc. I’ve looked at the Hello AI demo, but it doesn’t show this as far as I could tell. Any pointers? Also, this is something else I’ve not fully understood. Once I have a trained net in TF, do I must convert it to UFF/ONNX and then somehow to nvidia’s tensor flow plan? why so many error-prone steps? Isn’t there something simpler to take a trained net in TF and run the inference with C++ TensorRT? Hope this makes sense :) thanks Eyal ---
**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\ **Post date:** [April 20, 2020, 3:25am UTC](https://forums.developer.nvidia.com/t/tensorflow-to-c/120229/3 "2020-04-20T03:25:32Z")
Hi, These are two different frameworks: _TensorFlow_ and _TensorRT_. #### 1. It’s possible to inference TensorFlow in C++ interface without converting the model. You will need a TensorFlow C++ library. Please check this topic for some information: > [@Building Tensorflow 1.13 on Jetson Xavier](https://forums.developer.nvidia.com/t/building-tensorflow-1-13-on-jetson-xavier/75966): > > Hello All, I was struggling a lot building tensorflow on Jetson Xavier and I couldn’t find a working script which would guide through everything so I searched a lot and tried different things for days and finally was successful to build it from source. So I am going to share what I did here and hopefully it helps people who want to do the same in future. I have tried to specify all the steps I have done but I might have forgotten few things so please feel free to add anything related which impr… #### 2. However, we recommends to convert your model into TensorRT which is an optimizer for GPU-based inference. The first step is to check if all the used operation are supported by TensorRT first: > **[Documentation Archives :: NVIDIA Deep Learning TensorRT Documentation](https://docs.nvidia.com/deeplearning/tensorrt/archives/index.html)** > > This Archives document provides access to previously released NVIDIA TensorRT documentation versions. If yes, you will need to convert the TensorFlow into an intermediate model format _(UFF/ONNX)_ as TensorRT input. #### 3. There is also an alternative to convert the model into TensorRT within TensorFlow directly. You can check this sample for information: [https://github.com/tensorflow/tensorrt/tree/master/tftrt/examples/object\_detection](https://github.com/tensorflow/tensorrt/tree/master/tftrt/examples/object_detection) Thanks. ---
**Author:** ![eyalhir74](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@eyalhir74](https://forums.developer.nvidia.com/u/eyalhir74)\ **Post date:** [April 20, 2020, 11:46am UTC](https://forums.developer.nvidia.com/t/tensorflow-to-c/120229/5 "2020-04-20T11:46:06Z")
Hi, I’ve looked a bit more and if I understand this correctly, option 3 that you mention is the direct TF-TRT path? I.e. convert the graph to an optimized TensorRT plan file ready to run in a C++ application? The link you’ve put in option 3 seems not relevant? Also, if I do manage to create a .plan file for TensorRT from within TensorFlow on, say, a desktop with a GTX card, isn’t there an issue to take to a different platform (for example Xaviar)? isn’t the conversion platform dependant? thanks a lot Eyal ---
**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\ **Post date:** [April 21, 2020, 5:54am UTC](https://forums.developer.nvidia.com/t/tensorflow-to-c/120229/6 "2020-04-21T05:54:12Z")
Hi, Suppose there is a corresponding function in the C++ interface although the tutorial is python-based. The plan file is not portable. TensorRT will choose an optimal algorithm based on the GPU architecture when creating the engine. This limits you to use a plan file created from different platform. However, uff file is portable. It is an intermediate description for model that independent to the GPU architecture. Thanks. ---
**Author:** ![eyalhir74](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@eyalhir74](https://forums.developer.nvidia.com/u/eyalhir74)\ **Post date:** [April 21, 2020, 12:01pm UTC](https://forums.developer.nvidia.com/t/tensorflow-to-c/120229/7 "2020-04-21T12:01:29Z")
Hi, Thanks again for the answers. I guess something still missing for me. Assuming the development work is NOT done on Xavier, how would I run an optimized plan in C++ on the Xavier itself? If I understood correctly from what you say, I have only two options? - Develop, build TF net and save the TF output to TRT - ALL on the target machine - Xavier. - Develop on whatever platform I’d like (No Xavier) and via Onnx/Uff convert to Xavier. Is that correct? If so, this is extremely cumbersome :( thanks Eyal ---
**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\ **Post date:** [April 22, 2020, 2:05am UTC](https://forums.developer.nvidia.com/t/tensorflow-to-c/120229/8 "2020-04-22T02:05:45Z")
Hi, In general, the workflow like this: 1. Train your model on the host. 2. Convert your model into .uff or .onnx 3. Copy step.2 file to the device 4. Create a TensorRT engine from the file. Thanks. ---
**Author:** ![TomNVIDIA](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/tomnvidia/32/14181_2.png) [@TomNVIDIA](https://forums.developer.nvidia.com/u/TomNVIDIA)\ **Post date:** [October 18, 2021, 6:22pm UTC](https://forums.developer.nvidia.com/t/tensorflow-to-c/120229/10 "2021-10-18T18:22:49Z")