What is the approved method for accessing TensorRT using C code?

Starting with a tensorflow inception_v3 graph trainied on ImageNet I have:

  • retrained on a different set of images
  • frozen the graph
  • classified images with Python3 on a x86, Xavier and Tx2

Next I followed the example from:
https://medium.com/tensorflow/speed-up-tensorflow-inference-on-gpus-with-tensorrt-13b49f3db3fa

  • converted the frozen graph to a TensorRT graph
  • saw a speed up on both the TX2 and Xavier

This is all using python3.

I would now like to take the newly created TensorRT graph and classify the images using a C program.

What is the currently suggested method?
I read that UFF is on the outs and that TF_TRT is the way to go.
I would like to go that way from inside a C program.
Are there any example?
Is there an NVIDIA blessed approach?

In general, the way to call C++ code from C is via an interface layer: you wrap the C++ functions you’re interested in with appropriate extern “C” functions which are then callable by C (for example see https://stackoverflow.com/questions/199418/using-c-library-in-c-code or https://isocpp.org/wiki/faq/mixing-c-and-cpp). For your use-case, you shouldn’t need to wrap all of TensorRT: I bet you could get away with maybe a handful of functions which interface to the appropriate C++ functions.

But you also might consider what advantage C is giving you and if you can just use C++ directly.

Good luck,
Tom

Tom, Thanks for your reply.
You imply that there is a C++ API for tensorflow.
I would be happy to use it.

I was a little lack when I said C program, C++ works for me.
I am just not clear where to find the C++ or C API for tensorflow, and / or an example.

Hi Dbusby,

Both TensorRT and TensorFlow have C++ APIs. Tensorflow is written in C++. Your TensorRT install should include C++ libraries and headers which you can use. I’d personally suggest looking at the samples they include in their package to get started.

Cheers,
Tom

Hi Tom,

I was being lazy and hoped that there was a working sample, as there was (is) for 3.2. I have upgraded to 4.2 via the sdkmanager.

Yes I am looking at those. I am in the process of writing an image classification app that uses a frozen inception_v3 graph, which has been retrained (finetuned) and then converted by a python script to FP16.

Everything works well using python. The last step is to do the same classification via C++.

I am trying not to use the UFF method since I have read that it is not favored and may be dropped in future releases. I am now putting my nose to the grind stone.

Happy Sails To You,
David