Starting with a tensorflow inception_v3 graph trainied on ImageNet I have:
- retrained on a different set of images
- frozen the graph
- classified images with Python3 on a x86, Xavier and Tx2
Next I followed the example from:
- converted the frozen graph to a TensorRT graph
- saw a speed up on both the TX2 and Xavier
This is all using python3.
I would now like to take the newly created TensorRT graph and classify the images using a C program.
What is the currently suggested method?
I read that UFF is on the outs and that TF_TRT is the way to go.
I would like to go that way from inside a C program.
Are there any example?
Is there an NVIDIA blessed approach?
In general, the way to call C++ code from C is via an interface layer: you wrap the C++ functions you’re interested in with appropriate extern “C” functions which are then callable by C (for example see https://stackoverflow.com/questions/199418/using-c-library-in-c-code or https://isocpp.org/wiki/faq/mixing-c-and-cpp). For your use-case, you shouldn’t need to wrap all of TensorRT: I bet you could get away with maybe a handful of functions which interface to the appropriate C++ functions.
But you also might consider what advantage C is giving you and if you can just use C++ directly.
Tom, Thanks for your reply.
You imply that there is a C++ API for tensorflow.
I would be happy to use it.
I was a little lack when I said C program, C++ works for me.
I am just not clear where to find the C++ or C API for tensorflow, and / or an example.
Both TensorRT and TensorFlow have C++ APIs. Tensorflow is written in C++. Your TensorRT install should include C++ libraries and headers which you can use. I’d personally suggest looking at the samples they include in their package to get started.
I was being lazy and hoped that there was a working sample, as there was (is) for 3.2. I have upgraded to 4.2 via the sdkmanager.
Yes I am looking at those. I am in the process of writing an image classification app that uses a frozen inception_v3 graph, which has been retrained (finetuned) and then converted by a python script to FP16.
Everything works well using python. The last step is to do the same classification via C++.
I am trying not to use the UFF method since I have read that it is not favored and may be dropped in future releases. I am now putting my nose to the grind stone.
Happy Sails To You,