Starting with a tensorflow inception_v3 graph trainied on ImageNet I have:
retrained on a different set of images
frozen the graph
classified images with Python3 on a x86, Xavier and Tx2
Next I followed the example from:
converted the frozen graph to a TensorRT graph
saw a speed up on both the TX2 and Xavier
This is all using python3.
I would now like to take the newly created TensorRT graph and classify the images using a C program.
What is the currently suggested method?
I read that UFF is on the outs and that TF_TRT is the way to go.
I would like to go that way from inside a C program.
Are there any example?
Is there an NVIDIA blessed approach?
In general, the way to call C++ code from C is via an interface layer: you wrap the C++ functions you’re interested in with appropriate extern “C” functions which are then callable by C (for example see Using C++ library in C code - Stack Overflow or Standard C++). For your use-case, you shouldn’t need to wrap all of TensorRT: I bet you could get away with maybe a handful of functions which interface to the appropriate C++ functions.
But you also might consider what advantage C is giving you and if you can just use C++ directly.
Both TensorRT and TensorFlow have C++ APIs. Tensorflow is written in C++. Your TensorRT install should include C++ libraries and headers which you can use. I’d personally suggest looking at the samples they include in their package to get started.
I was being lazy and hoped that there was a working sample, as there was (is) for 3.2. I have upgraded to 4.2 via the sdkmanager.
Yes I am looking at those. I am in the process of writing an image classification app that uses a frozen inception_v3 graph, which has been retrained (finetuned) and then converted by a python script to FP16.
Everything works well using python. The last step is to do the same classification via C++.
I am trying not to use the UFF method since I have read that it is not favored and may be dropped in future releases. I am now putting my nose to the grind stone.