I am training an object detection model on my desktop and want to deploy this model on TX2.
I have installed tensorflow 1.2.1 on TX2, and tried to load the saved model (in .pb format) in python.
It turned out the runtime is 100 times slower than my desktop(GTX 1060 6GB) for detecting on image(I tried different architectures and hyperparamaters, it seems always around 100 times slower). According to the specs of TX2 compared with GTX 1060 (1.5 tflops V.s 4.4 tflops?), I think it shouldn’t be that slow. I am not quite sure if it is normal, or I was not running the model in an optimal way.
I noticed TensorRT seems the way to go, but the current version not support importing tesnorflow model according to the website(https://developer.nvidia.com/tensorrt will support in version 3).
An other way is to use tensorflow serving(https://tensorflow.github.io/serving/) to serve the model.
Anyone has some experience of deploying tensorflow models on TX2? What’s the runtime performance compared to your desktop? And how did you deploy the model?
Any related information will be really helpful.