I’ve been working to optimize an SSD Mobilenet V2 model to run in TensorRT on my Jetson, some info on versioning:
Jetson TX2
cuda - 10.0
TRT - 5.6.1
TF - 1.14.0
Distributor ID: Ubuntu
Description: Ubuntu 18.04.2 LTS
Release: 18.04
Codename: bionic
Originally I’d optimized using the standard TF-TRT flow and that works and it increases speed on a 300x300 image from about 1 FPS (TF only) to 4 FPS (TF-TRT). That’s ok, but ideally I’d like to try and get the 10-20 FPS that’s reported widely with the full TRT optimization which I can also more easily call from C++.
So we’ve been wrestling with this flow for a while and following great tutorials from dusty-nv:
https://github.com/NVIDIA-AI-IOT/tf_to_trt_image_classification
https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-console-2.md
and jkjung-avt:
https://github.com/jkjung-avt/tensorrt_demos#ssd
So I understand that mostly the optimization flow people have had success with is TF -> UFF -> TRT with some plugin work to get to UFF because of unsupported operations in TF. We had initially chosen the UFF flow because there were a lot of examples and people sounded like they had difficulty with the ONNX flow:
https://github.com/jkjung-avt/tensorrt_demos/issues/43#issuecomment-584534847
We’re now running into some significant issues with UFF and TF version support:
NOTE: UFF has been tested with TensorFlow 1.12.0. Other versions are not guaranteed to work
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
UFF Version 0.6.3
and we did end up running into issues specifically with TensorFlow 1.14.0 and cuda 10.0, and the best we could tell the solution others had suggested was to revert to TF 1.12.0 and cuda 9.0 which isn’t ideal as that’d require us to revert to an older version of the os/bsp.
So I guess my questions are:
-
Has anyone had luck using UFF with these newer versions of TF/Cuda?
-
Has anyone had any luck with the ONNX flow for ssd-mobilenet-v2?
Again all the official examples seem to be for UFF, but then UFF is supposed to be deprecated soon and ONNX is supposed to be the new format with support. Can we expect some ONNX demos like the official UFF demos anytime soon?
Thanks!