TensorRT Optimization for Tensorflow-Unet-Image-Semantic-segmentation

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

I tried to run my Tensorflow-Keras-Unet Model on Jetson-Nano and Jetson-TX2 ,
but the inference time was so worse in the range of 1.5 minutes for a single image inference.

the model was trained on a windows machine and works fine in windows and ubuntu18 with an inference time in the range of 10us to 19ms

I tried running the Jetson Boards (Nano&TX2) at Max Performance modes but no response of improvement . Still inference time as 90s to 100s

I tried with TensorRT Optimization Techniques but I couldn’t complete the process as lack of knowledge about the same I couldn’t found a resource to learn about this optimization for the Image Semantic Segmentation Problems (say Unet)

Can You Provide some resources to know about this more or any other methods to improve the inference latency

3

Hi,

UNet is one of our benchmark models.
We can get 17fps on Nano and 34fps on TX2 with TensorRT. (close to your desktop score)
Please check the detailed data below:

Below is an example for converting UNet to TensorRT for your reference:

More, please note that to maximize the device performance, you also need to fix the CPU/GPU clocks to the maximum.
The setting should look like below:

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

Thanks.

4

thanks alot,
but is there any reference related to Converting Unet Model with Tensorflow into TensorRT

5

Hi,

Unfortunately, we don’t have the exact example for UNet with TensorFlow.

But there are lots of examples in GitHub you can refer to.
For example, PSENet is a TensorFlow-based model.

Thanks.