Recently in our project we moved from Tensorflow 1.15 to Tensorflow 2.1. Until we were using Tensorflow 1.15 we were able to optimize frozen graph using python script with this guide https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#using-frozengraph and save that optimized frozen graph and later use it for inference.
Now with Tensorflow 2 we are facing issues:
- It is not possible to use frozen graph for TensorRT optimizations, only saved model.
- It is not possible to separate optimizations from the inference (is_dynamic_op=False is not available anymore)
How to resolve these issues?