How can I configure to load multiple models using TF-TRT (the models inference sequentially in a single thread)?
The models are optimized with TF-TRT and I use frozen graph to load model to inference but I cannot load more than one graph when executing. The error is:
tensorflow.python.framework.errors_impl.UnavailableError: Can’t provision more than one single cluster at a time
I am using Jetson TX2, Jetpack 4.3, Tensorflow 1.15.