FP32_SAVED_MODEL_DIR = SAVED_MODEL_DIR+"_TFTRT_FP32/1"
!rm -rf $FP32_SAVED_MODEL_DIR
tf.keras.backend.set_learning_phase(0)
#Now we create the TFTRT FP32 engine
trt.create_inference_graph(
input_graph_def=None,
outputs=None,
max_batch_size=1,
input_saved_model_dir=SAVED_MODEL_DIR,
output_saved_model_dir=FP32_SAVED_MODEL_DIR,
precision_mode="FP32")
benchmark_saved_model(FP32_SAVED_MODEL_DIR, BATCH_SIZE=1)
my GPU is RTX 2080ti,
my CUDA is: Cuda compilation tools, release 10.0, V10.0.130
and tensorflow-gpu==1.14.0
and i use the docker containers : nvcr.io/nvidia/tensorflow:19.08-py3