2021-08-12 10:50:28.998084: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0 2021-08-12 10:50:30.633276: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcuda.so.1 2021-08-12 10:50:30.662176: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: pciBusID: 0000:02:00.0 name: NVIDIA GeForce RTX 3080 computeCapability: 8.6 coreClock: 1.71GHz coreCount: 68 deviceMemorySize: 9.78GiB deviceMemoryBandwidth: 707.88GiB/s 2021-08-12 10:50:30.662228: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0 2021-08-12 10:50:30.666839: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.11 2021-08-12 10:50:30.666933: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.11 2021-08-12 10:50:30.668502: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcufft.so.10 2021-08-12 10:50:30.668916: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcurand.so.10 2021-08-12 10:50:30.670356: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusolver.so.11 2021-08-12 10:50:30.671590: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusparse.so.11 2021-08-12 10:50:30.671816: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8 2021-08-12 10:50:30.673337: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0 Loading TensorRT WARNING:tensorflow:SavedModel saved prior to TF 2.5 detected when loading Keras model. Please ensure that you are saving the model with model.save() or tf.keras.models.save_model(), *NOT* tf.saved_model.save(). To confirm, there should be a file named "keras_metadata.pb" in the SavedModel directory. 2021-08-12 10:50:30.800290: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2021-08-12 10:50:30.802868: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: pciBusID: 0000:02:00.0 name: NVIDIA GeForce RTX 3080 computeCapability: 8.6 coreClock: 1.71GHz coreCount: 68 deviceMemorySize: 9.78GiB deviceMemoryBandwidth: 707.88GiB/s 2021-08-12 10:50:30.805847: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0 2021-08-12 10:50:30.805959: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0 2021-08-12 10:50:31.821455: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-08-12 10:50:31.821506: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264] 0 2021-08-12 10:50:31.821519: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0: N 2021-08-12 10:50:31.823782: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8026 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3080, pci bus id: 0000:02:00.0, compute capability: 8.6) 2021-08-12 10:50:32.353075: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2) 2021-08-12 10:50:32.371781: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 1700000000 Hz 2021-08-12 10:50:32.440022: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libnvinfer.so.7 Finished Loading TensorRT Model TensorRT Model Inference BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:48.732407: I tensorflow/compiler/tf2tensorrt/common/utils.cc:58] Linked TensorRT version: 7.2.2 2021-08-12 10:50:48.732535: I tensorflow/compiler/tf2tensorrt/common/utils.cc:60] Loaded TensorRT version: 7.2.3 2021-08-12 10:50:48.735820: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libnvinfer_plugin.so.7 2021-08-12 10:50:53.188516: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 4.508435487747192 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.199583: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.01087331771850586 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.210054: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010375499725341797 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.220950: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010891199111938477 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.231558: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010548591613769531 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.242120: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.01050424575805664 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.252620: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010438203811645508 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.263287: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010632514953613281 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.273941: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010562896728515625 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.285779: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.01178121566772461 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.296565: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010719776153564453 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.307241: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010585308074951172 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.317128: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009809017181396484 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.327136: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009989500045776367 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.337206: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009980916976928711 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.347254: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.00998687744140625 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.357272: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009961605072021484 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.367342: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010035037994384766 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.377389: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.00997161865234375 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.387362: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009945154190063477 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.398257: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010762453079223633 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.407800: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009525537490844727 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.417820: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009961843490600586 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.427834: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.00995945930480957 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.437815: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009917497634887695 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.447841: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009964466094970703 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.457849: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.00996088981628418 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.467757: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009829521179199219 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.477743: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009947538375854492 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.487544: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.00971841812133789 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.497535: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009959697723388672 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.507597: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009980201721191406 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.517783: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010145187377929688 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.529085: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.011187314987182617 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.538607: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009446382522583008 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.548652: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.009968280792236328 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.558725: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010113716125488281 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.569517: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010606050491333008 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.580158: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010719537734985352 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.590795: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010446310043334961 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.601416: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010520458221435547 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.611900: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010432243347167969 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.622377: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010553836822509766 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.633107: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010666847229003906 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.644904: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.011739253997802734 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.655637: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010670661926269531 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.666325: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010628700256347656 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.677045: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010657787322998047 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.687775: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010670185089111328 16 BS: 16 Image Shape: (16, 96, 96, 3) in Infer 2021-08-12 10:50:53.698464: E tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:42] DefaultLogger INVALID_ARGUMENT: Cannot set empty memory. ##################### Batch time 0.010620594024658203 16