python prepare_trt_graph.py --output-path model --model ssd_mobilenet_v2_coco --trt-graph model/ssd_mobilenet_v2_coco_trt.pb 2021-05-19 20:09:15.242987: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.2 WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them. WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see: * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md * https://github.com/tensorflow/addons * https://github.com/tensorflow/io (for I/O related ops) If you depend on functionality not listed there, please file an issue. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/slim/nets/inception_resnet_v2.py:373: The name tf.GraphKeys is deprecated. Please use tf.compat.v1.GraphKeys instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/slim/nets/mobilenet/mobilenet.py:389: The name tf.nn.avg_pool is deprecated. Please use tf.nn.avg_pool2d instead. 2021-05-19 20:09:27.284219: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libnvinfer.so.7 2021-05-19 20:09:27.287374: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libnvinfer_plugin.so.7 [INFO] downloading model... --2021-05-19 20:09:27-- http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_coco_2018_03_29.tar.gz Resolving download.tensorflow.org (download.tensorflow.org)... 2a00:1450:4001:812::2010, 142.250.185.208 Connecting to download.tensorflow.org (download.tensorflow.org)|2a00:1450:4001:812::2010|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 187925923 (179M) [application/x-tar] Saving to: ‘model/ssd_mobilenet_v2_coco_2018_03_29.tar.gz’ model/ssd_mobilenet_v2_coco_2018_03_29.tar.gz 100%[================================================================================================================>] 179,22M 5,54MB/s in 32s 2021-05-19 20:09:59 (5,60 MB/s) - ‘model/ssd_mobilenet_v2_coco_2018_03_29.tar.gz’ saved [187925923/187925923] [INFO] preparing the TensorFlow graph... WARNING:tensorflow:From /home/nvidia/tf_trt_models/tf_trt_models/detection.py:139: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/tf_trt_models/detection.py:143: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead. 2021-05-19 20:10:04.951490: W tensorflow/core/platform/profile_utils/cpu_utils.cc:98] Failed to find bogomips in /proc/cpuinfo; cannot determine CPU frequency 2021-05-19 20:10:04.952497: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x362779b0 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2021-05-19 20:10:04.953027: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2021-05-19 20:10:04.959478: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1 2021-05-19 20:10:05.806249: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:05.806951: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3622b1a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2021-05-19 20:10:05.807024: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Xavier, Compute Capability 7.2 2021-05-19 20:10:05.807638: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:05.807775: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1665] Found device 0 with properties: name: Xavier major: 7 minor: 2 memoryClockRate(GHz): 1.109 pciBusID: 0000:00:00.0 2021-05-19 20:10:05.807844: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.2 2021-05-19 20:10:05.808041: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10 2021-05-19 20:10:05.811181: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10 2021-05-19 20:10:06.209481: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10 2021-05-19 20:10:06.489114: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10 2021-05-19 20:10:06.640435: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10 2021-05-19 20:10:06.642110: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8 2021-05-19 20:10:06.642818: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:06.643452: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:06.643746: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1793] Adding visible gpu devices: 0 2021-05-19 20:10:06.644039: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.2 2021-05-19 20:10:19.146887: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1206] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-05-19 20:10:19.147066: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] 0 2021-05-19 20:10:19.147112: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1225] 0: N 2021-05-19 20:10:19.147693: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:19.148154: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:19.148371: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1351] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1284 MB memory) -> physical GPU (device: 0, name: Xavier, pci bus id: 0000:00:00.0, compute capability: 7.2) WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/exporter.py:374: The name tf.gfile.MakeDirs is deprecated. Please use tf.io.gfile.makedirs instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/exporter.py:130: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/core/preprocessor.py:2340: The name tf.image.resize_images is deprecated. Please use tf.image.resize instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/meta_architectures/ssd_meta_arch.py:390: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/core/anchor_generator.py:149: The name tf.assert_equal is deprecated. Please use tf.compat.v1.assert_equal instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/core/box_predictor.py:704: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/exporter.py:245: The name tf.add_to_collection is deprecated. Please use tf.compat.v1.add_to_collection instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/exporter.py:402: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/exporter.py:406: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead. 2021-05-19 20:10:51.339250: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:51.339496: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1665] Found device 0 with properties: name: Xavier major: 7 minor: 2 memoryClockRate(GHz): 1.109 pciBusID: 0000:00:00.0 2021-05-19 20:10:51.339611: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.2 2021-05-19 20:10:51.339707: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10 2021-05-19 20:10:51.339774: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10 2021-05-19 20:10:51.339845: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10 2021-05-19 20:10:51.339903: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10 2021-05-19 20:10:51.339996: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10 2021-05-19 20:10:51.340051: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8 2021-05-19 20:10:51.340266: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:51.340454: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:51.340531: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1793] Adding visible gpu devices: 0 2021-05-19 20:10:51.342340: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:51.342982: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1665] Found device 0 with properties: name: Xavier major: 7 minor: 2 memoryClockRate(GHz): 1.109 pciBusID: 0000:00:00.0 2021-05-19 20:10:51.343096: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.2 2021-05-19 20:10:51.343188: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10 2021-05-19 20:10:51.343261: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10 2021-05-19 20:10:51.343319: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10 2021-05-19 20:10:51.343375: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10 2021-05-19 20:10:51.343429: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10 2021-05-19 20:10:51.343478: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8 2021-05-19 20:10:51.343635: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:51.343849: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:51.343925: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1793] Adding visible gpu devices: 0 2021-05-19 20:10:51.344028: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1206] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-05-19 20:10:51.344056: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] 0 2021-05-19 20:10:51.344080: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1225] 0: N 2021-05-19 20:10:51.344223: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:51.344385: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:10:51.344488: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1351] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1284 MB memory) -> physical GPU (device: 0, name: Xavier, pci bus id: 0000:00:00.0, compute capability: 7.2) WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/exporter.py:68: The name tf.GraphOptions is deprecated. Please use tf.compat.v1.GraphOptions instead. 2021-05-19 20:11:05.934814: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:05.935064: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1665] Found device 0 with properties: name: Xavier major: 7 minor: 2 memoryClockRate(GHz): 1.109 pciBusID: 0000:00:00.0 2021-05-19 20:11:05.935177: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.2 2021-05-19 20:11:05.935286: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10 2021-05-19 20:11:05.935354: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10 2021-05-19 20:11:05.935414: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10 2021-05-19 20:11:05.935514: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10 2021-05-19 20:11:05.935580: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10 2021-05-19 20:11:05.935637: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8 2021-05-19 20:11:05.935840: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:05.936011: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:05.937231: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1793] Adding visible gpu devices: 0 2021-05-19 20:11:05.937322: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1206] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-05-19 20:11:05.937353: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] 0 2021-05-19 20:11:05.937379: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1225] 0: N 2021-05-19 20:11:05.937643: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:05.937915: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:05.938055: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1351] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1284 MB memory) -> physical GPU (device: 0, name: Xavier, pci bus id: 0000:00:00.0, compute capability: 7.2) 2021-05-19 20:11:14.454861: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:14.455096: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1665] Found device 0 with properties: name: Xavier major: 7 minor: 2 memoryClockRate(GHz): 1.109 pciBusID: 0000:00:00.0 2021-05-19 20:11:14.455188: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.2 2021-05-19 20:11:14.455280: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10 2021-05-19 20:11:14.455344: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10 2021-05-19 20:11:14.455404: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10 2021-05-19 20:11:14.455462: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10 2021-05-19 20:11:14.455517: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10 2021-05-19 20:11:14.455568: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8 2021-05-19 20:11:14.455753: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:14.456034: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:14.456158: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1793] Adding visible gpu devices: 0 2021-05-19 20:11:14.456276: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1206] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-05-19 20:11:14.456311: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] 0 2021-05-19 20:11:14.456339: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1225] 0: N 2021-05-19 20:11:14.456525: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:14.456712: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:14.456827: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1351] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1284 MB memory) -> physical GPU (device: 0, name: Xavier, pci bus id: 0000:00:00.0, compute capability: 7.2) WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/exporter.py:285: The name tf.saved_model.builder.SavedModelBuilder is deprecated. Please use tf.compat.v1.saved_model.builder.SavedModelBuilder instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/exporter.py:285: The name tf.saved_model.builder.SavedModelBuilder is deprecated. Please use tf.compat.v1.saved_model.builder.SavedModelBuilder instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/exporter.py:294: The name tf.saved_model.signature_def_utils.build_signature_def is deprecated. Please use tf.compat.v1.saved_model.signature_def_utils.build_signature_def instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/exporter.py:294: The name tf.saved_model.signature_def_utils.build_signature_def is deprecated. Please use tf.compat.v1.saved_model.signature_def_utils.build_signature_def instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/exporter.py:300: The name tf.saved_model.tag_constants.SERVING is deprecated. Please use tf.saved_model.SERVING instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/exporter.py:300: The name tf.saved_model.tag_constants.SERVING is deprecated. Please use tf.saved_model.SERVING instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/utils/config_util.py:171: The name tf.gfile.Open is deprecated. Please use tf.io.gfile.GFile instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/third_party/models/research/object_detection/utils/config_util.py:171: The name tf.gfile.Open is deprecated. Please use tf.io.gfile.GFile instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/tf_trt_models/detection.py:154: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead. WARNING:tensorflow:From /home/nvidia/tf_trt_models/tf_trt_models/detection.py:154: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead. [INFO] creating TRT graph... 2021-05-19 20:11:22.978386: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libnvinfer.so.7 2021-05-19 20:11:29.968260: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:29.968443: I tensorflow/core/grappler/devices.cc:55] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0 2021-05-19 20:11:29.969365: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session 2021-05-19 20:11:29.975462: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:29.975676: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1665] Found device 0 with properties: name: Xavier major: 7 minor: 2 memoryClockRate(GHz): 1.109 pciBusID: 0000:00:00.0 2021-05-19 20:11:29.975790: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.2 2021-05-19 20:11:29.975907: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10 2021-05-19 20:11:29.975982: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10 2021-05-19 20:11:29.976047: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10 2021-05-19 20:11:29.976116: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10 2021-05-19 20:11:29.976230: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10 2021-05-19 20:11:29.976302: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8 2021-05-19 20:11:29.976677: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:29.976860: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:29.976930: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1793] Adding visible gpu devices: 0 2021-05-19 20:11:29.977023: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1206] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-05-19 20:11:29.977051: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] 0 2021-05-19 20:11:29.977075: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1225] 0: N 2021-05-19 20:11:29.977264: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:29.977454: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1049] ARM64 does not support NUMA - returning NUMA node zero 2021-05-19 20:11:29.977590: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1351] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1284 MB memory) -> physical GPU (device: 0, name: Xavier, pci bus id: 0000:00:00.0, compute capability: 7.2) 2021-05-19 20:11:38.116810: I tensorflow/compiler/tf2tensorrt/segment/segment.cc:486] There are 1850 ops of 29 different types in the graph that are not converted to TensorRT: Fill, Merge, Switch, Range, ConcatV2, ZerosLike, Identity, NonMaxSuppressionV3, Minimum, StridedSlice, ExpandDims, Unpack, TopKV2, Cast, Transpose, Placeholder, ResizeBilinear, Squeeze, Mul, Sub, Const, Greater, Shape, Where, Reshape, NoOp, GatherV2, AddV2, Pack, (For more information see https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#supported-ops). 2021-05-19 20:11:40.106323: I tensorflow/compiler/tf2tensorrt/convert/convert_graph.cc:647] Number of TensorRT candidate segments: 2 2021-05-19 20:11:40.837891: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libnvinfer.so.7 2021-05-19 20:11:40.993103: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libnvinfer_plugin.so.7 2021-05-19 20:15:59.768387: I tensorflow/compiler/tf2tensorrt/convert/convert_graph.cc:748] TensorRT node TRTEngineOp_0 added for segment 0 consisting of 846 nodes succeeded. 2021-05-19 20:16:02.135278: I tensorflow/compiler/tf2tensorrt/convert/convert_graph.cc:748] TensorRT node Postprocessor/BatchMultiClassNonMaxSuppression/TRTEngineOp_1 added for segment 1 consisting of 452 nodes succeeded. 2021-05-19 20:16:04.911652: W tensorflow/compiler/tf2tensorrt/convert/trt_optimization_pass.cc:183] TensorRTOptimizer is probably called on funcdef! This optimizer must *NOT* be called on function objects. 2021-05-19 20:16:05.578595: W tensorflow/compiler/tf2tensorrt/convert/trt_optimization_pass.cc:183] TensorRTOptimizer is probably called on funcdef! This optimizer must *NOT* be called on function objects. 2021-05-19 20:16:05.757716: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:821] Optimization results for grappler item: tf_graph 2021-05-19 20:16:05.757879: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] constant_folding: Graph size after: 6848 nodes (-1195), 8944 edges (-1805), time = 3723.29102ms. 2021-05-19 20:16:05.757941: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] layout: Graph size after: 6874 nodes (26), 8970 edges (26), time = 1687.50403ms. 2021-05-19 20:16:05.757969: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] constant_folding: Graph size after: 6874 nodes (0), 8970 edges (0), time = 915.408ms. 2021-05-19 20:16:05.757999: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] TensorRTOptimizer: Graph size after: 5578 nodes (-1296), 7557 edges (-1413), time = 265104.938ms. 2021-05-19 20:16:05.758037: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] constant_folding: Graph size after: 5574 nodes (-4), 7557 edges (0), time = 1420.297ms. 2021-05-19 20:16:05.758064: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:821] Optimization results for grappler item: TRTEngineOp_0_native_segment 2021-05-19 20:16:05.758088: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] constant_folding: Graph size after: 849 nodes (0), 961 edges (0), time = 256.173ms. 2021-05-19 20:16:05.758112: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] layout: Graph size after: 849 nodes (0), 961 edges (0), time = 258.762ms. 2021-05-19 20:16:05.758139: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] constant_folding: Graph size after: 849 nodes (0), 961 edges (0), time = 225.773ms. 2021-05-19 20:16:05.758162: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] TensorRTOptimizer: Graph size after: 849 nodes (0), 961 edges (0), time = 23.945ms. 2021-05-19 20:16:05.758191: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] constant_folding: Graph size after: 849 nodes (0), 961 edges (0), time = 315.906ms. 2021-05-19 20:16:05.758213: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:821] Optimization results for grappler item: Postprocessor/BatchMultiClassNonMaxSuppression/TRTEngineOp_1_native_segment 2021-05-19 20:16:05.758237: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] constant_folding: Graph size after: 543 nodes (0), 542 edges (0), time = 60.377ms. 2021-05-19 20:16:05.758259: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] layout: Graph size after: 543 nodes (0), 542 edges (0), time = 69.423ms. 2021-05-19 20:16:05.758281: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] constant_folding: Graph size after: 543 nodes (0), 542 edges (0), time = 69.765ms. 2021-05-19 20:16:05.758303: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] TensorRTOptimizer: Graph size after: 543 nodes (0), 542 edges (0), time = 7.281ms. 2021-05-19 20:16:05.758326: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:823] constant_folding: Graph size after: 543 nodes (0), 542 edges (0), time = 53.715ms. [INFO] serializing TRT graph...