(launcher) root@ipp1-1999:/ssd# cat /ssd/tlt_cv_samples_v1.1.0/ssd/specs/ssd_train_resnet18_kitti.txt random_seed: 42 ssd_config { aspect_ratios_global: "[1.0, 2.0, 0.5, 3.0, 1.0/3.0]" scales: "[0.05, 0.1, 0.25, 0.4, 0.55, 0.7, 0.85]" two_boxes_for_ar1: true clip_boxes: false variances: "[0.1, 0.1, 0.2, 0.2]" arch: "resnet" nlayers: 18 freeze_bn: false freeze_blocks: 0 } training_config { batch_size_per_gpu: 16 num_epochs: 20 enable_qat: false learning_rate { soft_start_annealing_schedule { min_learning_rate: 5e-5 max_learning_rate: 2e-2 soft_start: 0.15 annealing: 0.8 } } regularizer { type: L1 weight: 3e-5 } } eval_config { validation_period_during_training: 2 average_precision_mode: SAMPLE batch_size: 16 matching_iou_threshold: 0.5 } nms_config { confidence_threshold: 0.01 clustering_iou_threshold: 0.6 top_k: 200 } augmentation_config { output_width: 300 output_height: 300 output_channel: 3 } dataset_config { data_sources: { label_directory_path: "/ssd/training/label_2" image_directory_path: "/ssd/training/image_2" } include_difficult_in_training: true target_class_mapping { key: "car" value: "car" } target_class_mapping { key: "pedestrian" value: "pedestrian" } target_class_mapping { key: "cyclist" value: "cyclist" } target_class_mapping { key: "van" value: "car" } target_class_mapping { key: "person_sitting" value: "pedestrian" } validation_data_sources: { label_directory_path: "/ssd/val/label" image_directory_path: "/ssd/val/image" } } (launcher) root@ipp1-1999:/ssd# tlt ssd train --gpus 1 --gpu_index=0 -e /ssd/tlt_cv_samples_v1.1.0/ssd/specs/ssd_train_resnet18_kitti.txt -r /ssd/experiment_dir_unpruned -k nvidia_tlt -m /ssd/pretrained_resnet18/tlt_pretrained_object_detection_vresnet18/resnet_18.hdf5 2021-06-16 06:43:20,013 [INFO] root: Registry: ['nvcr.io'] 2021-06-16 06:43:20,159 [WARNING] tlt.components.docker_handler.docker_handler: Docker will run the commands as root. If you would like to retain your local host permissions, please add the "user":"UID:GID" in the DockerOptions portion of the ~/.tlt_mounts.json file. You can obtain your users UID and GID by using the "id -u" and "id -g" commands on the terminal. Using TensorFlow backend. Using TensorFlow backend. WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/horovod/tensorflow/__init__.py:117: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead. 2021-06-16 13:43:29,363 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/horovod/tensorflow/__init__.py:117: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/horovod/tensorflow/__init__.py:143: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead. 2021-06-16 13:43:29,363 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/horovod/tensorflow/__init__.py:143: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead. WARNING:tensorflow:From /opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/ssd/scripts/train.py:63: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead. 2021-06-16 13:43:29,484 [WARNING] tensorflow: From /opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/ssd/scripts/train.py:63: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead. WARNING:tensorflow:From /opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/ssd/scripts/train.py:66: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead. 2021-06-16 13:43:29,485 [WARNING] tensorflow: From /opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/ssd/scripts/train.py:66: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead. 2021-06-16 13:43:29,917 [INFO] iva.ssd.utils.spec_loader: Merging specification from /ssd/tlt_cv_samples_v1.1.0/ssd/specs/ssd_train_resnet18_kitti.txt 2021-06-16 13:43:29,941 [INFO] __main__: Loading pretrained weights. This may take a while... WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead. 2021-06-16 13:43:29,942 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead. 2021-06-16 13:43:29,943 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:1834: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead. 2021-06-16 13:43:29,965 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:1834: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4185: The name tf.truncated_normal is deprecated. Please use tf.random.truncated_normal instead. 2021-06-16 13:43:30,711 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4185: The name tf.truncated_normal is deprecated. Please use tf.random.truncated_normal instead. WARNING:tensorflow:From /opt/nvidia/third_party/keras/tensorflow_backend.py:187: The name tf.nn.avg_pool is deprecated. Please use tf.nn.avg_pool2d instead. 2021-06-16 13:43:32,880 [WARNING] tensorflow: From /opt/nvidia/third_party/keras/tensorflow_backend.py:187: The name tf.nn.avg_pool is deprecated. Please use tf.nn.avg_pool2d instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead. 2021-06-16 13:43:33,097 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:199: The name tf.is_variable_initialized is deprecated. Please use tf.compat.v1.is_variable_initialized instead. 2021-06-16 13:43:33,098 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:199: The name tf.is_variable_initialized is deprecated. Please use tf.compat.v1.is_variable_initialized instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:206: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead. 2021-06-16 13:43:33,787 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:206: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/optimizers.py:790: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead. 2021-06-16 13:43:34,654 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/optimizers.py:790: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3295: The name tf.log is deprecated. Please use tf.math.log instead. 2021-06-16 13:43:34,659 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3295: The name tf.log is deprecated. Please use tf.math.log instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:986: The name tf.assign_add is deprecated. Please use tf.compat.v1.assign_add instead. 2021-06-16 13:43:35,305 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:986: The name tf.assign_add is deprecated. Please use tf.compat.v1.assign_add instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:973: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead. 2021-06-16 13:43:35,483 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:973: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead. Initialize optimizer WARNING:tensorflow:From /opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/ssd/utils/tensor_utils.py:121: The name tf.local_variables_initializer is deprecated. Please use tf.compat.v1.local_variables_initializer instead. 2021-06-16 13:44:03,688 [WARNING] tensorflow: From /opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/ssd/utils/tensor_utils.py:121: The name tf.local_variables_initializer is deprecated. Please use tf.compat.v1.local_variables_initializer instead. WARNING:tensorflow:From /opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/ssd/utils/tensor_utils.py:122: The name tf.tables_initializer is deprecated. Please use tf.compat.v1.tables_initializer instead. 2021-06-16 13:44:03,688 [WARNING] tensorflow: From /opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/ssd/utils/tensor_utils.py:122: The name tf.tables_initializer is deprecated. Please use tf.compat.v1.tables_initializer instead. WARNING:tensorflow:From /opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/ssd/utils/tensor_utils.py:123: The name tf.get_collection is deprecated. Please use tf.compat.v1.get_collection instead. 2021-06-16 13:44:03,688 [WARNING] tensorflow: From /opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/ssd/utils/tensor_utils.py:123: The name tf.get_collection is deprecated. Please use tf.compat.v1.get_collection instead. __________________________________________________________________________________________________ Layer (type) Output Shape Param # Connected to ================================================================================================== Input (InputLayer) (None, 3, 300, 300) 0 __________________________________________________________________________________________________ conv1 (Conv2D) (None, 64, 150, 150) 9408 Input[0][0] __________________________________________________________________________________________________ bn_conv1 (BatchNormalization) (None, 64, 150, 150) 256 conv1[0][0] __________________________________________________________________________________________________ activation_1 (Activation) (None, 64, 150, 150) 0 bn_conv1[0][0] __________________________________________________________________________________________________ block_1a_conv_1 (Conv2D) (None, 64, 75, 75) 36864 activation_1[0][0] __________________________________________________________________________________________________ block_1a_bn_1 (BatchNormalizati (None, 64, 75, 75) 256 block_1a_conv_1[0][0] __________________________________________________________________________________________________ block_1a_relu_1 (Activation) (None, 64, 75, 75) 0 block_1a_bn_1[0][0] __________________________________________________________________________________________________ block_1a_conv_2 (Conv2D) (None, 64, 75, 75) 36864 block_1a_relu_1[0][0] __________________________________________________________________________________________________ block_1a_conv_shortcut (Conv2D) (None, 64, 75, 75) 4096 activation_1[0][0] __________________________________________________________________________________________________ block_1a_bn_2 (BatchNormalizati (None, 64, 75, 75) 256 block_1a_conv_2[0][0] __________________________________________________________________________________________________ block_1a_bn_shortcut (BatchNorm (None, 64, 75, 75) 256 block_1a_conv_shortcut[0][0] __________________________________________________________________________________________________ add_1 (Add) (None, 64, 75, 75) 0 block_1a_bn_2[0][0] block_1a_bn_shortcut[0][0] __________________________________________________________________________________________________ block_1a_relu (Activation) (None, 64, 75, 75) 0 add_1[0][0] __________________________________________________________________________________________________ block_1b_conv_1 (Conv2D) (None, 64, 75, 75) 36864 block_1a_relu[0][0] __________________________________________________________________________________________________ block_1b_bn_1 (BatchNormalizati (None, 64, 75, 75) 256 block_1b_conv_1[0][0] __________________________________________________________________________________________________ block_1b_relu_1 (Activation) (None, 64, 75, 75) 0 block_1b_bn_1[0][0] __________________________________________________________________________________________________ block_1b_conv_2 (Conv2D) (None, 64, 75, 75) 36864 block_1b_relu_1[0][0] __________________________________________________________________________________________________ block_1b_conv_shortcut (Conv2D) (None, 64, 75, 75) 4096 block_1a_relu[0][0] __________________________________________________________________________________________________ block_1b_bn_2 (BatchNormalizati (None, 64, 75, 75) 256 block_1b_conv_2[0][0] __________________________________________________________________________________________________ block_1b_bn_shortcut (BatchNorm (None, 64, 75, 75) 256 block_1b_conv_shortcut[0][0] __________________________________________________________________________________________________ add_2 (Add) (None, 64, 75, 75) 0 block_1b_bn_2[0][0] block_1b_bn_shortcut[0][0] __________________________________________________________________________________________________ block_1b_relu (Activation) (None, 64, 75, 75) 0 add_2[0][0] __________________________________________________________________________________________________ block_2a_conv_1 (Conv2D) (None, 128, 38, 38) 73728 block_1b_relu[0][0] __________________________________________________________________________________________________ block_2a_bn_1 (BatchNormalizati (None, 128, 38, 38) 512 block_2a_conv_1[0][0] __________________________________________________________________________________________________ block_2a_relu_1 (Activation) (None, 128, 38, 38) 0 block_2a_bn_1[0][0] __________________________________________________________________________________________________ block_2a_conv_2 (Conv2D) (None, 128, 38, 38) 147456 block_2a_relu_1[0][0] __________________________________________________________________________________________________ block_2a_conv_shortcut (Conv2D) (None, 128, 38, 38) 8192 block_1b_relu[0][0] __________________________________________________________________________________________________ block_2a_bn_2 (BatchNormalizati (None, 128, 38, 38) 512 block_2a_conv_2[0][0] __________________________________________________________________________________________________ block_2a_bn_shortcut (BatchNorm (None, 128, 38, 38) 512 block_2a_conv_shortcut[0][0] __________________________________________________________________________________________________ add_3 (Add) (None, 128, 38, 38) 0 block_2a_bn_2[0][0] block_2a_bn_shortcut[0][0] __________________________________________________________________________________________________ block_2a_relu (Activation) (None, 128, 38, 38) 0 add_3[0][0] __________________________________________________________________________________________________ block_2b_conv_1 (Conv2D) (None, 128, 38, 38) 147456 block_2a_relu[0][0] __________________________________________________________________________________________________ block_2b_bn_1 (BatchNormalizati (None, 128, 38, 38) 512 block_2b_conv_1[0][0] __________________________________________________________________________________________________ block_2b_relu_1 (Activation) (None, 128, 38, 38) 0 block_2b_bn_1[0][0] __________________________________________________________________________________________________ block_2b_conv_2 (Conv2D) (None, 128, 38, 38) 147456 block_2b_relu_1[0][0] __________________________________________________________________________________________________ block_2b_conv_shortcut (Conv2D) (None, 128, 38, 38) 16384 block_2a_relu[0][0] __________________________________________________________________________________________________ block_2b_bn_2 (BatchNormalizati (None, 128, 38, 38) 512 block_2b_conv_2[0][0] __________________________________________________________________________________________________ block_2b_bn_shortcut (BatchNorm (None, 128, 38, 38) 512 block_2b_conv_shortcut[0][0] __________________________________________________________________________________________________ add_4 (Add) (None, 128, 38, 38) 0 block_2b_bn_2[0][0] block_2b_bn_shortcut[0][0] __________________________________________________________________________________________________ block_2b_relu (Activation) (None, 128, 38, 38) 0 add_4[0][0] __________________________________________________________________________________________________ block_3a_conv_1 (Conv2D) (None, 256, 19, 19) 294912 block_2b_relu[0][0] __________________________________________________________________________________________________ block_3a_bn_1 (BatchNormalizati (None, 256, 19, 19) 1024 block_3a_conv_1[0][0] __________________________________________________________________________________________________ block_3a_relu_1 (Activation) (None, 256, 19, 19) 0 block_3a_bn_1[0][0] __________________________________________________________________________________________________ block_3a_conv_2 (Conv2D) (None, 256, 19, 19) 589824 block_3a_relu_1[0][0] __________________________________________________________________________________________________ block_3a_conv_shortcut (Conv2D) (None, 256, 19, 19) 32768 block_2b_relu[0][0] __________________________________________________________________________________________________ block_3a_bn_2 (BatchNormalizati (None, 256, 19, 19) 1024 block_3a_conv_2[0][0] __________________________________________________________________________________________________ block_3a_bn_shortcut (BatchNorm (None, 256, 19, 19) 1024 block_3a_conv_shortcut[0][0] __________________________________________________________________________________________________ add_5 (Add) (None, 256, 19, 19) 0 block_3a_bn_2[0][0] block_3a_bn_shortcut[0][0] __________________________________________________________________________________________________ block_3a_relu (Activation) (None, 256, 19, 19) 0 add_5[0][0] __________________________________________________________________________________________________ block_3b_conv_1 (Conv2D) (None, 256, 19, 19) 589824 block_3a_relu[0][0] __________________________________________________________________________________________________ block_3b_bn_1 (BatchNormalizati (None, 256, 19, 19) 1024 block_3b_conv_1[0][0] __________________________________________________________________________________________________ block_3b_relu_1 (Activation) (None, 256, 19, 19) 0 block_3b_bn_1[0][0] __________________________________________________________________________________________________ block_3b_conv_2 (Conv2D) (None, 256, 19, 19) 589824 block_3b_relu_1[0][0] __________________________________________________________________________________________________ block_3b_conv_shortcut (Conv2D) (None, 256, 19, 19) 65536 block_3a_relu[0][0] __________________________________________________________________________________________________ block_3b_bn_2 (BatchNormalizati (None, 256, 19, 19) 1024 block_3b_conv_2[0][0] __________________________________________________________________________________________________ block_3b_bn_shortcut (BatchNorm (None, 256, 19, 19) 1024 block_3b_conv_shortcut[0][0] __________________________________________________________________________________________________ add_6 (Add) (None, 256, 19, 19) 0 block_3b_bn_2[0][0] block_3b_bn_shortcut[0][0] __________________________________________________________________________________________________ block_3b_relu (Activation) (None, 256, 19, 19) 0 add_6[0][0] __________________________________________________________________________________________________ block_4a_conv_1 (Conv2D) (None, 512, 19, 19) 1179648 block_3b_relu[0][0] __________________________________________________________________________________________________ block_4a_bn_1 (BatchNormalizati (None, 512, 19, 19) 2048 block_4a_conv_1[0][0] __________________________________________________________________________________________________ block_4a_relu_1 (Activation) (None, 512, 19, 19) 0 block_4a_bn_1[0][0] __________________________________________________________________________________________________ block_4a_conv_2 (Conv2D) (None, 512, 19, 19) 2359296 block_4a_relu_1[0][0] __________________________________________________________________________________________________ block_4a_conv_shortcut (Conv2D) (None, 512, 19, 19) 131072 block_3b_relu[0][0] __________________________________________________________________________________________________ block_4a_bn_2 (BatchNormalizati (None, 512, 19, 19) 2048 block_4a_conv_2[0][0] __________________________________________________________________________________________________ block_4a_bn_shortcut (BatchNorm (None, 512, 19, 19) 2048 block_4a_conv_shortcut[0][0] __________________________________________________________________________________________________ add_7 (Add) (None, 512, 19, 19) 0 block_4a_bn_2[0][0] block_4a_bn_shortcut[0][0] __________________________________________________________________________________________________ block_4a_relu (Activation) (None, 512, 19, 19) 0 add_7[0][0] __________________________________________________________________________________________________ block_4b_conv_1 (Conv2D) (None, 512, 19, 19) 2359296 block_4a_relu[0][0] __________________________________________________________________________________________________ block_4b_bn_1 (BatchNormalizati (None, 512, 19, 19) 2048 block_4b_conv_1[0][0] __________________________________________________________________________________________________ block_4b_relu_1 (Activation) (None, 512, 19, 19) 0 block_4b_bn_1[0][0] __________________________________________________________________________________________________ block_4b_conv_2 (Conv2D) (None, 512, 19, 19) 2359296 block_4b_relu_1[0][0] __________________________________________________________________________________________________ block_4b_conv_shortcut (Conv2D) (None, 512, 19, 19) 262144 block_4a_relu[0][0] __________________________________________________________________________________________________ block_4b_bn_2 (BatchNormalizati (None, 512, 19, 19) 2048 block_4b_conv_2[0][0] __________________________________________________________________________________________________ block_4b_bn_shortcut (BatchNorm (None, 512, 19, 19) 2048 block_4b_conv_shortcut[0][0] __________________________________________________________________________________________________ add_8 (Add) (None, 512, 19, 19) 0 block_4b_bn_2[0][0] block_4b_bn_shortcut[0][0] __________________________________________________________________________________________________ block_4b_relu (Activation) (None, 512, 19, 19) 0 add_8[0][0] __________________________________________________________________________________________________ ssd_expand_block_0_conv_0 (Conv (None, 256, 19, 19) 131328 block_4b_relu[0][0] __________________________________________________________________________________________________ ssd_expand_block_0_relu_0 (ReLU (None, 256, 19, 19) 0 ssd_expand_block_0_conv_0[0][0] __________________________________________________________________________________________________ ssd_expand_block_0_conv_1 (Conv (None, 256, 19, 19) 589824 ssd_expand_block_0_relu_0[0][0] __________________________________________________________________________________________________ ssd_expand_block_0_bn_1 (BatchN (None, 256, 19, 19) 1024 ssd_expand_block_0_conv_1[0][0] __________________________________________________________________________________________________ ssd_expand_block_0_relu_1 (ReLU (None, 256, 19, 19) 0 ssd_expand_block_0_bn_1[0][0] __________________________________________________________________________________________________ ssd_expand_block_1_conv_0 (Conv (None, 128, 19, 19) 32896 ssd_expand_block_0_relu_1[0][0] __________________________________________________________________________________________________ ssd_expand_block_1_relu_0 (ReLU (None, 128, 19, 19) 0 ssd_expand_block_1_conv_0[0][0] __________________________________________________________________________________________________ ssd_expand_block_1_conv_1 (Conv (None, 256, 10, 10) 294912 ssd_expand_block_1_relu_0[0][0] __________________________________________________________________________________________________ ssd_expand_block_1_bn_1 (BatchN (None, 256, 10, 10) 1024 ssd_expand_block_1_conv_1[0][0] __________________________________________________________________________________________________ ssd_expand_block_1_relu_1 (ReLU (None, 256, 10, 10) 0 ssd_expand_block_1_bn_1[0][0] __________________________________________________________________________________________________ ssd_expand_block_2_conv_0 (Conv (None, 64, 10, 10) 16448 ssd_expand_block_1_relu_1[0][0] __________________________________________________________________________________________________ ssd_expand_block_2_relu_0 (ReLU (None, 64, 10, 10) 0 ssd_expand_block_2_conv_0[0][0] __________________________________________________________________________________________________ ssd_expand_block_2_conv_1 (Conv (None, 128, 5, 5) 73728 ssd_expand_block_2_relu_0[0][0] __________________________________________________________________________________________________ ssd_expand_block_2_bn_1 (BatchN (None, 128, 5, 5) 512 ssd_expand_block_2_conv_1[0][0] __________________________________________________________________________________________________ ssd_expand_block_2_relu_1 (ReLU (None, 128, 5, 5) 0 ssd_expand_block_2_bn_1[0][0] __________________________________________________________________________________________________ ssd_expand_block_3_conv_0 (Conv (None, 64, 5, 5) 8256 ssd_expand_block_2_relu_1[0][0] __________________________________________________________________________________________________ ssd_expand_block_3_relu_0 (ReLU (None, 64, 5, 5) 0 ssd_expand_block_3_conv_0[0][0] __________________________________________________________________________________________________ ssd_expand_block_3_conv_1 (Conv (None, 128, 3, 3) 73728 ssd_expand_block_3_relu_0[0][0] __________________________________________________________________________________________________ ssd_expand_block_3_bn_1 (BatchN (None, 128, 3, 3) 512 ssd_expand_block_3_conv_1[0][0] __________________________________________________________________________________________________ ssd_expand_block_3_relu_1 (ReLU (None, 128, 3, 3) 0 ssd_expand_block_3_bn_1[0][0] __________________________________________________________________________________________________ ssd_expand_block_4_conv_0 (Conv (None, 64, 3, 3) 8256 ssd_expand_block_3_relu_1[0][0] __________________________________________________________________________________________________ ssd_expand_block_4_relu_0 (ReLU (None, 64, 3, 3) 0 ssd_expand_block_4_conv_0[0][0] __________________________________________________________________________________________________ ssd_expand_block_4_conv_1 (Conv (None, 128, 2, 2) 73728 ssd_expand_block_4_relu_0[0][0] __________________________________________________________________________________________________ ssd_expand_block_4_bn_1 (BatchN (None, 128, 2, 2) 512 ssd_expand_block_4_conv_1[0][0] __________________________________________________________________________________________________ ssd_expand_block_4_relu_1 (ReLU (None, 128, 2, 2) 0 ssd_expand_block_4_bn_1[0][0] __________________________________________________________________________________________________ ssd_conf_0 (Conv2D) (None, 24, 38, 38) 27672 block_2b_relu[0][0] __________________________________________________________________________________________________ ssd_conf_1 (Conv2D) (None, 24, 19, 19) 55320 ssd_expand_block_0_relu_1[0][0] __________________________________________________________________________________________________ ssd_conf_2 (Conv2D) (None, 24, 10, 10) 55320 ssd_expand_block_1_relu_1[0][0] __________________________________________________________________________________________________ ssd_conf_3 (Conv2D) (None, 24, 5, 5) 27672 ssd_expand_block_2_relu_1[0][0] __________________________________________________________________________________________________ ssd_conf_4 (Conv2D) (None, 24, 3, 3) 27672 ssd_expand_block_3_relu_1[0][0] __________________________________________________________________________________________________ ssd_conf_5 (Conv2D) (None, 24, 2, 2) 27672 ssd_expand_block_4_relu_1[0][0] __________________________________________________________________________________________________ permute_1 (Permute) (None, 38, 38, 24) 0 ssd_conf_0[0][0] __________________________________________________________________________________________________ permute_2 (Permute) (None, 19, 19, 24) 0 ssd_conf_1[0][0] __________________________________________________________________________________________________ permute_3 (Permute) (None, 10, 10, 24) 0 ssd_conf_2[0][0] __________________________________________________________________________________________________ permute_4 (Permute) (None, 5, 5, 24) 0 ssd_conf_3[0][0] __________________________________________________________________________________________________ permute_5 (Permute) (None, 3, 3, 24) 0 ssd_conf_4[0][0] __________________________________________________________________________________________________ permute_6 (Permute) (None, 2, 2, 24) 0 ssd_conf_5[0][0] __________________________________________________________________________________________________ conf_reshape_0 (Reshape) (None, 8664, 1, 4) 0 permute_1[0][0] __________________________________________________________________________________________________ conf_reshape_1 (Reshape) (None, 2166, 1, 4) 0 permute_2[0][0] __________________________________________________________________________________________________ conf_reshape_2 (Reshape) (None, 600, 1, 4) 0 permute_3[0][0] __________________________________________________________________________________________________ conf_reshape_3 (Reshape) (None, 150, 1, 4) 0 permute_4[0][0] __________________________________________________________________________________________________ conf_reshape_4 (Reshape) (None, 54, 1, 4) 0 permute_5[0][0] __________________________________________________________________________________________________ conf_reshape_5 (Reshape) (None, 24, 1, 4) 0 permute_6[0][0] __________________________________________________________________________________________________ mbox_conf (Concatenate) (None, 11658, 1, 4) 0 conf_reshape_0[0][0] conf_reshape_1[0][0] conf_reshape_2[0][0] conf_reshape_3[0][0] conf_reshape_4[0][0] conf_reshape_5[0][0] __________________________________________________________________________________________________ ssd_loc_0 (Conv2D) (None, 24, 38, 38) 27672 block_2b_relu[0][0] __________________________________________________________________________________________________ ssd_loc_1 (Conv2D) (None, 24, 19, 19) 55320 ssd_expand_block_0_relu_1[0][0] __________________________________________________________________________________________________ ssd_loc_2 (Conv2D) (None, 24, 10, 10) 55320 ssd_expand_block_1_relu_1[0][0] __________________________________________________________________________________________________ ssd_loc_3 (Conv2D) (None, 24, 5, 5) 27672 ssd_expand_block_2_relu_1[0][0] __________________________________________________________________________________________________ ssd_loc_4 (Conv2D) (None, 24, 3, 3) 27672 ssd_expand_block_3_relu_1[0][0] __________________________________________________________________________________________________ ssd_loc_5 (Conv2D) (None, 24, 2, 2) 27672 ssd_expand_block_4_relu_1[0][0] __________________________________________________________________________________________________ before_softmax_permute (Permute (None, 4, 1, 11658) 0 mbox_conf[0][0] __________________________________________________________________________________________________ permute_7 (Permute) (None, 38, 38, 24) 0 ssd_loc_0[0][0] __________________________________________________________________________________________________ permute_8 (Permute) (None, 19, 19, 24) 0 ssd_loc_1[0][0] __________________________________________________________________________________________________ permute_9 (Permute) (None, 10, 10, 24) 0 ssd_loc_2[0][0] __________________________________________________________________________________________________ permute_10 (Permute) (None, 5, 5, 24) 0 ssd_loc_3[0][0] __________________________________________________________________________________________________ permute_11 (Permute) (None, 3, 3, 24) 0 ssd_loc_4[0][0] __________________________________________________________________________________________________ permute_12 (Permute) (None, 2, 2, 24) 0 ssd_loc_5[0][0] __________________________________________________________________________________________________ ssd_anchor_0 (AnchorBoxes) (None, 1444, 6, 8) 0 ssd_loc_0[0][0] __________________________________________________________________________________________________ ssd_anchor_1 (AnchorBoxes) (None, 361, 6, 8) 0 ssd_loc_1[0][0] __________________________________________________________________________________________________ ssd_anchor_2 (AnchorBoxes) (None, 100, 6, 8) 0 ssd_loc_2[0][0] __________________________________________________________________________________________________ ssd_anchor_3 (AnchorBoxes) (None, 25, 6, 8) 0 ssd_loc_3[0][0] __________________________________________________________________________________________________ ssd_anchor_4 (AnchorBoxes) (None, 9, 6, 8) 0 ssd_loc_4[0][0] __________________________________________________________________________________________________ ssd_anchor_5 (AnchorBoxes) (None, 4, 6, 8) 0 ssd_loc_5[0][0] __________________________________________________________________________________________________ mbox_conf_softmax_ (Softmax) (None, 4, 1, 11658) 0 before_softmax_permute[0][0] __________________________________________________________________________________________________ loc_reshape_0 (Reshape) (None, 8664, 1, 4) 0 permute_7[0][0] __________________________________________________________________________________________________ loc_reshape_1 (Reshape) (None, 2166, 1, 4) 0 permute_8[0][0] __________________________________________________________________________________________________ loc_reshape_2 (Reshape) (None, 600, 1, 4) 0 permute_9[0][0] __________________________________________________________________________________________________ loc_reshape_3 (Reshape) (None, 150, 1, 4) 0 permute_10[0][0] __________________________________________________________________________________________________ loc_reshape_4 (Reshape) (None, 54, 1, 4) 0 permute_11[0][0] __________________________________________________________________________________________________ loc_reshape_5 (Reshape) (None, 24, 1, 4) 0 permute_12[0][0] __________________________________________________________________________________________________ anchor_reshape_0 (Reshape) (None, 8664, 1, 8) 0 ssd_anchor_0[0][0] __________________________________________________________________________________________________ anchor_reshape_1 (Reshape) (None, 2166, 1, 8) 0 ssd_anchor_1[0][0] __________________________________________________________________________________________________ anchor_reshape_2 (Reshape) (None, 600, 1, 8) 0 ssd_anchor_2[0][0] __________________________________________________________________________________________________ anchor_reshape_3 (Reshape) (None, 150, 1, 8) 0 ssd_anchor_3[0][0] __________________________________________________________________________________________________ anchor_reshape_4 (Reshape) (None, 54, 1, 8) 0 ssd_anchor_4[0][0] __________________________________________________________________________________________________ anchor_reshape_5 (Reshape) (None, 24, 1, 8) 0 ssd_anchor_5[0][0] __________________________________________________________________________________________________ mbox_conf_softmax (Permute) (None, 11658, 1, 4) 0 mbox_conf_softmax_[0][0] __________________________________________________________________________________________________ mbox_loc (Concatenate) (None, 11658, 1, 4) 0 loc_reshape_0[0][0] loc_reshape_1[0][0] loc_reshape_2[0][0] loc_reshape_3[0][0] loc_reshape_4[0][0] loc_reshape_5[0][0] __________________________________________________________________________________________________ mbox_priorbox (Concatenate) (None, 11658, 1, 8) 0 anchor_reshape_0[0][0] anchor_reshape_1[0][0] anchor_reshape_2[0][0] anchor_reshape_3[0][0] anchor_reshape_4[0][0] anchor_reshape_5[0][0] __________________________________________________________________________________________________ concatenate_1 (Concatenate) (None, 11658, 1, 16) 0 mbox_conf_softmax[0][0] mbox_loc[0][0] mbox_priorbox[0][0] __________________________________________________________________________________________________ ssd_predictions (Reshape) (None, 11658, 16) 0 concatenate_1[0][0] ================================================================================================== Total params: 13,291,808 Trainable params: 13,268,960 Non-trainable params: 22,848 __________________________________________________________________________________________________ 2021-06-16 13:44:03,972 [INFO] __main__: Number of images in the training dataset: 6733 2021-06-16 13:44:03,972 [INFO] __main__: Number of images in the validation dataset: 748 Epoch 1/20 2/421 [..............................] - ETA: 1:05:16 - loss: 45.7689/usr/local/lib/python3.6/dist-packages/keras/callbacks.py:122: UserWarning: Method on_batch_end() is slow compared to the batch update (1.064897). Check your callbacks. 421/421 [==============================] - 210s 500ms/step - loss: 18.2864 5c90b0b24053:70:114 [0] NCCL INFO Bootstrap : Using [0]lo:127.0.0.1<0> [1]eth0:172.17.0.2<0> 5c90b0b24053:70:114 [0] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation 5c90b0b24053:70:114 [0] NCCL INFO NET/IB : No device found. 5c90b0b24053:70:114 [0] NCCL INFO NET/Socket : Using [0]lo:127.0.0.1<0> [1]eth0:172.17.0.2<0> 5c90b0b24053:70:114 [0] NCCL INFO Using network Socket NCCL version 2.7.8+cuda11.1 5c90b0b24053:70:114 [0] NCCL INFO Channel 00/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 01/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 02/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 03/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 04/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 05/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 06/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 07/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 08/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 09/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 10/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 11/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 12/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 13/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 14/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 15/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 16/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 17/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 18/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 19/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 20/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 21/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 22/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 23/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 24/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 25/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 26/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 27/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 28/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 29/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 30/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Channel 31/32 : 0 5c90b0b24053:70:114 [0] NCCL INFO Trees [0] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [1] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [2] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [3] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [4] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [5] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [6] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [7] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [8] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [9] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [10] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [11] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [12] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [13] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [14] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [15] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [16] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [17] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [18] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [19] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [20] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [21] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [22] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [23] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [24] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [25] -1/-1/-1->0->-1|-1->0->-1/-1/-1 [26] -1/-1/-1->0->-1|-1->0->-1/ 5c90b0b24053:70:114 [0] NCCL INFO Setting affinity for GPU 0 to 0fffff 5c90b0b24053:70:114 [0] NCCL INFO 32 coll channels, 32 p2p channels, 32 p2p channels per peer 5c90b0b24053:70:114 [0] NCCL INFO comm 0x7fb822f95110 rank 0 nranks 1 cudaDev 0 busId 65000 - Init COMPLETE Epoch 00001: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_001.tlt Epoch 2/20 421/421 [==============================] - 188s 448ms/step - loss: 12.2559 Epoch 00002: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_002.tlt Producing predictions: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47/47 [00:13<00:00, 3.55it/s] Start to calculate AP for each class ******************************* car AP 0.38762 cyclist AP 0.0 pedestrian AP 0.00489 mAP 0.13084 ******************************* Validation loss: 78.6052410513322 Epoch 3/20 421/421 [==============================] - 192s 457ms/step - loss: 10.7979 Epoch 00003: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_003.tlt Epoch 4/20 421/421 [==============================] - 190s 452ms/step - loss: 9.7997 Epoch 00004: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_004.tlt Producing predictions: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47/47 [00:11<00:00, 4.05it/s] Start to calculate AP for each class ******************************* car AP 0.68893 cyclist AP 0.00142 pedestrian AP 0.11212 mAP 0.26749 ******************************* Validation loss: 61.192934821633735 Epoch 5/20 421/421 [==============================] - 191s 453ms/step - loss: 8.7171 Epoch 00005: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_005.tlt Epoch 6/20 421/421 [==============================] - 190s 450ms/step - loss: 7.9509 Epoch 00006: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_006.tlt Producing predictions: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47/47 [00:11<00:00, 4.04it/s] Start to calculate AP for each class ******************************* car AP 0.71912 cyclist AP 0.00126 pedestrian AP 0.17928 mAP 0.29989 ******************************* Validation loss: 57.19995823008492 Epoch 7/20 421/421 [==============================] - 189s 450ms/step - loss: 7.3942 Epoch 00007: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_007.tlt Epoch 8/20 421/421 [==============================] - 191s 453ms/step - loss: 6.9229 Epoch 00008: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_008.tlt Producing predictions: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47/47 [00:11<00:00, 4.00it/s] Start to calculate AP for each class ******************************* car AP 0.74931 cyclist AP 0.0101 pedestrian AP 0.16613 mAP 0.30851 ******************************* Validation loss: 54.9457478650751 Epoch 9/20 421/421 [==============================] - 194s 462ms/step - loss: 6.5240 Epoch 00009: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_009.tlt Epoch 10/20 421/421 [==============================] - 188s 446ms/step - loss: 6.1547 Epoch 00010: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_010.tlt Producing predictions: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47/47 [00:11<00:00, 4.02it/s] Start to calculate AP for each class ******************************* car AP 0.77438 cyclist AP 0.03636 pedestrian AP 0.13926 mAP 0.31667 ******************************* Validation loss: 50.45935105512486 Epoch 11/20 421/421 [==============================] - 194s 460ms/step - loss: 5.9282 Epoch 00011: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_011.tlt Epoch 12/20 421/421 [==============================] - 190s 452ms/step - loss: 5.7086 Epoch 00012: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_012.tlt Producing predictions: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47/47 [00:11<00:00, 3.95it/s] Start to calculate AP for each class ******************************* car AP 0.76995 cyclist AP 0.04046 pedestrian AP 0.15561 mAP 0.322 ******************************* Validation loss: 52.74628642281109 Epoch 13/20 421/421 [==============================] - 196s 465ms/step - loss: 5.5302 Epoch 00013: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_013.tlt Epoch 14/20 421/421 [==============================] - 192s 457ms/step - loss: 5.3936 Epoch 00014: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_014.tlt Producing predictions: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47/47 [00:12<00:00, 3.88it/s] Start to calculate AP for each class ******************************* car AP 0.75034 cyclist AP 0.10046 pedestrian AP 0.15908 mAP 0.33662 ******************************* Validation loss: 52.46265849965141 Epoch 15/20 421/421 [==============================] - 195s 463ms/step - loss: 5.2835 Epoch 00015: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_015.tlt Epoch 16/20 421/421 [==============================] - 193s 459ms/step - loss: 5.1598 Epoch 00016: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_016.tlt Producing predictions: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47/47 [00:11<00:00, 3.95it/s] Start to calculate AP for each class ******************************* car AP 0.77775 cyclist AP 0.10227 pedestrian AP 0.19898 mAP 0.35967 ******************************* Validation loss: 52.400096158930324 Epoch 17/20 421/421 [==============================] - 192s 457ms/step - loss: 4.8734 Epoch 00017: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_017.tlt Epoch 18/20 421/421 [==============================] - 193s 458ms/step - loss: 4.6081 Epoch 00018: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_018.tlt Producing predictions: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47/47 [00:11<00:00, 3.95it/s] Start to calculate AP for each class ******************************* car AP 0.81842 cyclist AP 0.03006 pedestrian AP 0.29078 mAP 0.37975 ******************************* Validation loss: 42.67810399264575 Epoch 19/20 421/421 [==============================] - 193s 458ms/step - loss: 4.5491 Epoch 00019: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_019.tlt Epoch 20/20 421/421 [==============================] - 193s 458ms/step - loss: 4.5192 Epoch 00020: saving model to /ssd/experiment_dir_unpruned/weights/ssd_resnet18_epoch_020.tlt Producing predictions: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47/47 [00:11<00:00, 3.96it/s] Start to calculate AP for each class ******************************* car AP 0.82053 cyclist AP 0.04547 pedestrian AP 0.30603 mAP 0.39067 ******************************* Validation loss: 42.06282478985302 2021-06-16 07:55:21,568 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.