I encounter a problem with the pruning of the PeopleNet_2.6 model, I trained the model with a dataset of person, the model after training has an accuracy of 88%.
tao detectnet_v2 evaluate -e /raid/train_experiment/tao_experiments/person/person-interieur-fevrier-with-v2.6/model_pruned/result-after-pruning/experiment_spec.txt -m /raid/train_experiment/tao_experiments/person/person-interieur-fevrier-with-v2.6/model.step-9345.tlt -k tlt_encode
Total params: 22,240,663
Trainable params: 13,661,445
Non-trainable params: 8,579,218
__________________________________________________________________________________________________
2023-02-17 15:41:49,874 [INFO] root: Model constructed.
2023-02-17 15:41:49,875 [INFO] root: {
"size": 85.49764251708984,
"param_count": 22.240663
}
INFO:tensorflow:Graph was finalized.
2023-02-17 15:41:51,906 [INFO] tensorflow: Graph was finalized.
INFO:tensorflow:Running local_init_op.
2023-02-17 15:41:54,850 [INFO] tensorflow: Running local_init_op.
INFO:tensorflow:Done running local_init_op.
2023-02-17 15:41:55,577 [INFO] tensorflow: Done running local_init_op.
2023-02-17 15:41:57,866 [INFO] iva.detectnet_v2.evaluation.evaluation: step 0 / 43, 0.00s/step
2023-02-17 15:42:09,880 [INFO] iva.detectnet_v2.evaluation.evaluation: step 10 / 43, 1.20s/step
2023-02-17 15:42:15,135 [INFO] iva.detectnet_v2.evaluation.evaluation: step 20 / 43, 0.53s/step
2023-02-17 15:42:20,726 [INFO] iva.detectnet_v2.evaluation.evaluation: step 30 / 43, 0.56s/step
2023-02-17 15:42:26,841 [INFO] iva.detectnet_v2.evaluation.evaluation: step 40 / 43, 0.61s/step
Matching predictions to ground truth, class 1/1.: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20856/20856 [00:05<00:00, 3798.46it/s]
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:95: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.
2023-02-17 15:42:35,811 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:95: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:98: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.
2023-02-17 15:42:35,812 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:98: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.
Validation cost: 0.000890
Mean average_precision (in %): 85.6119
class name average precision (in %)
------------ --------------------------
person 85.6119
Median Inference Time: 0.011561
2023-02-17 15:42:36,067 [INFO] __main__: Evaluation complete.
Time taken to run __main__:main: 0:01:04.549897.
2023-02-17 16:42:39,510 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.
But when I prune it with different threshold, as soon as it loses a few parameters its accuracy goes to 0%.
With a Pruning ratio = 0.903 i had an accuracy of 0.00250412% and with a Pruning ratio = 0.9612 i had an accuracy of 1.27705%
tao detectnet_v2 evaluate -e /raid/train_experiment/tao_experiments/person/person-interieur-fevrier-with-v2.6/model_pruned/result-after-pruning/experiment_spec.txt -m /raid/train_experiment/tao_experiments/person/person-interieur-fevrier-with-v2.6/model_pruned/model_v1_pruned_0.3.tlt -k tlt_encode
==================================================================================================
Total params: 20,104,975
Trainable params: 11,527,293
Non-trainable params: 8,577,682
__________________________________________________________________________________________________
2023-02-17 16:05:03,074 [INFO] root: Model constructed.
2023-02-17 16:05:03,074 [INFO] root: {
"size": 77.35216522216797,
"param_count": 20.104975
}
INFO:tensorflow:Graph was finalized.
2023-02-17 16:05:04,676 [INFO] tensorflow: Graph was finalized.
INFO:tensorflow:Running local_init_op.
2023-02-17 16:05:06,658 [INFO] tensorflow: Running local_init_op.
INFO:tensorflow:Done running local_init_op.
2023-02-17 16:05:07,262 [INFO] tensorflow: Done running local_init_op.
2023-02-17 16:05:09,576 [INFO] iva.detectnet_v2.evaluation.evaluation: step 0 / 43, 0.00s/step
2023-02-17 16:06:17,605 [INFO] iva.detectnet_v2.evaluation.evaluation: step 10 / 43, 6.80s/step
2023-02-17 16:07:18,499 [INFO] iva.detectnet_v2.evaluation.evaluation: step 20 / 43, 6.09s/step
2023-02-17 16:08:19,844 [INFO] iva.detectnet_v2.evaluation.evaluation: step 30 / 43, 6.13s/step
2023-02-17 16:09:20,409 [INFO] iva.detectnet_v2.evaluation.evaluation: step 40 / 43, 6.06s/step
Matching predictions to ground truth, class 1/1.: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 220636/220636 [00:19<00:00, 11130.96it/s]
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:95: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.
2023-02-17 16:10:05,874 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:95: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:98: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.
2023-02-17 16:10:05,875 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:98: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.
Validation cost: 0.009478
Mean average_precision (in %): 0.0025
class name average precision (in %)
------------ --------------------------
person 0.00250412
Median Inference Time: 0.010796
2023-02-17 16:10:06,016 [INFO] __main__: Evaluation complete.
Time taken to run __main__:main: 0:05:15.901036.
2023-02-17 17:10:08,466 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.
tao detectnet_v2 evaluate -e /raid/train_experiment/tao_experiments/person/person-interieur-fevrier-with-v2.6/model_pruned/result-after-pruning/experiment_spec.txt -m /raid/train_experiment/tao_experiments/person/person-interieur-fevrier-with-v2.6/model_pruned/model_v1_pruned_0.2.tlt -k tlt_encode
Total params: 21,377,983
Trainable params: 12,799,437
Non-trainable params: 8,578,546
__________________________________________________________________________________________________
INFO:tensorflow:Graph was finalized.
2023-02-17 16:15:44,754 [INFO] tensorflow: Graph was finalized.
INFO:tensorflow:Running local_init_op.
2023-02-17 16:15:46,777 [INFO] tensorflow: Running local_init_op.
INFO:tensorflow:Done running local_init_op.
2023-02-17 16:15:47,425 [INFO] tensorflow: Done running local_init_op.
2023-02-17 16:15:49,875 [INFO] iva.detectnet_v2.evaluation.evaluation: step 0 / 43, 0.00s/step
2023-02-17 16:16:04,048 [INFO] iva.detectnet_v2.evaluation.evaluation: step 10 / 43, 1.42s/step
2023-02-17 16:16:11,245 [INFO] iva.detectnet_v2.evaluation.evaluation: step 20 / 43, 0.72s/step
2023-02-17 16:16:18,388 [INFO] iva.detectnet_v2.evaluation.evaluation: step 30 / 43, 0.71s/step
2023-02-17 16:16:25,761 [INFO] iva.detectnet_v2.evaluation.evaluation: step 40 / 43, 0.74s/step
Matching predictions to ground truth, class 1/1.: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42311/42311 [00:05<00:00, 8077.39it/s]
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:95: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.
2023-02-17 16:16:35,157 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:95: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:98: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.
2023-02-17 16:16:35,158 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:98: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.
Validation cost: 0.001681
Mean average_precision (in %): 1.2771
class name average precision (in %)
------------ --------------------------
person 1.27705
Median Inference Time: 0.010996
2023-02-17 16:16:35,301 [INFO] __main__: Evaluation complete.
Time taken to run __main__:main: 0:01:05.641826.
2023-02-17 17:16:37,855 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.
I don’t have this problem by training the Peoplenet 2.1 model that I was using before. I trained it with the same dataset and the same spec file, I just changed the path of the pretrained model, I only lose 3.2% accuracy with a Pruning ratio = 0.098.
model_config {
arch: "resnet"
pretrained_model_file: "/home/tao_experiments/nvidia_pretrained/peopleNet/v2.1/unpruned/resnet34_peoplenet.tlt"
freeze_blocks: 0
freeze_blocks: 1
freeze_blocks: 2
freeze_blocks: 3
all_projections: True
num_layers: 34
use_pooling: False
use_batch_norm: True
dropout_rate: 0.15
training_precision: {
backend_floatx: FLOAT32
}
objective_set: {
cov {}
bbox {
scale: 35.0
offset: 0.5
}
}
}
############################################################
bbox_rasterizer_config {
target_class_config {
key: "person"
value: {
cov_center_x: 0.5
cov_center_y: 0.5
cov_radius_x: 0.4
cov_radius_y: 0.4
bbox_min_radius: 1.0
}
}
deadzone_radius: 0.67
}
############################################################
postprocessing_config {
target_class_config {
key: "person"
value: {
clustering_config {
coverage_threshold: 0.005
dbscan_eps: 0.15
dbscan_min_samples: 0.05
minimum_bounding_box_height: 20
}
}
}
}
############################################################
cost_function_config {
target_classes {
name: "person"
class_weight: 1.0
coverage_foreground_weight: 0.05
objectives {
name: "cov"
initial_weight: 1.0
weight_target: 1.0
}
objectives {
name: "bbox"
initial_weight: 10.0
weight_target: 1.0
}
}
enable_autoweighting: True
max_objective_weight: 0.9999
min_objective_weight: 0.0001
}
############################################################
training_config {
batch_size_per_gpu: 32
num_epochs: 80
enable_qat: True
checkpoint_interval: 5
learning_rate {
soft_start_annealing_schedule {
min_learning_rate: 2e-6#7e-7
max_learning_rate: 7e-4#7e-5
soft_start: 0.1
annealing: 0.7
}
}
regularizer {
type: L1
weight: 3e-9
}
optimizer {
adam {
epsilon: 1e-08
beta1: 0.9
beta2: 0.999
}
}
cost_scaling {
enabled: False
initial_exponent: 20.0
increment: 0.005
decrement: 1.0
}
}
############################################################
augmentation_config {
preprocessing {
output_image_width: 960
output_image_height: 544
output_image_channel: 3
min_bbox_width: 1.0
min_bbox_height: 1.0
}
spatial_augmentation {
hflip_probability: 0.5
vflip_probability: 0.0
zoom_min: 1.0
zoom_max: 1.0
translate_max_x: 8.0
translate_max_y: 8.0
}
color_augmentation {
color_shift_stddev: 0.0
hue_rotation_max: 25.0
saturation_shift_max: 0.2
contrast_scale_max: 0.1
contrast_center: 0.5
}
}
############################################################
evaluation_config {
average_precision_mode: INTEGRATE
validation_period_during_training: 5
first_validation_epoch: 5
minimum_detection_ground_truth_overlap {
key: "person"
value: 0.6
}
evaluation_box_config {
key: "person"
value {
minimum_height: 4
maximum_height: 9999
minimum_width: 4
maximum_width: 9999
}
}
}
############################################################
dataset_config {
data_sources: {
tfrecords_path: "/raid/dataset/dataset_for_tao2/Personne_interieur/all/tfrecords/trainval*"
image_directory_path: "/raid/dataset/dataset_for_tao2/Personne_interieur/all/"
}
image_extension: "jpg"
target_class_mapping {
key: "person"
value: "person"
}
validation_fold: 0
}
tao detectnet_v2 evaluate -e /raid/train_experiment/tao_experiments/person/person-interieur-fevrier-with-v2.6/model_pruned/result-after-pruning/experiment_spec.txt -m /raid/train_experiment/tao_experiments/person/person-interieur-fevrier-with-v2.1/model.step-9345.tlt -k tlt_encode
Total params: 22,240,663
Trainable params: 13,661,445
Non-trainable params: 8,579,218
__________________________________________________________________________________________________
2023-02-17 13:41:24,420 [INFO] root: Model constructed.
2023-02-17 13:41:24,420 [INFO] root: {
"size": 85.49764251708984,
"param_count": 22.240663
}
INFO:tensorflow:Graph was finalized.
2023-02-17 13:41:25,920 [INFO] tensorflow: Graph was finalized.
INFO:tensorflow:Running local_init_op.
2023-02-17 13:41:27,790 [INFO] tensorflow: Running local_init_op.
INFO:tensorflow:Done running local_init_op.
2023-02-17 13:41:28,346 [INFO] tensorflow: Done running local_init_op.
2023-02-17 13:41:30,479 [INFO] iva.detectnet_v2.evaluation.evaluation: step 0 / 43, 0.00s/step
2023-02-17 13:41:40,638 [INFO] iva.detectnet_v2.evaluation.evaluation: step 10 / 43, 1.02s/step
2023-02-17 13:41:45,206 [INFO] iva.detectnet_v2.evaluation.evaluation: step 20 / 43, 0.46s/step
2023-02-17 13:41:50,073 [INFO] iva.detectnet_v2.evaluation.evaluation: step 30 / 43, 0.49s/step
2023-02-17 13:41:54,853 [INFO] iva.detectnet_v2.evaluation.evaluation: step 40 / 43, 0.48s/step
Matching predictions to ground truth, class 1/1.: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16973/16973 [00:02<00:00, 7503.15it/s]
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:95: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.
2023-02-17 13:41:59,424 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:95: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:98: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.
2023-02-17 13:41:59,425 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:98: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.
Validation cost: 0.000859
Mean average_precision (in %): 86.0512
class name average precision (in %)
------------ --------------------------
person 86.0512
Median Inference Time: 0.010335
2023-02-17 13:41:59,566 [INFO] __main__: Evaluation complete.
Time taken to run __main__:main: 0:00:47.573606.
2023-02-17 14:42:01,819 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.
tao detectnet_v2 evaluate -e /raid/train_experiment/tao_experiments/person/person-interieur-fevrier-with-v2.6/model_pruned/result-after-pruning/experiment_spec.txt -m /raid/train_experiment/tao_experiments/person/person-interieur-fevrier-with-v2.1/pruned/model_v1_pruned_0.1.tlt -k tlt_encode
Total params: 2,180,711
Trainable params: 598,053
Non-trainable params: 1,582,658
__________________________________________________________________________________________________
2023-02-17 13:50:36,967 [INFO] root: Model constructed.
2023-02-17 13:50:36,968 [INFO] root: {
"size": 8.975570678710938,
"param_count": 2.180711
}
INFO:tensorflow:Graph was finalized.
2023-02-17 13:50:38,474 [INFO] tensorflow: Graph was finalized.
INFO:tensorflow:Running local_init_op.
2023-02-17 13:50:40,304 [INFO] tensorflow: Running local_init_op.
INFO:tensorflow:Done running local_init_op.
2023-02-17 13:50:40,898 [INFO] tensorflow: Done running local_init_op.
2023-02-17 13:50:43,135 [INFO] iva.detectnet_v2.evaluation.evaluation: step 0 / 43, 0.00s/step
2023-02-17 13:50:52,871 [INFO] iva.detectnet_v2.evaluation.evaluation: step 10 / 43, 0.97s/step
2023-02-17 13:50:56,329 [INFO] iva.detectnet_v2.evaluation.evaluation: step 20 / 43, 0.35s/step
2023-02-17 13:50:59,782 [INFO] iva.detectnet_v2.evaluation.evaluation: step 30 / 43, 0.35s/step
2023-02-17 13:51:03,381 [INFO] iva.detectnet_v2.evaluation.evaluation: step 40 / 43, 0.36s/step
Matching predictions to ground truth, class 1/1.: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16504/16504 [00:02<00:00, 7493.61it/s]
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:95: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.
2023-02-17 13:51:07,549 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:95: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:98: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.
2023-02-17 13:51:07,549 [WARNING] tensorflow: From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:98: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.
Validation cost: 0.000928
Mean average_precision (in %): 82.8259
class name average precision (in %)
------------ --------------------------
person 82.8259
Median Inference Time: 0.006414
2023-02-17 13:51:07,680 [INFO] __main__: Evaluation complete.
Time taken to run __main__:main: 0:00:42.552095.
2023-02-17 14:51:10,042 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.
Is there another method to prune the peopleNet 2.6 model?
Thank you !