Please provide the following information when requesting support.
Please provide the following information when requesting support.
• Hardware (T4/V100/Xavier/Nano/etc)
NVIDIA RTX A5000
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc)
YOLOv4 Object Detection with ResNet 18 Backbone
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here)
Command ‘tlt’ not found, did you mean:
• Training spec file(If have, please share here)
Training spec (train.yaml)
random_seed: 42
yolov4_config {
big_anchor_shape: "[(40, 30), (50, 25), (80, 50)]"
mid_anchor_shape: "[(20,10), (30, 16), (40, 20)]"
small_anchor_shape: "[(8, 5), (16, 8), (10, 20)]"
box_matching_iou: 0.5
matching_neutral_box_iou: 0.5
arch: "resnet"
nlayers: 18
arch_conv_blocks: 2
loss_loc_weight: 1.0
loss_neg_obj_weights: 1.0
loss_class_weights: 1.0
label_smoothing: 0.0
big_grid_xy_extend: 0.05
mid_grid_xy_extend: 0.1
small_grid_xy_extend: 0.2
freeze_bn: true
freeze_blocks: [0, 1, 2, 3, 4, 5, 6, 7]
force_relu: false
}
training_config {
batch_size_per_gpu: 4
num_epochs: 100
enable_qat: false
checkpoint_interval: 3
learning_rate {
soft_start_cosine_annealing_schedule {
min_learning_rate: 1e-8
max_learning_rate: 1e-5
soft_start: 0.3
}
}
visualizer {
clearml_config {
project: "TAO Toolkit ClearML Demo"
task: "YOLOV4"
tags: "YOLOV4"
tags: "training"
tags: "resnet18"
tags: "unpruned"
},
enabled: True
}
regularizer {
type: L1
weight: 3e-5
}
optimizer {
adam {
epsilon: 1e-7
beta1: 0.9
beta2: 0.999
amsgrad: false
}
}
# Make sure to change the path below to the model to resume training from
#resume_model_path : "/workspace/tao-experiments/object_detection/models/pretrained_resnet18/pretrained_object_detection_vresnet18/resnet_18.hdf5"
pretrain_model_path: "/workspace/tao-experiments/object_detection/models/pretrained_resnet18/pretrained_object_detection_vresnet18/resnet_18.hdf5"
}
eval_config {
average_precision_mode: INTEGRATE
batch_size: 8
matching_iou_threshold: 0.5
}
nms_config {
confidence_threshold: 0.001
clustering_iou_threshold: 0.5
force_on_cpu: false
top_k: 200
}
augmentation_config {
hue: 0.1
saturation: 1.5
exposure:1.5
vertical_flip:0.1
horizontal_flip: 0.5
jitter: 0.4
output_width: 512
output_height: 512
output_channel: 3
randomize_input_shape_period: 3
mosaic_prob: 0.2
mosaic_min_ratio:0.1
}
dataset_config {
data_sources: {
tfrecords_path: "/workspace/tao-experiments/object_detection/data/train/tfrecords/train*"
image_directory_path: "/workspace/tao-experiments/object_detection/data/train/"
}
include_difficult_in_training: true
image_extension: "jpg"
target_class_mapping {
key: "aeroplane"
value: "aeroplane"
}
validation_data_sources: {
tfrecords_path: "/workspace/tao-experiments/object_detection/data/val/tfrecords/val*"
image_directory_path: "/workspace/tao-experiments/object_detection/data/val/"
}
}
This leaves the network structure (truncated below)
2024-01-30 14:36:49,231 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:973: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
Input (InputLayer) (None, 3, 512, 512) 0
__________________________________________________________________________________________________
Input_qdq (QDQ) (None, 3, 512, 512) 1 Input[0][0]
__________________________________________________________________________________________________
conv1 (QuantizedConv2D) (None, 64, 256, 256) 9408 Input_qdq[0][0]
__________________________________________________________________________________________________
bn_conv1 (BatchNormalization) (None, 64, 256, 256) 256 conv1[0][0]
__________________________________________________________________________________________________
activation_2 (ReLU) (None, 64, 256, 256) 0 bn_conv1[0][0]
__________________________________________________________________________________________________
activation_2_qdq (QDQ) (None, 64, 256, 256) 1 activation_2[0][0]
__________________________________________________________________________________________________
block_1a_conv_1 (QuantizedConv2 (None, 64, 128, 128) 36864 activation_2_qdq[0][0]
__________________________________________________________________________________________________
block_1a_bn_1 (BatchNormalizati (None, 64, 128, 128) 256 block_1a_conv_1[0][0]
__________________________________________________________________________________________________
block_1a_relu_1 (ReLU) (None, 64, 128, 128) 0 block_1a_bn_1[0][0]
__________________________________________________________________________________________________
block_1a_relu_1_qdq (QDQ) (None, 64, 128, 128) 1 block_1a_relu_1[0][0]
....
encoded_sm (Concatenate) (None, 12288, 12) 0 sm_anchor[0][0]
sm_bbox_processor[0][0]
__________________________________________________________________________________________________
encoded_detections (Concatenate (None, 16128, 12) 0 encoded_bg[0][0]
encoded_md[0][0]
encoded_sm[0][0]
__________________________________________________________________________________________________
decoded_predictions (YOLOv4Deco (None, 16128, 5) 0 encoded_detections[0][0]
__________________________________________________________________________________________________
NMS (NMSLayer) (None, 200, 6) 0 decoded_predictions[0][0]
==================================================================================================
Total params: 34,824,270
Trainable params: 34,790,646
Non-trainable params: 33,624
__________________________________________________________________________________________________
I have frozen the Batch Norm and all the blocks, why are basically all of the parameters trainable?
Thanks!
