Some problems with the pruned model

[fdkfmsd]

• Network Type YOLOv4 Tiny
• TLT Version TAO 5.0

The figure below is the comparison result of yolov4 tiny pruned and pruned before and after retraining.

I use “!tao model yolo_v4_tiny prune -m $USER_EXPERIMENT_DIR/experiment_dir_unpruned/weights/yolov4_cspdarknet_tiny_epoch_$EPOCH.hdf5
-e $SPECS_DIR/yolo_v4_tiny_train_kitti.txt
-o $USER_EXPERIMENT_DIR/experiment_dir_pruned/yolov4_cspdarknet_tiny_pruned.hdf5
-eq intersection
-pth 0.1” to prune the model.

Question1：Why does the retraining pruning model become larger after the pruning model size becomes smaller, so does pruning really reduce the number of parameters?

Question2：The pruned model has no loss in accuracy when QAT is not used, but the accuracy drops a lot after QAT is used. Why?

[Morganh]

@fdkfmsd

Why does the retraining pruning model become larger after the pruning model size becomes smaller, so does pruning really reduce the number of parameters?

Please make sure load_graph is set to true when you run training with a pruned model.

@fdkfmsd

The pruned model has no loss in accuracy when QAT is not used, but the accuracy drops a lot after QAT is used. Why?

From the table you shared, we can see the “retrain pruned QAT model” has no accuracy drop. Also, it is needed to run training if a model is pruned. So, we need to run training against “pruned QAT model”.

[fdkfmsd]

@Morganh

Please make sure load_graph is set to true when you run training with a pruned model.

Where should I add load_graph?
I try to add yolov4_config in yolo_v4_tiny_retrain_kitti.txt but I get error: message type ‘YOLOv4Config’ has no field named ‘load_graph’.
Is there a detailed description of load_graph that I can read?
I don’t see a description of load_graph.

@Morganh

From the table you shared, we can see the “retrain pruned QAT model” has no accuracy drop. Also, it is needed to run training if a model is pruned. So, we need to run training against “pruned QAT model”.

If a model is pruned, I must to run training?

[Morganh]

@fdkfmsd

Where should I add load_graph?

Sorry, ignore this since you are running yolov4_tiny.
Did you set the pruned model to pruned_model_path ?

@fdkfmsd

If a model is pruned, I must to run training?

To regain accuracy, we recommend retraining the pruned model over the same dataset.

[fdkfmsd]

@Morganh

Sorry, ignore this since you are running yolov4_tiny.
Did you set the pruned model to pruned_model_path ?

Yes. I’m run the official yolo_v4_tiny.ipynb.
There is not much difference in accuracy and file size. In general, what changes does retraining a pruned model bring?

[Morganh]

Can you share the training spec file and full training log?

[fdkfmsd]

Sorry, I didn’t save the training log, so I ran again.
yolo_v4_tiny_retrain_kitti.txt (2.3 KB)
retrain_model_log.txt (5.0 MB)

[Morganh]

Thanks for the info. I will check further.

[Morganh]

That is expected. The hdf5 file or tlt file will still be 68M.
After you run exporting to onnx, the onnx file will be 23M.

yolo_v4_tiny export -m yolov4_cspdarknet_tiny_epoch_xxx.hdf5 -o yolov4_cspdarknet_tiny_epoch_xxx.onnx -e yolo_v4_tiny_retrain_kitti.txt -k nvidia_tlt