Is there any way to implement weight decay using tao-toolkit for training yolo_v4?

[AnantSakhare]

Please provide the following information when requesting support.

• Hardware :- v100
• Network Type Yolo_v4
• TLT Version :- toolkit_version: 3.22.05

• Training spec file :- I’m using recommended parameters only
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)
Is there any way to implement weight decay using tao-toolkit for training yolo_v4 ?

[Morganh]

TAO YOLOv4 does not implement weight decay. It implements the L2 regularization.
You can use L2 instead.

[AnantSakhare]

@Morganh

TAO YOLOv4 does not implement weight decay. It implements the L2 regularization.

Thank you. but there is an issue with adam & L2 regularization which is not able to achieve proper results like SGD with L2. A more detailed explanation is mentioned in the mentioned article.

Paper :- [1711.05101] Decoupled Weight Decay Regularization

If is there any way I can implement the weight decay in the upcoming days?

[Morganh]

I will sync with internal team further for this request.