I’m currently running through the tutorials and I’m investigating ways to speed up convergence.
Page 115 in the user guides states:
The default is an exponential decay, but you can choose among exponential decay, polynomial decay, exponential decay with warm-up or cyclic schedules for the learning rates.
learning_rate.py but I don’t recognise any cyclic learning rate class.
My question is what is the recommended way to implement cyclic learning rate?