Measuring the local shape of learning-rate transfer curves

Satoki Ishikawa, Rio Yokota, Ryo Karakida

Understanding the learning-rate (LR) dependence of trained model performance is essential for better optimization of neural networks, but the appropriate size of the LR depends on…