Applying Cyclical Learning Rate to Neural Machine Translation

Choon Meng Lee, Jianfeng Liu, Wei Peng

In training deep learning networks, the optimizer and related learning rate are often used without much thought or with minimal tuning, even though it is crucial in ensuring a fast…