Improving Layer-wise Adaptive Rate Methods using Trust Ratio Clipping

Jeffrey Fong, Siwei Chen, Kaiqi Chen

Training neural networks with large batch is of fundamental significance to deep learning. Large batch training remarkably reduces the amount of training time but has difficulties…