Generalization and Optimization of SGD with Lookahead

Kangcheng Li, Yunwen Lei

The Lookahead optimizer enhances deep learning models by employing a dual-weight update mechanism, which has been shown to improve the performance of underlying optimizers such as…