On the Performance Analysis of Momentum Method: A Frequency Domain Perspective

Xianliang Li, Jun Luo, Zhiwei Zheng +5

Momentum-based optimizers are widely adopted for training neural networks. However, the optimal selection of momentum coefficients remains elusive. This uncertainty impedes a clear…