Momentum as Residual-Driven Multiplier Correction for Deep Learning Optimization

Zhixin Ren, Yau Lyu, Congrong Li +2

Momentum-based optimizers are widely used in modern deep learning, yet the relations among momentum recursion, update geometry, and acceleration remain only partially understood. W…