Alada: Alternating Adaptation of Momentum Method for Memory-Efficient Matrix Optimization

Xiaoyu He, Yu Cai, Jin Jia +3

This work proposes Alada, an adaptive momentum method for stochastic optimization over large-scale matrices. Alada employs a rank-one factorization approach to estimate the second…