Last Iterate Convergence of Incremental Methods and Applications in Continual Learning

Xufeng Cai, Jelena Diakonikolas

Incremental gradient and incremental proximal methods are a fundamental class of optimization algorithms used for solving finite sum problems, broadly studied in the literature. Ye…