COREM: Cosine-Relation Momentum Reshaping with Stateful Writeback

Yan Wang, Xiaochuan Wang, Yuxiang Sun

Matrix-valued optimizer states may contain relational structure that is not captured by treating their entries independently. We study whether relations within matrix-valued optimi…