Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics
Cong Xu, Wenbin Liang, Mo Yu +7
The rapid scaling of models has led to prohibitively high training and fine-tuning costs. A major factor accounting for memory consumption is the widespread use of stateful optimiz…
cs.LG2024
CAdam: Confidence-Based Optimization for Online Learning
Shaowen Wang, Anan Liu, Jian Xiao +9
Modern recommendation systems frequently employ online learning to dynamically update their models with freshly collected data. The most commonly used optimizer for updating neural…