Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation
Hao Gu, Mao-Lin Luo, Zi-Hao Zhou +3
Parameter-efficient continual learning aims to adapt pre-trained models to sequential tasks without forgetting previously acquired knowledge. Most existing approaches treat continu…
cs.LG2026
Decoupled Orthogonal Dynamics: Regularization for Deep Network Optimizers
Hao Chen, Jinghui Yuan, Hanmin Zhang
Is the standard weight decay in AdamW truly optimal? Although AdamW decouples weight decay from adaptive gradient scaling, a fundamental conflict remains: the Radial Tug-of-War. In…
cs.LG2025
Understanding the Generalization of Stochastic Gradient Adam in Learning Neural Networks
Xuan Tang, Han Zhang, Yuan Cao +1
Adam is a popular and widely used adaptive gradient method in deep learning, which has also received tremendous focus in theoretical research. However, most existing theoretical wo…