2 citations · 2 across the 11 of their papers we have counts for
Showing 2026Show all
3 papers · 1 filter
cs.LG2026
Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less
Yuxing Liu, Jianyu Wang, Tong Zhang
Optimizers play an important role in both pretraining and finetuning stages when training large language models (LLMs). In this paper, we present an observation that full finetunin…
cs.LG2026
StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
Dingzhi Yu, Rui Pan, Yuxing Liu +2
Sign-based optimization algorithms, such as SignSGD, have garnered attention for their performance in distributed learning and training large foundation models. Despite their empir…
cs.LG2026
Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences
Jiarui Cao, Zixuan Wei, Yuxin Liu
We reveal a precise mathematical framework about a new family of generative models which we call Gradient Flow Drifting. With this framework, we prove an equivalence between the re…