2 papers
cs.LG2026
MpSub: A Momentum -Dimensional Subspace Trust-Region Method for Derivative-Free Fine-Tuning of Large Language Models
Yuyang Wang, Haoyu Yao, Pengcheng Xie
Full-parameter fine-tuning of large language models has substantial memory costs because backpropagation stores activations and gradients. Zeroth-order optimization avoids this by…
cs.LG2026
Constant-Stepsize Stochastic Approximation: Finite-Time Convergence, Gaussian Approximation, and Tail Bounds
Zedong Wang, Yuyang Wang, Ijay Narang +3
Constant-stepsize stochastic approximation (SA) is widely used in learning for computational efficiency, yet the distribution of the iterates is typically intractable. Classical as…