2 papers
cs.LG2026
MpSub: A Momentum -Dimensional Subspace Trust-Region Method for Derivative-Free Fine-Tuning of Large Language Models
Yuyang Wang, Haoyu Yao, Pengcheng Xie
Full-parameter fine-tuning of large language models has substantial memory costs because backpropagation stores activations and gradients. Zeroth-order optimization avoids this by…
math.OC2026
TOBYQA: A Time-Augmented Model-Based Method for Derivative-Free Optimization under Noise and Temporal Drift
Haoyu Yao, Pengcheng Xie
Derivative-free optimization (DFO) is challenging when the observation channel varies over time and evaluations are noisy. Conventional model-based methods assume stationary observ…