Showing 2026Show all
3 papers · 1 filter
cs.LG2026
Learning When to Adapt
Ali Zindari, Xiaowen Jiang, Rotem Mulayoff +1
Low-rank adaptation (LoRA) is a widely used parameter-efficient fine-tuning method, yet its learned correction is static: the same low-rank update is applied to every input. This i…
math.OC2026
A practical randomized trust-region method to escape saddle points in high dimension
Radu-Alexandru Dragomir, Xiaowen Jiang, Bonan Sun +1
Without randomization, escaping the saddle points of requires at least pieces of information about (values, gradients, Hessian-vec…
cs.LG2026
Enhancing LLM Training via Spectral Clipping
Xiaowen Jiang, Andrei Semenov, Sebastian U. Stich
While spectral-based optimizers like Muon operate directly on the spectrum of updates, standard adaptive methods such as AdamW do not account for the spectral structure of weights…