1 citations · 1 across the 8 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation
Kexuan Shi, Hanxuan Li, Zeju Qiu +3
We introduce Pion, a spectrum-preserving optimizer for large language model (LLM) training based on orthogonal equivalence transformation. Unlike additive optimizers such as Adam a…
cs.LG2026
Orthogonal Model Merging
Sihan Yang, Kexuan Shi, Weiyang Liu
Merging finetuned Large Language Models (LLMs) has become increasingly important for integrating diverse capabilities into a single unified model. However, prevailing model merging…
cs.LG2025
Model Merging with Functional Dual Anchors
Kexuan Shi, Yandong Wen, Weiyang Liu
Model merging is an efficient post-training strategy for integrating knowledge from multiple finetuned checkpoints of a shared foundation model. Existing methods operate in the par…