Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Rethinking Sequence Compression with Test-time Regression
Sihan Yang, Jie Huang, Hanxuan Li +4
Sequence compression is a fundamental operation and a critical information bottleneck in efficient language and multimodal models, and yet prevailing approaches based on pooling or…
cs.LG2026
Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation
Kexuan Shi, Hanxuan Li, Zeju Qiu +3
We introduce Pion, a spectrum-preserving optimizer for large language model (LLM) training based on orthogonal equivalence transformation. Unlike additive optimizers such as Adam a…