6 citations · 13 across the 14 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
SPRI: SVD-Partitioned Residual Initialization for Data-Constrained MoE Upcycling
Weiqiao Shan, Ruixiang Mao, Yuang Li +10
Mixture-of-Experts (MoE) models enable efficient scaling, but training them from scratch remains prohibitively expensive. MoE upcycling mitigates this cost by converting pretrained…
cs.LG2026
SHAPE: Coalition-Aware Expert Pruning for Sparse Mixture-of-Experts LLMs
Yuhao Zhang
Sparse Mixture-of-Experts (MoE) large language models achieve strong quality with low per-token compute, yet their deployment is often limited by the memory wall: the full expert p…
cs.LG2020★ 1 cited
Learning Architectures from an Extended Search Space for Language Modeling
Yinqiao Li, Chi Hu, Yuhao Zhang +6
Neural architecture search (NAS) has advanced significantly in recent years but most NAS systems restrict search to learning architectures of a recurrent or convolutional cell. In…