3 citations · 7 across the 15 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
Wenyong Zhou, Yuannuo Feng, Yizhe Chen +6
Large language models (LLMs) with mixture-of-experts (MoE) architectures achieve remarkable scalability by sparsely activating a subset of experts per token, yet their frequent exp…
cs.LG2026
Nonparametric Teaching of Attention Learners
Chen Zhang, Jianghui Wang, Bingyang Cheng +7
Attention learners, neural networks built on the attention mechanism, e.g., transformers, excel at learning the implicit relationships that relate sequences to their corresponding…
cs.LG2024
Mixture-of-Subspaces in Low-Rank Adaptation
Taiqiang Wu, Jiahao Wang, Zhe Zhao +1
In this paper, we introduce a subspace-inspired Low-Rank Adaptation (LoRA) method, which is computationally efficient, easy to implement, and readily applicable to large language,…