2 papers
cs.AI2026
Motif-Mamba: network motif improved mamba for long-range sequence modeling
Chonghe Hao, Yue Sun, Jian Zhang +4
Efficient long-sequence modeling remains a central challenge for large language models, as self-attention scales quadratically with sequence length. Mamba offers a linear-time alte…
cs.DC2026
MemFine: Memory-Aware Fine-Grained Scheduling for MoE Training
Lu Zhao, Rong Shi, Shaoqing Zhang +21
The training of large-scale Mixture of Experts (MoE) models faces a critical memory bottleneck due to severe load imbalance caused by dynamic token routing. This imbalance leads to…