2 papers
cs.DC2026
Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference
Xun Sun, Shaoyuan Chen, Pingchuan Ma +18
Mixture-of-Experts (MoE) serving relies on wide expert parallelism (EP) to aggregate the memory capacity and bandwidth of many GPUs within one inference instance. This efficiency c…
cs.LG2025
Efficient Modular Learning through Naive LoRA Summation: Leveraging Orthogonality in High-Dimensional Models
Zhanhao Cao, Clement Truong, Andrew Lizarraga
Recent advances in large language models are driven by scale, while parameter-efficient fine-tuning (PEFT) enables updating only a small fraction of parameters. Low-Rank Adaptation…