4 papers
MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE
Zongle Huang, Lei Zhu, Zongyuan Zhan +5
Large Language Models (LLMs) have achieved remarkable success across many applications, with Mixture of Experts (MoE) models demonstrating great potential. Compared to traditional…
Faster and Better LLMs via Latency-Aware Test-Time Scaling
Zili Wang, Tianyu Zhang, Haoli Bai +5
Test-Time Scaling (TTS) has proven effective in improving the performance of Large Language Models (LLMs) during inference. However, existing research has overlooked the efficiency…
Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
Yehui Tang, Xiaosong Li, Fangcheng Liu +19
The surgence of Mixture of Experts (MoE) in Large Language Models promises a small price of execution cost for a much larger model parameter count and learning capacity, because on…
Pangu Ultra MoE: How to Train Your Big MoE on Ascend NPUs
Yehui Tang, Yichun Yin, Yaoyuan Wang +71
Sparse large language models (LLMs) with Mixture of Experts (MoE) and close to a trillion parameters are dominating the realm of most capable language models. However, the massive…