MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoE

Soheil Zibakhsh, Mohammad Samragh, Kumari Nishu +3

The generation quality of large language models (LLMs) is often improved by utilizing inference-time sequence-level scaling methods (e.g., Chain-of-Thought). We introduce hyper-par…