4 citations · 6 across the 9 of their papers we have counts for
3 papers · 1 filter
MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoE
Soheil Zibakhsh, Mohammad Samragh, Kumari Nishu +3
The generation quality of large language models (LLMs) is often improved by utilizing inference-time sequence-level scaling methods (e.g., Chain-of-Thought). We introduce hyper-par…
Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential
Mohammad Samragh, Arnav Kundu, David Harrison +4
Autoregressive language models are constrained by their inherently sequential nature, generating one token at a time. This paradigm limits inference speed and parallelism, especial…
SPD: Sync-Point Drop for Efficient Tensor Parallelism of Large Language Models
Han-Byul Kim, Duc Hoang, Arnav Kundu +2
With the rapid expansion in the scale of large language models (LLMs), enabling efficient distributed inference across multiple computing units has become increasingly critical. Ho…