Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Long-Context Drafting Does Not Require Long-Context Drafters
Aref Jafari, Vansh Bhatia, Matt Qin +5
Speculative decoding pairs a cheap drafter with parallel verification against the target. At ultra-long context it stops paying, because a drafter that attends over the full prefix…
cs.LG2025
FarSkip-Collective: Unhobbling Blocking Communication in Mixture of Experts Models
Yonatan Dukler, Guihong Li, Deval Shah +3
Blocking communication presents a major hurdle in running MoEs efficiently in distributed settings. To address this, we present FarSkip-Collective which modifies the architecture o…
cs.LG2025
Zebra-Llama: Towards Extremely Efficient Hybrid Models
Mingyu Yang, Mehdi Rezagholizadeh, Guihong Li +2
With the growing demand for deploying large language models (LLMs) across diverse applications, improving their inference efficiency is crucial for sustainable and democratized acc…