5 papers · 1 filter
RankLane: Efficient MoE Prefill on Accelerators with Ranked Expert Capacities
Wentao Ni, Chihao Yu, Hong Cai +4
Mixture-of-experts (MoE) models expand model capacity by activating only a subset of experts for each token. On accelerators with static execution plans, expert capacities and plac…
TileWeave: Heterogeneous Tiling for Sparse Prefill on Mobile SoCs
Wentao Ni, Hong Cai, Mingu Lee +7
Smartphone applications increasingly require large language models to process long contexts, whereas high prefill latency and memory demands constrain local inference. Block-sparse…
Graph Coarsening Enables Escaping Local Optima in Compute-Constrained Nonconvex MAX-CUT
M. Harper Langston, Pierre-David Letourneau, Dalton Jones +7
Because its binary optimization landscape contains exponentially many local optima, MAX-CUT is highly sensitive to initialization. We study Particle Polynomial Relaxation (PPR), a…
Double-P: Hierarchical Top-P Sparse Attention for Long-Context LLMs
Wentao Ni, Kangqi Zhang, Zhongming Yu +7
As long-context inference becomes central to large language models (LLMs), attention over growing key-value caches emerges as a dominant decoding bottleneck, motivating sparse atte…
Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphs
Sudhanshu Agrawal, Risheek Garrepalli, Raghavv Goel +3
Diffusion LLMs (dLLMs) have recently emerged as a powerful alternative to autoregressive LLMs (AR-LLMs) with the potential to operate at significantly higher token-generation rates…