3 papers
cs.CL2026
DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention
Younjoo Lee, Seungkyun Dan, Junghoo Lee +2
Masked diffusion language models enable parallel token decoding, providing a promising alternative to the sequential nature of autoregressive generation. However, their iterative d…
cs.LG2026
From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill
Gunjun Lee, Jiwon Kim, Jaiyoung Park +2
Large Language Model (LLM) inference in production must meet stringent service-level objectives for both time-to-first-token (TTFT) and time-between-token (TBT) while maximizing th…
cs.AR2026
Rethinking LLM Inference Bottlenecks: Insights from Latent Attention and Mixture-of-Experts
Sungmin Yun, Seonyong Park, Hwayong Nam +10
Computational workloads composing traditional transformer models are starkly bifurcated. Multi-Head Attention (MHA) and Grouped-Query Attention are memory-bound due to low arithmet…