4 papers
SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization
Gunjun Lee, Sehwan Son, Younjoo Lee +2
Weight-only post-training quantization (PTQ) enables the deployment of large language models under tight memory budgets, but accuracy often collapses at 2-3 bits. Existing backprop…
DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention
Younjoo Lee, Seungkyun Dan, Junghoo Lee +2
Masked diffusion language models enable parallel token decoding, providing a promising alternative to the sequential nature of autoregressive generation. However, their iterative d…
From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill
Gunjun Lee, Jiwon Kim, Jaiyoung Park +2
Large Language Model (LLM) inference in production must meet stringent service-level objectives for both time-to-first-token (TTFT) and time-between-token (TBT) while maximizing th…
Rethinking LLM Inference Bottlenecks: Insights from Latent Attention and Mixture-of-Experts
Sungmin Yun, Seonyong Park, Hwayong Nam +10
Computational workloads composing traditional transformer models are starkly bifurcated. Multi-Head Attention (MHA) and Grouped-Query Attention are memory-bound due to low arithmet…