2 papers
cs.LG2026
QUILT: Rethinking Sparse-Attention Prefill through Shared Query Execution
Zhenduo Zhao, Qihui Zhou, Mingcong Song +7
Sparse attention reduces the cost of long-context attention, but existing kernels typically process queries independently, repeatedly loading and dequantizing KV entries shared acr…
cs.AI2024
Tackling the Dynamicity in a Production LLM Serving System with SOTA Optimizations via Hybrid Prefill/Decode/Verify Scheduling on Efficient Meta-kernels
Mingcong Song, Xinru Tang, Fengfan Hou +9
Meeting growing demands for low latency and cost efficiency in production-grade large language model (LLM) serving systems requires integrating advanced optimization techniques. Ho…