5 papers
SPLASH: Switching Parallel Layouts of Attention with Seamless Handoff for LLM Serving
Chuan Liu, Shuoming Zhang, Zhicheng Li +6
No single way of parallelizing attention serves large language models well under all loads. Low concurrency favors tensor parallelism, many independent requests favor data-parallel…
Decode-Time Grammars: Constrained LLM Generation over a Refinement Order of Grammar Fragments
Shuoming Zhang, Ruiyuan Xu, Haofeng Li +7
Large language models now write a growing share of the world's code, increasingly inside agents and serving systems that compile, execute, or dispatch generated code without line-b…
KLineage: Recovering the Missing When of Kernel Optimization by Deoptimizing Experts
Shuoming Zhang, Qiuchu Yu, Yangyu Zhang +11
LLM-based agents are increasingly used to generate GPU kernels, but they often struggle to determine when an optimization is sound because its required code state and dependencies…
Ave: Guiding Agentic GPU Optimization Using Data-Flow Invariants
Haohui Mai, Xiaoyan Guo, Xiangyun Ding +7
LLM coding agents can generate correct GPU kernels, but their performance still trails expert libraries. Reaching peak throughput requires coordinating low-level optimizations such…
The New Compiler Stack: A Survey on the Synergy of LLMs and Compilers
Shuoming Zhang, Jiacheng Zhao, Qiuchu Yu +4
This survey has provided a systematic overview of the emerging field of LLM-enabled compilation by addressing several key research questions. We first answered how LLMs are being i…