5 papers
SPLASH: Switching Parallel Layouts of Attention with Seamless Handoff for LLM Serving
Chuan Liu, Shuoming Zhang, Zhicheng Li +6
No single way of parallelizing attention serves large language models well under all loads. Low concurrency favors tensor parallelism, many independent requests favor data-parallel…
Decode-Time Grammars: Constrained LLM Generation over a Refinement Order of Grammar Fragments
Shuoming Zhang, Ruiyuan Xu, Haofeng Li +7
Large language models now write a growing share of the world's code, increasingly inside agents and serving systems that compile, execute, or dispatch generated code without line-b…
KLineage: Recovering the Missing When of Kernel Optimization by Deoptimizing Experts
Shuoming Zhang, Qiuchu Yu, Yangyu Zhang +11
LLM-based agents are increasingly used to generate GPU kernels, but they often struggle to determine when an optimization is sound because its required code state and dependencies…
QiMeng-MuPa: Mutual-Supervised Learning for Sequential-to-Parallel Code Translation
Changxin Ke, Rui Zhang, Shuo Wang +11
The rise of GPU-based high-performance computing (HPC) has driven the widespread adoption of parallel programming models such as CUDA. Yet, the inherent complexity of parallel prog…
When Grammar Guides the Attack: Uncovering Control-Plane Vulnerabilities in LLMs with Structured Output
Shuoming Zhang, Jiacheng Zhao, Hanyuan Dong +9
Content Warning: This paper may contain unsafe or harmful content generated by LLMs that may be offensive to readers. Large Language Models (LLMs) increasingly serve as tooling pla…