3 papers
cs.DC2026
Axe: A Simple Unified Layout Abstraction for Machine Learning Compilers
Bohan Hou, Hongyi Jin, Guanjie Wang +7
Scaling modern deep learning workloads demands coordinated placement of data and compute across device meshes, memory hierarchies, and heterogeneous accelerators. We present Axe La…
cs.DC2024
A System for Microserving of LLMs
Hongyi Jin, Ruihang Lai, Charlie F. Ruan +5
The recent advances in LLMs bring a strong demand for efficient system support to improve overall serving efficiency. As LLM inference scales towards multiple GPUs and even multipl…
cs.CL2024
XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models
Yixin Dong, Charlie F. Ruan, Yaxing Cai +4
The applications of LLM Agents are becoming increasingly complex and diverse, leading to a high demand for structured outputs that can be parsed into code, structured function call…