8 citations · 10 across the 7 of their papers we have counts for
7 papers
PithTrain: A Compact and Agent-Native MoE Training System
Ruihang Lai, Hao Kang, Haozhan Tang +6
Mixture-of-Experts (MoE) has become the dominant architecture for frontier language models. To meet this demand, production frameworks have built optimized MoE training stacks over…
Axe: A Simple Unified Layout Abstraction for Machine Learning Compilers
Bohan Hou, Hongyi Jin, Guanjie Wang +7
Scaling modern deep learning workloads demands coordinated placement of data and compute across device meshes, memory hierarchies, and heterogeneous accelerators. We present Axe La…
XGrammar-2: Dynamic and Efficient Structured Generation Engine for Agentic LLMs
Linzhang Li, Yixin Dong, Guanjie Wang +3
Modern LLM agents increasingly rely on dynamic structured generation, such as tool calling and response protocols. Unlike traditional structured generation with static structures,…
MPK: A Compiler and Runtime for Mega-Kernelizing Tensor Programs
Xinhao Cheng, Zhihao Zhang, Yu Zhou +17
We introduce Mirage Persistent Kernel (MPK), the first compiler and runtime system that automatically transforms multi-GPU model inference into a single high-performance mega-kerne…
A System for Microserving of LLMs
Hongyi Jin, Ruihang Lai, Charlie F. Ruan +5
The recent advances in LLMs bring a strong demand for efficient system support to improve overall serving efficiency. As LLM inference scales towards multiple GPUs and even multipl…
XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models
Yixin Dong, Charlie F. Ruan, Yaxing Cai +4
The applications of LLM Agents are becoming increasingly complex and diverse, leading to a high demand for structured outputs that can be parsed into code, structured function call…
Productively Deploying Emerging Models on Emerging Platforms: A Top-Down Approach for Testing and Debugging
Siyuan Feng, Jiawei Liu, Ruihang Lai +4
While existing machine learning (ML) frameworks focus on established platforms, like running CUDA on server-grade GPUs, there have been growing demands to enable emerging AI applic…