6 papers
Coral: Cost-Efficient Multi-LLM Serving over Heterogeneous Cloud GPUs
Yixuan Mei, Zikun Li, Zixuan Chen +5
The usage of large language models (LLMs) has grown increasingly fragmented, with no single model dominating. Meanwhile, cloud providers offer a wide range of mid-tier and older-ge…
Prism: Symbolic Superoptimization of Tensor Programs
Mengdi Wu, Xiaoyu Jiang, Oded Padon +1
This paper presents Prism, the first symbolic superoptimizer for tensor programs. The key idea is sGraph, a symbolic, hierarchical representation that compactly encodes large class…
MPK: A Compiler and Runtime for Mega-Kernelizing Tensor Programs
Xinhao Cheng, Zhihao Zhang, Yu Zhou +17
We introduce Mirage Persistent Kernel (MPK), the first compiler and runtime system that automatically transforms multi-GPU model inference into a single high-performance mega-kerne…
Identity Testing for Circuits with Exponentiation Gates
Jiatu Li, Mengdi Wu
Motivated by practical applications in the design of optimization compilers for neural networks, we initiated the study of identity testing problems for arithmetic circuits augment…
GraphPipe: Improving Performance and Scalability of DNN Training with Graph Pipeline Parallelism
Byungsoo Jeon, Mengdi Wu, Shiyi Cao +11
Deep neural networks (DNNs) continue to grow rapidly in size, making them infeasible to train on a single device. Pipeline parallelism is commonly used in existing DNN systems to s…
Mirage: A Multi-Level Superoptimizer for Tensor Programs
Mengdi Wu, Xinhao Cheng, Shengyu Liu +7
We introduce Mirage, the first multi-level superoptimizer for tensor programs. A key idea in Mirage is Graphs, a uniform representation of tensor programs at the kernel, thread…