1 citations · 2 across the 6 of their papers we have counts for
7 papers
Coral: Cost-Efficient Multi-LLM Serving over Heterogeneous Cloud GPUs
Yixuan Mei, Zikun Li, Zixuan Chen +5
The landscape of large language models (LLMs) has become increasingly fragmented, with no single model dominating the field. Meanwhile, cloud providers offer a diverse array of mid…
Prism: Symbolic Superoptimization of Tensor Programs
Mengdi Wu, Xiaoyu Jiang, Oded Padon +1
This paper presents Prism, the first symbolic superoptimizer for tensor programs. The key idea is sGraph, a symbolic, hierarchical representation that compactly encodes large class…
MPK: A Compiler and Runtime for Mega-Kernelizing Tensor Programs
Xinhao Cheng, Zhihao Zhang, Yu Zhou +17
We introduce Mirage Persistent Kernel (MPK), the first compiler and runtime system that automatically transforms multi-GPU model inference into a single high-performance mega-kerne…
Identity Testing for Circuits with Exponentiation Gates
Jiatu Li, Mengdi Wu
Motivated by practical applications in the design of optimization compilers for neural networks, we initiated the study of identity testing problems for arithmetic circuits augment…
GraphPipe: Improving Performance and Scalability of DNN Training with Graph Pipeline Parallelism
Byungsoo Jeon, Mengdi Wu, Shiyi Cao +11
Deep neural networks (DNNs) continue to grow rapidly in size, making them infeasible to train on a single device. Pipeline parallelism is commonly used in existing DNN systems to s…
Mirage: A Multi-Level Superoptimizer for Tensor Programs
Mengdi Wu, Xinhao Cheng, Shengyu Liu +7
We introduce Mirage, the first multi-level superoptimizer for tensor programs. A key idea in Mirage is Graphs, a uniform representation of tensor programs at the kernel, thread…
FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees
Gabriele Oliaro, Xupeng Miao, Xinhao Cheng +9
Finetuning large language models (LLMs) is essential for task adaptation, yet today's serving stacks isolate inference and finetuning on separate GPU clusters -- wasting resources…