101 citations · 164 across the 15 of their papers we have counts for
9 papers · 1 filter
A System for Microserving of LLMs
Hongyi Jin, Ruihang Lai, Charlie F. Ruan +5
The recent advances in LLMs bring a strong demand for efficient system support to improve overall serving efficiency. As LLM inference scales towards multiple GPUs and even multipl…
Atlas: Hierarchical Partitioning for Quantum Circuit Simulation on GPUs (Extended Version)
Mingkuan Xu, Shiyi Cao, Xupeng Miao +2
This paper presents techniques for theoretically and practically efficient and scalable Schrödinger-style quantum circuit simulation. Our approach partitions a quantum circuit into…
Optimal Kernel Orchestration for Tensor Programs with Korch
Muyan Hu, Ashwin Venkatram, Shreyashri Biswas +7
Kernel orchestration is the task of mapping the computation defined in different operators of a deep neural network (DNN) to the execution of GPU kernels on modern hardware platfor…
GraphPipe: Improving Performance and Scalability of DNN Training with Graph Pipeline Parallelism
Byungsoo Jeon, Mengdi Wu, Shiyi Cao +11
Deep neural networks (DNNs) continue to grow rapidly in size, making them infeasible to train on a single device. Pipeline parallelism is commonly used in existing DNN systems to s…
Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow
Yixuan Mei, Yonghao Zhuang, Xupeng Miao +3
This paper introduces Helix, a distributed system for high-throughput, low-latency large language model (LLM) serving in heterogeneous GPU clusters. The key idea behind Helix is to…
Mirage: A Multi-Level Superoptimizer for Tensor Programs
Mengdi Wu, Xinhao Cheng, Shengyu Liu +7
We introduce Mirage, the first multi-level superoptimizer for tensor programs. A key idea in Mirage is Graphs, a uniform representation of tensor programs at the kernel, thread…
Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible Instances
Jiangfei Duan, Ziang Song, Xupeng Miao +5
Deep neural networks (DNNs) are becoming progressively large and costly to train. This paper aims to reduce DNN training costs by leveraging preemptible instances on modern clouds,…
FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees
Gabriele Oliaro, Xupeng Miao, Xinhao Cheng +9
Finetuning large language models (LLMs) is essential for task adaptation, yet today's serving stacks isolate inference and finetuning on separate GPU clusters -- wasting resources…
Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models
Zhengxin Zhang, Dan Zhao, Xupeng Miao +4
Finetuning large language models (LLMs) has been empirically effective on a variety of downstream tasks. Existing approaches to finetuning an LLM either focus on parameter-efficien…