36 citations · 110 across the 10 of their papers we have counts for
6 papers · 1 filter
DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
Yinmin Zhong, Shengyu Liu, Junda Chen +5
DistServe improves the performance of large language models (LLMs) serving by disaggregating the prefill and decoding computation. Existing LLM serving systems colocate the two pha…
Oobleck: Resilient Distributed Training of Large Models Using Pipeline Templates
Insu Jang, Zhenning Yang, Zhen Zhang +2
Oobleck enables resilient distributed training of large DNN models with guaranteed fault tolerance. It takes a planning-execution co-design approach, where it first generates a set…
Energy-Efficient GPU Clusters Scheduling for Deep Learning
Diandian Gu, Xintong Xie, Gang Huang +2
Training deep neural networks (DNNs) is a major workload in datacenters today, resulting in a tremendously fast growth of energy consumption. It is important to reduce the energy c…
MuxFlow: Efficient and Safe GPU Sharing in Large-Scale Production Deep Learning Clusters
Yihao Zhao, Xin Liu, Shufan Liu +5
Large-scale GPU clusters are widely-used to speed up both latency-critical (online) and best-effort (offline) deep learning (DL) workloads. However, most DL clusters either dedicat…
FaaSLight: General Application-Level Cold-Start Latency Optimization for Function-as-a-Service in Serverless Computing
Xuanzhe Liu, Jinfeng Wen, Zhenpeng Chen +5
Serverless computing is a popular cloud computing paradigm that frees developers from server management. Function-as-a-Service (FaaS) is the most popular implementation of serverle…
MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud
Zhen Zhang, Shuai Zheng, Yida Wang +5
Existing general purpose frameworks for gigantic model training, i.e., dense models with billions of parameters, cannot scale efficiently on cloud environment with various networki…