2 citations · 2 across the 4 of their papers we have counts for
7 papers · 1 filter
Understanding GPU Resource Interference One Level Deeper
Paul Elvinger, Foteini Strati, Natalie Enright Jerger +1
GPUs are vastly underutilized, even when running resource-intensive AI applications, as GPU kernels within each job have diverse resource profiles that may saturate some parts of a…
ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
Youhe Jiang, Fangcheng Fu, Xiaozhe Yao +4
Recent developments in large language models (LLMs) have demonstrated their remarkable proficiency in a range of tasks. Compared to in-house homogeneous GPU clusters, deploying LLM…
Unlocking True Elasticity for the Cloud-Native Era with Dandelion
Tom Kuchler, Pinghe Li, Yazhuo Zhang +8
Elasticity is fundamental to cloud computing, as it enables quickly allocating resources to match the demand of each workload as it arrives, rather than pre-provisioning resources…
Sailor: Automating Distributed Training over Dynamic, Heterogeneous, and Geo-distributed Clusters
Foteini Strati, Zhendong Zhang, George Manos +7
The high GPU demand of ML training makes it hard to allocate large homogeneous clusters of high-end GPUs in a single availability zone. Leveraging heterogeneous GPUs available with…
Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
Youhe Jiang, Fangcheng Fu, Xiaozhe Yao +6
Recent advancements in Large Language Models (LLMs) have led to increasingly diverse requests, accompanied with varying resource (compute and memory) demands to serve them. However…
DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs
Xiaozhe Yao, Qinghao Hu, Ana Klimovic
Fine-tuning large language models (LLMs) greatly improves model quality for downstream tasks. However, serving many fine-tuned LLMs concurrently is challenging due to the sporadic,…