106 citations · 167 across the 26 of their papers we have counts for
14 papers · 1 filter
Mosaic: GPU Sharing with Latency Guarantees through Kernel-Level Interference Prediction
Foteini Strati, Ethan Graham, Leo Stephan +2
GPUs are increasingly in demand for AI workloads, yet often remain substantially underutilized, motivating workload colocation. However, colocation introduces interference that can…
Unlocking True Elasticity for the Cloud-Native Era with Dandelion
Tom Kuchler, Pinghe Li, Yazhuo Zhang +8
Elasticity is fundamental to cloud computing, as it enables quickly allocating resources to match the demand of each workload as it arrives, rather than pre-provisioning resources…
Sailor: Automating Distributed Training over Dynamic, Heterogeneous, and Geo-distributed Clusters
Foteini Strati, Zhendong Zhang, George Manos +7
The high GPU demand of ML training makes it hard to allocate large homogeneous clusters of high-end GPUs in a single availability zone. Leveraging heterogeneous GPUs available with…
ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
Youhe Jiang, Fangcheng Fu, Xiaozhe Yao +4
Recent developments in large language models (LLMs) have demonstrated their remarkable proficiency in a range of tasks. Compared to in-house homogeneous GPU clusters, deploying LLM…
Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
Youhe Jiang, Fangcheng Fu, Xiaozhe Yao +6
Recent advancements in Large Language Models (LLMs) have led to increasingly diverse requests, accompanied with varying resource (compute and memory) demands to serve them. However…
Understanding GPU Resource Interference One Level Deeper
Paul Elvinger, Foteini Strati, Natalie Enright Jerger +1
GPUs are vastly underutilized, even when running resource-intensive AI applications, as GPU kernels within each job have diverse resource profiles that may saturate some parts of a…
Imaginary Machines: A Serverless Model for Cloud Applications
Michael Wawrzoniak, Rodrigo Bruno, Ana Klimovic +1
Serverless Function-as-a-Service (FaaS) platforms provide applications with resources that are highly elastic, quick to instantiate, accounted at fine granularity, and without the…
Boxer: FaaSt Ephemeral Elasticity for Off-the-Shelf Cloud Applications
Michael Wawrzoniak, Rodrigo Bruno, Ana Klimovic +1
Elasticity is a key property of cloud computing. However, elasticity is offered today at the granularity of virtual machines, which take tens of seconds to start. This is insuffici…
Dirigent: Lightweight Serverless Orchestration
Lazar Cvetković, François Costa, Mihajlo Djokic +2
While Function as a Service (FaaS) platforms can initialize function sandboxes on worker nodes in 10-100s of milliseconds, the latency to schedule functions in real FaaS clusters c…
DéjàVu: KV-cache Streaming for Fast, Fault-tolerant Generative LLM Serving
Foteini Strati, Sara Mcallister, Amar Phanishayee +2
Distributed LLM serving is costly and often underutilizes hardware accelerators due to three key challenges: bubbles in pipeline-parallel deployments caused by the bimodal latency…
DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs
Xiaozhe Yao, Qinghao Hu, Ana Klimovic
Fine-tuning large language models (LLMs) greatly improves model quality for downstream tasks. However, serving many fine-tuned LLMs concurrently is challenging due to the sporadic,…
An Elastic Ephemeral Datastore using Cheap, Transient Cloud Resources
Malte Brodmann, Nikolas Ioannou, Bernard Metzler +2
Spot instances are virtual machines offered at 60-90% lower cost that can be reclaimed at any time, with only a short warning period. Spot instances have already been used to signi…
Short-lived Datacenter
Michael Wawrzoniak, Ingo Müller, Rodrigo Bruno +2
Serverless platforms have attracted attention due to their promise of elasticity, low cost, and fast deployment. Instead of using a fixed virtual machine (VM) infrastructure, which…
Towards Demystifying Serverless Machine Learning Training
Jiawei Jiang, Shaoduo Gan, Yue Liu +6
The appeal of serverless (FaaS) has triggered a growing interest on how to use it in data-intensive applications such as ETL, query processing, or machine learning (ML). Several sy…