26 citations · 51 across the 8 of their papers we have counts for
11 papers · 1 filter
DéjàVu: KV-cache Streaming for Fast, Fault-tolerant Generative LLM Serving
Foteini Strati, Sara Mcallister, Amar Phanishayee +2
Distributed LLM serving is costly and often underutilizes hardware accelerators due to three key challenges: bubbles in pipeline-parallel deployments caused by the bimodal latency…
Blox: A Modular Toolkit for Deep Learning Schedulers
Saurabh Agarwal, Amar Phanishayee, Shivaram Venkataraman
Deep Learning (DL) workloads have rapidly increased in popularity in enterprise clusters and several new cluster schedulers have been proposed in recent years to support these work…
Packrat: Automatic Reconfiguration for Latency Minimization in CPU-based DNN Serving
Ankit Bhardwaj, Amar Phanishayee, Deepak Narayanan +2
In this paper, we investigate how to push the performance limits of serving Deep Neural Network (DNN) models on CPU-based servers. Specifically, we observe that while intra-operato…
Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads
Deepak Narayanan, Keshav Santhanam, Fiodar Kazhamiaka +2
Specialized accelerators such as GPUs, TPUs, FPGAs, and custom ASICs have been increasingly deployed to train deep learning models. These accelerators exhibit heterogeneous perform…
Analyzing and Mitigating Data Stalls in DNN Training
Jayashree Mohan, Amar Phanishayee, Ashish Raniwala +1
Training Deep Neural Networks (DNNs) is resource-intensive and time-consuming. While prior research has explored many different ways of reducing DNN training time, the impact of in…
Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training
Hongyu Zhu, Amar Phanishayee, Gennady Pekhimenko
Modern deep neural network (DNN) training jobs use complex and heterogeneous software/hardware stacks. The efficacy of software-level optimizations can vary significantly when used…