587 citations · 1.5k across the 25 of their papers we have counts for
4 papers · 1 filter
Coral: Cost-Efficient Multi-LLM Serving over Heterogeneous Cloud GPUs
Yixuan Mei, Zikun Li, Zixuan Chen +5
The usage of large language models (LLMs) has grown increasingly fragmented, with no single model dominating. Meanwhile, cloud providers offer a wide range of mid-tier and older-ge…
Arithmetic-Intensity-Guided Fault Tolerance for Neural Network Inference on GPUs
Jack Kosaian, K. V. Rashmi
Neural networks (NNs) are increasingly employed in safety-critical domains and in environments prone to unreliability (e.g., soft errors), such as on spacecraft. Therefore, it is c…
PACEMAKER: Avoiding HeART attacks in storage clusters with disk-adaptive redundancy
Saurabh Kadekodi, Francisco Maturana, Suhas Jayaram Subramanya +3
Data redundancy provides resilience in large-scale storage clusters, but imposes significant cost overhead. Substantial space-savings can be realized by tuning redundancy schemes t…
Parity Models: A General Framework for Coding-Based Resilience in ML Inference
Jack Kosaian, K. V. Rashmi, Shivaram Venkataraman
Machine learning models are becoming the primary workhorses for many applications. Production services deploy models through prediction serving systems that take in queries and ret…