15 citations · 26 across the 7 of their papers we have counts for
3 papers · 1 filter
Is the GPU Half-Empty or Half-Full? Practical Scheduling Techniques for LLMs
Ferdi Kossmann, Bruce Fontaine, Daya Khudia +2
Serving systems for Large Language Models (LLMs) improve throughput by processing several requests concurrently. However, multiplexing hardware resources between concurrent request…
Blueprinting the Cloud: Unifying and Automatically Optimizing Cloud Data Infrastructures with BRAD -- Extended Version
Geoffrey X. Yu, Ziniu Wu, Ferdi Kossmann +5
Modern organizations manage their data with a wide variety of specialized cloud database engines (e.g., Aurora, BigQuery, etc.). However, designing and managing such infrastructure…
CascadeServe: Unlocking Model Cascades for Inference Serving
Ferdi Kossmann, Ziniu Wu, Alex Turk +3
Machine learning (ML) models are increasingly deployed to production, calling for efficient inference serving systems. Efficient inference serving is complicated by two challenges:…