2 citations · 3 across the 8 of their papers we have counts for
5 papers · 1 filter
Cascade: Exploiting SLO-Aware latency budget for fair and high goodput LLM inference serving
Muhammad Adnan, Rohan Mahapatra, Prashant J. Nair +4
The reasoning and agentic capabilities of large language models have expanded the range of applications they support, from short interactive exchanges to long, compute-heavy reques…
StreamWise: Serving Multi-Modal Generation in Real-Time at Scale
Haoran Qiu, Gohar Irfan Chaudhry, Chaojie Zhang +4
Advances in multi-modal generative models are enabling new applications, from storytelling to automated media synthesis. Most current workloads generate simple outputs (e.g., image…
ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving
Haoran Qiu, Anish Biswas, Zihan Zhao +9
Large multimodal models (LMMs) demonstrate impressive capabilities in understanding images, videos, and audio beyond text. However, efficiently serving LMMs in production environme…
TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms
Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri +5
The rising demand for generative large language models (LLMs) poses challenges for thermal and power management in cloud datacenters. Traditional techniques often are inadequate fo…
Workload Intelligence: Punching Holes Through the Cloud Abstraction
Lexiang Huang, Anjaly Parayil, Jue Zhang +13
Today, cloud workloads are essentially opaque to the cloud platform. Typically, the only information the platform receives is the virtual machine (VM) type and possibly a decoratio…