1 citations · 1 across the 8 of their papers we have counts for
9 papers
PowerSlider: Exploiting Phase Asymmetry for LLM Serving under Demand Response
Yueying Li, Jiayang Chen, Yuanfan Chen +5
AI inference clusters are increasingly constrained by instantaneous power, not just energy: grid operators condition new capacity on demand response, imposing time-varying power ca…
OpScale: Operator-level Provisioning and Autoscaling for LLM Serving
Xingqi Cui, Chieh-Jan Mike Liang, Ziang Tang +2
Achieving cost efficiency while meeting strict user-facing SLOs (e.g., time-to-first-token) remains a fundamental challenge for cloud GPU clusters serving large language models (LL…
Beyond Prediction: Tail-Aware Scheduling for LLM Inference
Yueying Li, Yuanfan Chen, Jiayang Chen +6
LLM serving exhibits extreme length variability, making size-based scheduling difficult in practice. Recent LLM schedulers approximate SJF/SRPT using predicted decode lengths or ra…
StreamWise: Serving Multi-Modal Generation in Real-Time at Scale
Haoran Qiu, Gohar Irfan Chaudhry, Chaojie Zhang +4
Advances in multi-modal generative models are enabling new applications, from storytelling to automated media synthesis. Most current workloads generate simple outputs (e.g., image…
From Models to Operators: Rethinking Autoscaling Granularity for Large Generative Models
Xingqi Cui, Chieh-Jan Mike Liang, Jiarong Xing +1
Serving large generative models such as LLMs and multi- modal transformers requires balancing user-facing SLOs (e.g., time-to-first-token, time-between-tokens) with provider goals…
Sherlock: Reliable and Efficient Agentic Workflow Execution
Yeonju Ro, Haoran Qiu, Íñigo Goiri +6
With the increasing adoption of large language models (LLM), agentic workflows, which compose multiple LLM calls with tools, retrieval, and reasoning steps, are increasingly replac…
Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platforms
Gohar Irfan Chaudhry, Esha Choukse, Haoran Qiu +4
Agentic workflows commonly coordinate multiple models and tools with complex control logic. They are quickly becoming the dominant paradigm for AI applications. However, serving th…
ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving
Haoran Qiu, Anish Biswas, Zihan Zhao +9
Large multimodal models (LMMs) demonstrate impressive capabilities in understanding images, videos, and audio beyond text. However, efficiently serving LMMs in production environme…
TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms
Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri +5
The rising demand for generative large language models (LLMs) poses challenges for thermal and power management in cloud datacenters. Traditional techniques often are inadequate fo…