3 citations · 3 across the 7 of their papers we have counts for
9 papers · 1 filter
PowerSlider: Exploiting Phase Asymmetry for LLM Serving under Demand Response
Yueying Li, Jiayang Chen, Yuanfan Chen +5
AI inference clusters are increasingly constrained by instantaneous power, not just energy: grid operators condition new capacity on demand response, imposing time-varying power ca…
Cascade: Exploiting SLO-Aware latency budget for fair and high goodput LLM inference serving
Muhammad Adnan, Rohan Mahapatra, Prashant J. Nair +4
The reasoning and agentic capabilities of large language models have expanded the range of applications they support, from short interactive exchanges to long, compute-heavy reques…
Rethinking AI Cloud Infrastructure for Agentic Serving Systems with the Aries Experimentation Framework
Leonid Kondrashov, Hongrui Liu, JooYoung Park +14
Autonomous agents challenge conventional LLM serving by coupling repeated inference with persistent context and sandboxed tool execution. We present Aries, a full-stack experimenta…
StreamWise: Serving Multi-Modal Generation in Real-Time at Scale
Haoran Qiu, Gohar Irfan Chaudhry, Chaojie Zhang +4
Advances in multi-modal generative models are enabling new applications, from storytelling to automated media synthesis. Most current workloads generate simple outputs (e.g., image…
ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving
Haoran Qiu, Anish Biswas, Zihan Zhao +9
Large multimodal models (LMMs) demonstrate impressive capabilities in understanding images, videos, and audio beyond text. However, efficiently serving LMMs in production environme…
Towards Resource-Efficient Compound AI Systems
Gohar Irfan Chaudhry, Esha Choukse, Ãñigo Goiri +3
Compound AI Systems, integrating multiple interacting components like models, retrievers, and external tools, have emerged as essential for addressing complex AI tasks. However, cu…