3 citations · 3 across the 6 of their papers we have counts for
19 papers
PowerSlider: Exploiting Phase Asymmetry for LLM Serving under Demand Response
Yueying Li, Jiayang Chen, Yuanfan Chen +5
AI inference clusters are increasingly constrained by instantaneous power, not just energy: grid operators condition new capacity on demand response, imposing time-varying power ca…
Cascade: Exploiting SLO-Aware latency budget for fair and high goodput LLM inference serving
Muhammad Adnan, Rohan Mahapatra, Prashant J. Nair +4
The reasoning and agentic capabilities of large language models have expanded the range of applications they support, from short interactive exchanges to long, compute-heavy reques…
Rethinking AI Cloud Infrastructure for Agentic Serving Systems with the Aries Experimentation Framework
Leonid Kondrashov, Hongrui Liu, JooYoung Park +14
Autonomous agents challenge conventional LLM serving by coupling repeated inference with persistent context and sandboxed tool execution. We present Aries, a full-stack experimenta…
Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale
Banruo Liu, Haoran Qiu, Ãñigo Goiri +3
AI coding agents like GitHub Copilot, Claude Code, and Codex interleave multi-step LLM inference with tool execution, creating a workload different from chatbots. We present the fi…
Beyond Prediction: Tail-Aware Scheduling for LLM Inference
Yueying Li, Yuanfan Chen, Jiayang Chen +6
LLM serving exhibits extreme length variability, making size-based scheduling difficult in practice. Recent LLM schedulers approximate SJF/SRPT using predicted decode lengths or ra…
Energy Use of AI Inference, Efficiency Pathways, and Test-Time Scaling
Felipe Oviedo, Fiodar Kazhamiaka, Esha Choukse +5
As AI inference scales to billions of queries, estimates of per-query energy use are increasingly important for capacity planning, efficiency interventions, and policy. Yet many pu…