2 citations · 7 across the 15 of their papers we have counts for
4 papers · 1 filter
ORACLE: Agentic AI Orchestrator Routing Via Adaptive Verifier Calibration Feedback
Ritik Raj, Souvik Kundu, Sarbartha Banerjee +3
Modern enterprise agent deployments consist of a heterogeneous pool of large language models (LLMs) having diverse capabilities and cost. Existing model routing strategies optimize…
Serving Agentic Workflows with a Physical-Plan Compiler and Adaptive Runtime
Jiayi Qian, Yichong Zhang, Zishen Wan +4
Efficient serving of agentic workflows requires selecting each LLM node's model, verification policy, and backend to balance output quality, latency, and throughput. These assignme…
Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs
Mingyu Lee, Akshat Ramachandran, Souvik Kundu +1
The inference efficiency of diffusion large language models (dLLMs) is constrained by two challenges: bidirectional attention precludes efficient KV-cache reuse, while increasing d…
How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
Hanjiang Wu, Abhimanyu Rajeshkumar Bambhaniya, Sarbartha Banerjee +9
Modern large language model (LLM) inference has progressively disaggregated to keep pace with growing model sizes and tight TTFT and TPOT service-level objectives: from chunked-pre…