Showing cs.DCShow all
2 papers · 1 filter
cs.DC2025
Aragog: Just-in-Time Model Routing for Scalable Serving of Agentic Workflows
Yinwei Dai, Zhuofu Chen, Anand Iyer +1
Agentic workflows have emerged as a powerful paradigm for solving complex, multi-stage tasks, but serving them at scale is computationally expensive given the many LLM inferences t…
cs.DC2023
Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving
Yinwei Dai, Rui Pan, Anand Iyer +2
Machine learning (ML) inference platforms are tasked with balancing two competing goals: ensuring high throughput given many requests, and delivering low-latency responses to suppo…