19 citations · 89 across the 28 of their papers we have counts for
6 papers · 1 filter
Aragog: Just-in-Time Model Routing for Scalable Serving of Agentic Workflows
Yinwei Dai, Zhuofu Chen, Anand Iyer +1
Agentic workflows have emerged as a powerful paradigm for solving complex, multi-stage tasks, but serving them at scale is computationally expensive given the many LLM inferences t…
SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning
Rui Pan, Yinwei Dai, Zhihao Zhang +3
Recent advances in inference-time compute have significantly improved performance on complex tasks by generating long chains of thought (CoTs) using Large Reasoning Models (LRMs).…
Legilimens: Performant Video Analytics on the System-on-Chip Edge
Murali Ramanujam, Yinwei Dai, Kyle Jamieson +1
Continually retraining models has emerged as a primary technique to enable high-accuracy video analytics on edge devices. Yet, existing systems employ such adaptation by relying on…
Guillotine: Hypervisors for Isolating Malicious AIs
James Mickens, Sarah Radway, Ravi Netravali
As AI models become more embedded in critical sectors like finance, healthcare, and the military, their inscrutable behavior poses ever-greater risks to society. To mitigate this r…
Scalable Video Conferencing Using SDN Principles
Oliver Michel, Satadal Sengupta, Hyojoon Kim +2
Video-conferencing applications face an unwavering surge in traffic, stressing their underlying infrastructure in unprecedented ways. This paper rethinks the key building block for…
GPUs, CPUs, and... NICs: Rethinking the Network's Role in Serving Complex AI Pipelines
Mike Wong, Ulysses Butler, Emma Farkash +3
The increasing prominence of AI necessitates the deployment of inference platforms for efficient and effective management of AI pipelines and compute resources. As these pipelines…