19 citations · 89 across the 28 of their papers we have counts for
29 papers
Lizard: Bandwidth-Adaptive Real-Time Video Analytics through Content-Aware Packet Discarding at Last-Mile Edge Routers
Shan Yu, Yu Chen, Yifan Qiao +3
The timeliness and accuracy of edge-based video analytics can be hindered by drastic reductions in available bandwidth (ABW) at last-mile edge routers, causing prolonged queuing de…
Thought-Level Beam Search for Reasoning
Lijie Yang, Hongyin Luo, Jiawei Zhao +2
Test-time compute scaling is a primary driver of performance in large reasoning models (LRMs), but extreme inefficiency bounds current approaches, shifting the critical question fr…
Skim: Speculative Execution for Fast and Efficient Web Agents
Mike Wong, Kevin Hsieh, Suman Nath +1
Skim is a speculative execution framework for web agents that exploits the predictable structure of purpose-built websites. Today's web-agent expense is not intrinsic to the tasks…
Kairos: A Scalable Serving System for Physical AI
Yinwei Dai, Ganesh Ananthanarayanan, Landon Cox +3
Physical AI is experiencing rapid growth with frontier foundation models increasing its capabilities across general environments. Physical AI tasks are characterized by inference p…
Geometry Guided Self-Consistency for Physical AI
Yinwei Dai, Zhuofu Chen, Lijie Yang +1
State-of-the-art physical AI models generate a chunk of actions per inference through diffusion or flow matching, iteratively refining an initial noise sample into an action trajec…
Slipstream: Trajectory-Grounded Compaction Validation for Long-Horizon Agents
Zhuofu Chen, Rui Pan, Yinwei Dai +1
To cope with the large contexts that long-horizon LLM agents produce, modern frameworks increasingly rely on compaction -- invoking an LLM to rewrite the accumulated trajectory int…
Wherefore Art Thou? Provenance-Guided Automatic Online Debugging with Lumos
Jingyuan Chen, Lei Zhang, Leon Schuermann +3
Debugging distributed systems in-production is inevitable and hard. Myriad interactions between concurrent components in modern, complex and large-scale systems cause non-determini…
Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs
Rui Pan, Zhuofu Chen, Hongyi Liu +2
Diffusion Large Language Models (dLLMs) offer fast, parallel token generation, but their standalone use is plagued by an inherent efficiency-quality tradeoff. We show that, if care…
Aragog: Just-in-Time Model Routing for Scalable Serving of Agentic Workflows
Yinwei Dai, Zhuofu Chen, Anand Iyer +1
Agentic workflows have emerged as a powerful paradigm for solving complex, multi-stage tasks, but serving them at scale is computationally expensive given the many LLM inferences t…
SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning
Rui Pan, Yinwei Dai, Zhihao Zhang +3
Recent advances in inference-time compute have significantly improved performance on complex tasks by generating long chains of thought (CoTs) using Large Reasoning Models (LRMs).…
Legilimens: Performant Video Analytics on the System-on-Chip Edge
Murali Ramanujam, Yinwei Dai, Kyle Jamieson +1
Continually retraining models has emerged as a primary technique to enable high-accuracy video analytics on edge devices. Yet, existing systems employ such adaptation by relying on…
Guillotine: Hypervisors for Isolating Malicious AIs
James Mickens, Sarah Radway, Ravi Netravali
As AI models become more embedded in critical sectors like finance, healthcare, and the military, their inscrutable behavior poses ever-greater risks to society. To mitigate this r…
Scalable Video Conferencing Using SDN Principles
Oliver Michel, Satadal Sengupta, Hyojoon Kim +2
Video-conferencing applications face an unwavering surge in traffic, stressing their underlying infrastructure in unprecedented ways. This paper rethinks the key building block for…
GPUs, CPUs, and... NICs: Rethinking the Network's Role in Serving Complex AI Pipelines
Mike Wong, Ulysses Butler, Emma Farkash +3
The increasing prominence of AI necessitates the deployment of inference platforms for efficient and effective management of AI pipelines and compute resources. As these pipelines…
METIS: Fast Quality-Aware RAG Systems with Configuration Adaptation
Siddhant Ray, Rui Pan, Zhuohan Gu +5
RAG (Retrieval Augmented Generation) allows LLMs (large language models) to generate better responses with external knowledge, but using more external knowledge often improves gene…
Marconi: Prefix Caching for the Era of Hybrid LLMs
Rui Pan, Zhuang Wang, Zhen Jia +5
Hybrid models that combine the language modeling capabilities of Attention layers with the efficiency of Recurrent layers (e.g., State Space Models) have gained traction in practic…
Mowgli: Passively Learned Rate Control for Real-Time Video
Neil Agarwal, Rui Pan, Francis Y. Yan +1
Rate control algorithms are at the heart of video conferencing platforms, determining target bitrates that match dynamic network characteristics for high quality. Recent data-drive…
Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving
Yinwei Dai, Rui Pan, Anand Iyer +2
Machine learning (ML) inference platforms are tasked with balancing two competing goals: ensuring high throughput given many requests, and delivering low-latency responses to suppo…
MadEye: Boosting Live Video Analytics Accuracy with Adaptive Camera Configurations
Mike Wong, Murali Ramanujam, Guha Balakrishnan +1
Camera orientations (i.e., rotation and zoom) govern the content that a camera captures in a given scene, which in turn heavily influences the accuracy of live video analytics pipe…
Marvolo: Programmatic Data Augmentation for Practical ML-Driven Malware Detection
Michael D. Wong, Edward Raff, James Holt +1
Data augmentation has been rare in the cyber security domain due to technical difficulties in altering data in a manner that is semantically consistent with the original data. This…
Dashlet: Taming Swipe Uncertainty for Robust Short Video Streaming
Zhuqi Li, Yaxiong Xie, Ravi Netravali +1
Short video streaming applications have recently gained substantial traction, but the non-linear video presentation they afford swiping users fundamentally changes the problem of m…
Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs
John Thorpe, Pengzhan Zhao, Jonathan Eyolfson +5
DNN models across many domains continue to grow in size, resulting in high resource requirements for effective training, and unpalatable (and often unaffordable) costs for organiza…
Canvas: Isolated and Adaptive Swapping for Multi-Applications on Remote Memory
Chenxi Wang, Yifan Qiao, Haoran Ma +6
Remote memory techniques for datacenter applications have recently gained a great deal of popularity. Existing remote memory techniques focus on the efficiency of a single applicat…
GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge
Arthi Padmanabhan, Neil Agarwal, Anand Iyer +5
Video analytics pipelines have steadily shifted to edge deployments to reduce bandwidth overheads and privacy violations, but in doing so, face an ever-growing resource tension. Mo…