1 citations · 2 across the 13 of their papers we have counts for
13 papers
One Threshold Is Not Enough: Prompt-Invariant Caching Schedules for Video Diffusion
Om Agrawal, Saurabh Agarwal, Aditya Akella
Video diffusion models remain expensive because denoising repeatedly evaluates a costly model backbone. Dynamic caching methods like TeaCache, EasyCache, and DiCache reduce this co…
Sangam: Efficiently Serving Diffusion LLMs with the AR Stack
Nitin Kedia, Saurabh Agarwal, Myungjin Lee +1
Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation. Their bidirectional attent…
CUCo: An Agentic Framework for Compute and Communication Co-design
Yoga Sri Varshan Varadharajan, Bodun Hu, Saurabh Agarwal +1
Computation and communication in distributed LLM training and inference are traditionally optimized in isolation; expert-crafted systems such as DeepEP, FLUX, and TokenWeave show t…
Nalar: Workflow-Aware Management of Agentic Applications
Marco Laju, Donghyun Son, Saurabh Agarwal +4
LLM-driven agentic applications automate complex, multi-step tasks, but serving them efficiently remains difficult due to heterogeneous components, dynamic model-driven control flo…
Software-Defined Agentic Serving
Saurabh Agarwal, Marco Laju, Jayanth Srinivasa +2
As multi-agent LLM pipelines grow in complexity, existing serving paradigms fail to adapt to the dynamic serving conditions. We argue that agentic serving systems should be program…
SuperGen: An Efficient Ultra-high-resolution Video Generation System with Sketching and Tiling
Fanjiang Ye, Zepeng Zhao, Yi Mu +11
Diffusion models have recently achieved remarkable success in generative tasks (e.g., image and video generation), and the demand for high-quality content (e.g., 2K/4K videos) is r…
Harmonia: End-to-End RAG Serving Optimization
Saurabh Agarwal, Bodun Hu, Luis Pabon +3
Retrieval-Augmented Generation (RAG) improves the reliability of large language models by integrating external knowledge, but serving RAG pipelines efficiently is challenging becau…
SYMPHONY: Improving Memory Management for LLM Inference Workloads
Saurabh Agarwal, Anyong Mao, Aditya Akella +1
Large Language Models (LLMs) are increasingly being deployed in applications such as chatbots, code editors, and conversational agents. A key feature of LLMs is their ability to en…
TrainMover: An Interruption-Resilient Runtime for ML Training
ChonLam Lao, Jiaqi Gao, Jiamin Cao +13
Large-scale ML training jobs are frequently interrupted by hardware and software anomalies, failures, and management events. Existing solutions like checkpoint-restart or runtime r…
EdgeSight: Enabling Modeless and Cost-Efficient Inference at the Edge
ChonLam Lao, Jiaqi Gao, Ganesh Ananthanarayanan +2
Traditional ML inference is evolving toward modeless inference, which abstracts the complexity of model selection from users, allowing the system to automatically choose the most a…
FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping
Ajay Jaiswal, Bodun Hu, Lu Yin +4
Autoregressive Large Language Models (e.g., LLaMa, GPTs) are omnipresent achieving remarkable success in language understanding and generation. However, such impressive capability…
BlockLLM: Multi-tenant Finer-grained Serving for Large Language Models
Bodun Hu, Jiamin Li, Le Xu +5
The increasing demand for Large Language Models (LLMs) across various applications has led to a significant shift in the design of deep learning serving systems. Deploying LLMs, pa…
MOSEL: Inference Serving Using Dynamic Modality Selection
Bodun Hu, Le Xu, Jeongyoon Moon +2
Rapid advancements over the years have helped machine learning models reach previously hard-to-achieve goals, sometimes even exceeding human capabilities. However, to attain the de…