Publications (9)
Towards Resource-Efficient Compound AI Systems
Gohar Irfan Chaudhry, Esha Choukse, Ãñigo Goiri +3
Compound AI Systems, integrating multiple interacting components like models, retrievers, and external tools, have emerged as essential for addressing complex AI tasks. However, cu…
Junctiond: Extending FaaS Runtimes with Kernel-Bypass
Enrique Saurez, Joshua Fried, Gohar Irfan Chaudhry +5
This report explores the use of kernel-bypass networking in FaaS runtimes and demonstrates how using Junction, a novel kernel-bypass system, as the backend for executing components…
Checkmate: Zero-Overhead Model Checkpointing via Network Gradient Replication
Ankit Bhardwaj, Weiyang Wang, Jeremy Carin +2
This paper presents Checkmate, a system that enables per-iteration checkpointing in DNN training without any training slowdown. The traditional approach to checkpointing requires a…
Triton for MTIA: Bridging the Programming Model Gaps for Custom AI Accelerators
Haishan Zhu, Domi Yan, Michael Levesque-Dion +37
The rapid growth in machine learning workloads has fueled the proliferation of custom accelerator architectures. Designed from the ground up, these accelerators often expose progra…
Taming Serverless Cold Starts Through OS Co-Design
Ben Holmes, Baltasar Dinis, Lana Honcharuk +2
Serverless computing promises fine-grained elasticity and operational simplicity, fueling widespread interest from both industry and academia. Yet this promise is undercut by the c…
Treehouse: A Case For Carbon-Aware Datacenter Software
Thomas Anderson, Adam Belay, Mosharaf Chowdhury +2
The end of Dennard scaling and the slowing of Moore's Law has put the energy use of datacenters on an unsustainable path. Datacenters are already a significant fraction of worldwid…
Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platforms
Gohar Irfan Chaudhry, Esha Choukse, Haoran Qiu +4
Agentic workflows commonly coordinate multiple models and tools with complex control logic. They are quickly becoming the dominant paradigm for AI applications. However, serving th…
Benchmarking Compound AI Applications for Hardware-Software Co-Design
Paramuth Samuthrsindh, Angel Cervantes, Varun Gohil +3
Compound AI applications, composed from interactions between Large Language Models (LLMs), Machine Learning (ML) models, external tools and data sources are quickly becoming an int…
Efficient Strong Scaling Through Burst Parallel Training
Seo Jin Park, Joshua Fried, Sunghyun Kim +2
As emerging deep neural network (DNN) models continue to grow in size, using large GPU clusters to train DNNs is becoming an essential requirement to achieving acceptable training…