3 citations · 5 across the 5 of their papers we have counts for
8 papers
TRANSIT: Transparent Scale-in for Multi-Node LLM Training
Hyungyo Kim, Nicholas Satchanov, Hrishi Shah +9
TRANSIT is a transparent scale-in framework to enable multi-node model training on fewer GPUs while maintaining training efficiency by transparently leveraging CPU DRAM as an exten…
Efficient Iterative Retrieval with Heterogeneous Batching
Dohyun Park, Hubertus Franke, Daniel G. Waddington +2
Modern information retrieval increasingly employs both embedding and generative models to handle complex queries. However, current serving systems suffer from low throughput and po…
Towards an Agent Operating System - Lessons from Classical and Cloud OS
Gosia Steinder, Hubertus Franke
Every major wave of platform software follows the same arc: an initial period of experimentation with competing frameworks and ad-hoc implementations, followed by the articulation…
RAGPerf: An End-to-End Benchmarking Framework for Retrieval-Augmented Generation Systems
Shaobo Li, Yirui Zhou, Yuan Xu +5
We present the design and implementation of a RAG-based AI system benchmarking (RAGPerf) framework for characterizing the system behaviors of RAG pipelines. To facilitate detailed…
Cache is King: Smart Page Eviction with eBPF
Tal Zussman, Ioannis Zarkadas, Jeremy Carin +4
The page cache is a central part of an OS. It reduces repeated accesses to storage by deciding which pages to retain in memory. As a result, the page cache has a significant impact…
Safe and usable kernel extensions with Rex
Jinghao Jia, Ruowen Qin, Milo Craun +7
Safe kernel extensions have gained significant traction, evolving from simple packet filters to large, complex programs that customize storage, networking, and scheduling. Existing…
Chameleon: Adaptive Caching and Scheduling for Many-Adapter LLM Inference Environments
Nikoleta Iliakopoulou, Jovan Stojkovic, Chloe Alverti +3
The widespread adoption of LLMs has driven an exponential rise in their deployment, imposing substantial demands on inference clusters. These clusters must handle numerous concurre…
Transforming the Hybrid Cloud for Emerging AI Workloads
Deming Chen, Alaa Youssef, Ruchi Pendse +42
This white paper, developed through close collaboration between IBM Research and UIUC researchers within the IIDAI Institute, envisions transforming hybrid cloud systems to meet th…