3 citations · 5 across the 9 of their papers we have counts for
10 papers
Fast Recovery for LLM Serving via Decoupled Device Memory Lifetime in Dynamo
Schwinn Saereesitthipitak, Mohammed Abdulwahhab, Hannah Zhang +6
Large language model (LLM) inference replicas run across tightly coupled GPUs and serve traffic continuously for weeks. Hardware and software failures are therefore inevitable, and…
IRB: Automated Generation of Robust Factuality Benchmarks
Lam Thanh Do, Bhagyashree Taleka, Hozaifa Ammar Bhutta +3
Static benchmarks for RAG systems often suffer from rapid saturation and require significant manual effort to maintain robustness. To address this, we present IRB, a framework for…
VibeTensor: System Software for Deep Learning, Fully Generated by AI Agents
Bing Xu, Terry Chen, Fengzhe Zhou +12
VIBETENSOR is an open-source research system software stack for deep learning, generated by LLM-powered coding agents under high-level human guidance. In this paper, "fully generat…
Five-Minute Rule 40 Years Later: A First-Principles Revisit for Modern Memory Hierarchy
Tong Zhang, Vikram Sharma Mailthody, Fei Sun +7
In 1987, Jim Gray and Gianfranco Putzolu introduced the five-minute rule, a simple, storage-memory-economics-based heuristic for deciding when data should live in DRAM rather than…
AUGUSTUS: An LLM-Driven Multimodal Agent System with Contextualized User Memory
Jitesh Jain, Shubham Maheshwari, Ning Yu +2
Riding on the success of LLMs with retrieval-augmented generation (RAG), there has been a growing interest in augmenting agent systems with external memory databases. However, the…
Generalized Neighborhood Attention: Multi-dimensional Sparse Attention at the Speed of Light
Ali Hassani, Fengzhe Zhou, Aditya Kane +13
Many sparse attention mechanisms such as Neighborhood Attention have typically failed to consistently deliver speedup over the self attention baseline. This is largely due to the l…
HiCCL: A Hierarchical Collective Communication Library
Mert Hidayetoglu, Simon Garcia de Gonzalo, Elliott Slaughter +4
HiCCL (Hierarchical Collective Communication Library) addresses the growing complexity and diversity in high-performance network architectures. As GPU systems have envolved into ne…
LSM-GNN: Large-scale Storage-based Multi-GPU GNN Training by Optimizing Data Transfer Scheme
Jeongmin Brian Park, Kun Wu, Vikram Sharma Mailthody +3
Graph Neural Networks (GNNs) are widely used today in recommendation systems, fraud detection, and node/link classification tasks. Real world GNNs continue to scale in size and req…
Faster Neighborhood Attention: Reducing the O(n^2) Cost of Self Attention at the Threadblock Level
Ali Hassani, Wen-Mei Hwu, Humphrey Shi
Neighborhood attention reduces the cost of self attention by restricting each token's attention span to its nearest neighbors. This restriction, parameterized by a window size and…
RackBlox: A Software-Defined Rack-Scale Storage System with Network-Storage Co-Design
Benjamin Reidys, Yuqi Xue, Daixuan Li +5
Software-defined networking (SDN) and software-defined flash (SDF) have been serving as the backbone of modern data centers. They are managed separately to handle I/O requests. At…