5 papers
Five-Minute Rule 40 Years Later: A First-Principles Revisit for Modern Memory Hierarchy
Tong Zhang, Vikram Sharma Mailthody, Fei Sun +7
In 1987, Jim Gray and Gianfranco Putzolu introduced the five-minute rule, a simple, storage-memory-economics-based heuristic for deciding when data should live in DRAM rather than…
LSM-GNN: Large-scale Storage-based Multi-GPU GNN Training by Optimizing Data Transfer Scheme
Jeongmin Brian Park, Kun Wu, Vikram Sharma Mailthody +3
Graph Neural Networks (GNNs) are widely used today in recommendation systems, fraud detection, and node/link classification tasks. Real world GNNs continue to scale in size and req…
IRB: Automated Generation of Robust Factuality Benchmarks
Lam Thanh Do, Bhagyashree Taleka, Hozaifa Ammar Bhutta +3
Static benchmarks for RAG systems often suffer from rapid saturation and require significant manual effort to maintain robustness. To address this, we present IRB, a framework for…
Strata: Hierarchical Context Caching for Long Context Language Model Serving
Zhiqiang Xie, Ziyi Xu, Mark Zhao +5
Large Language Models (LLMs) with expanding context windows face significant performance hurdles. While caching key-value (KV) states is critical for avoiding redundant computation…
SSDTrain: An Activation Offloading Framework to SSDs for Faster Large Language Model Training
Kun Wu, Jeongmin Brian Park, Xiaofan Zhang +5
The growth rate of the GPU memory capacity has not been able to keep up with that of the size of large language models (LLMs), hindering the model training process. In particular,…