6 citations · 14 across the 8 of their papers we have counts for
8 papers
EROICA: Online Performance Troubleshooting for Large-scale Model Training
Yu Guan, Zhiyu Yin, Haoyu Chen +11
Troubleshooting performance problems of large model training (LMT) is immensely challenging, due to unprecedented scales of modern GPU clusters, the complexity of software-hardware…
Memory Offloading for Large Language Model Inference with Latency SLO Guarantees
Chenxiang Ma, Zhisheng Ye, Hanyu Zhao +9
Offloading large language models (LLMs) state to host memory during inference promises to reduce operational costs by supporting larger models, longer inputs, and larger batch size…
Enterprise Experimentation with Hierarchical Entities
Shan Ba, Shilpa Garg, Jitendra Agarwal +1
In this paper, we address the challenges in running enterprise experimentation with hierarchical entities and present the methodologies behind the implementation of the Enterprise…
Rubick: Exploiting Job Reconfigurability for Deep Learning Cluster Scheduling
Xinyi Zhang, Hanyu Zhao, Wencong Xiao +5
The era of large deep learning models has given rise to advanced training strategies such as 3D parallelism and the ZeRO series. These strategies enable various (re-)configurable e…
Llumnix: Dynamic Scheduling for Large Language Model Serving
Biao Sun, Ziming Huang, Hanyu Zhao +4
Inference serving for large language models (LLMs) is the key to unleashing their potential in people's daily lives. However, efficient LLM serving remains challenging today becaus…
Enhancing Large-Scale AI Training Efficiency: The C4 Solution for Real-Time Anomaly Detection and Communication Optimization
Jianbo Dong, Bin Luo, Jun Zhang +22
The emergence of Large Language Models (LLMs) has necessitated the adoption of distributed training techniques, involving the deployment of thousands of GPUs to train a single mode…
Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache
Bin Lin, Chen Zhang, Tao Peng +12
Large Language Models (LLMs) demonstrate substantial potential across a diverse array of domains via request serving. However, as trends continue to push for expanding context size…
ROAM: memory-efficient large DNN training via optimized operator ordering and memory layout
Huiyao Shu, Ang Wang, Ziji Shi +3
As deep learning models continue to increase in size, the memory requirements for training have surged. While high-level techniques like offloading, recomputation, and compression…