163 citations · 203 across the 7 of their papers we have counts for
8 papers
ARC-KV: Amortizing Anchor Search for Reconstruction-Based KV Cache Compaction
Zheyu Shen, Guanhua Wang, Dezhan Tu +5
Long-context large language model inference is bottlenecked by KV caches that grow linearly with sequence length. This burden is especially severe for long, reusable context prefix…
Domino: Eliminating Communication in LLM Training via Generic Tensor Slicing and Overlapping
Guanhua Wang, Chengming Zhang, Zheyu Shen +2
Given the popularity of generative AI, Large Language Models (LLMs) often consume hundreds or thousands of GPUs for parallelizing and accelerating the training process. Communicati…
FastPersist: Accelerating Model Checkpointing in Deep Learning
Guanhua Wang, Olatunji Ruwase, Bing Xie +1
Model checkpoints are critical Deep Learning (DL) artifacts that enable fault tolerance for training and downstream applications, such as inference. However, writing checkpoints to…
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Marah Abdin, Jyoti Aneja, Hany Awadalla +126
We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal test…
ZeRO++: Extremely Efficient Collective Communication for Giant Model Training
Guanhua Wang, Heyang Qin, Sam Ade Jacobs +6
Zero Redundancy Optimizer (ZeRO) has been used to train a wide range of large language models on massive GPUs clusters due to its ease of use, efficiency, and good scalability. How…
Composing MPC with LQR and Neural Network for Amortized Efficiency and Stable Control
Fangyu Wu, Guanhua Wang, Siyuan Zhuang +4
Model predictive control (MPC) is a powerful control method that handles dynamical systems with constraints. However, solving MPC iteratively in real time, i.e., implicit MPC, rema…
ResiliNet: Failure-Resilient Inference in Distributed Neural Networks
Ashkan Yousefpour, Brian Q. Nguyen, Siddartha Devic +5
Federated Learning aims to train distributed deep models without sharing the raw data with the centralized server. Similarly, in distributed inference of neural networks, by partit…
Blink: Fast and Generic Collectives for Distributed ML
Guanhua Wang, Shivaram Venkataraman, Amar Phanishayee +3
Model parameter synchronization across GPUs introduces high overheads for data-parallel training at scale. Existing parameter synchronization protocols cannot effectively leverage…