most citedDeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference

6 citations · 6 across the 2 of their papers we have counts for

collaborators
Showing cs.DCShow all

1 paper · 1 filter

cs.DC2024

Universal Checkpointing: A Flexible and Efficient Distributed Checkpointing System for Large-Scale DNN Training with Reconfigurable Parallelis

Xinyu Lian, Sam Ade Jacobs, Lev Kurilenko +4

Deep neural network (DNN) training continues to scale rapidly in terms of model size, data volume, and sequence length, to the point where multiple machines are required to fit lar…