117 citations · 134 across the 3 of their papers we have counts for
3 papers
cs.AR2021★ 1 cited
Tackling Variabilities in Autonomous Driving
Yuqiong Qi, Yang Hu, Haibin Wu +5
The state-of-the-art driving automation system demands extreme computational resources to meet rigorous accuracy and latency requirements. Though emerging driving automation comput…
cs.LG2021★ 16 cited
PipeTransformer: Automated Elastic Pipelining for Distributed Training of Transformers
Chaoyang He, Shen Li, Mahdi Soltanolkotabi +1
The size of Transformer models is growing at an unprecedented pace. It has only taken less than one year to reach trillion-level parameters after the release of GPT-3 (175B). Train…
cs.DC2020★ 117 cited
PyTorch Distributed: Experiences on Accelerating Data Parallel Training
Shen Li, Yanli Zhao, Rohan Varma +8
This paper presents the design, implementation, and evaluation of the PyTorch distributed data parallel module. PyTorch is a widely-adopted scientific computing package used in dee…