203 citations · 593 across the 26 of their papers we have counts for
4 papers · 1 filter
A Data-Centric Optimization Framework for Machine Learning
Oliver Rausch, Tal Ben-Nun, Nikoli Dryden +3
Rapid progress in deep learning is leading to a diverse set of quickly changing models, with a dramatically growing demand for compute. However, as frameworks specialize performanc…
Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines
Shigang Li, Torsten Hoefler
Training large deep learning models at scale is very challenging. This paper proposes Chimera, a novel pipeline parallelism scheme which combines bidirectional pipelines for effici…
Flare: Flexible In-Network Allreduce
Daniele De Sensi, Salvatore Di Girolamo, Saleh Ashkboos +2
The allreduce operation is one of the most commonly used communication routines in distributed applications. To improve its bandwidth and to reduce network traffic, this operation…
AGCM-3DLF: Accelerating Atmospheric General Circulation Model via 3D Parallelization and Leap-Format
Hang Cao, Liang Yuan, He Zhang +1
The Atmospheric General Circulation Model (AGCM) has been an important research tool in the study of climate change for decades. As the demand for high-resolution simulation is bec…