341 citations · 357 across the 4 of their papers we have counts for
6 papers
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun +2
The growing energy and performance costs of deep learning have driven the community to reduce the size of neural networks by selectively pruning components. Similarly to their biol…
Clairvoyant Prefetching for Distributed Machine Learning I/O
Nikoli Dryden, Roman Böhringer, Tal Ben-Nun +1
I/O is emerging as a major bottleneck for machine learning training, especially in distributed environments. Indeed, at large scale, I/O takes as much as 85% of training time. Addr…
The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism
Yosuke Oyama, Naoya Maruyama, Nikoli Dryden +6
We present scalable hybrid-parallel algorithms for training large-scale 3D convolutional neural networks. Deep learning-based emerging scientific workflows often require model trai…
Deep Learning for Post-Processing Ensemble Weather Forecasts
Peter Grönquist, Chengyuan Yao, Tal Ben-Nun +4
Quantifying uncertainty in weather forecasts is critical, especially for predicting extreme weather events. This is typically accomplished with ensemble prediction systems, which c…
Predicting Weather Uncertainty with Deep Convnets
Peter Grönquist, Tal Ben-Nun, Nikoli Dryden +4
Modern weather forecast models perform uncertainty quantification using ensemble prediction systems, which collect nonparametric statistics based on multiple perturbed simulations.…
Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism
Nikoli Dryden, Naoya Maruyama, Tom Benson +3
Scaling CNN training is necessary to keep up with growing datasets and reduce training time. We also see an emerging need to handle datasets with very large samples, where memory r…