282 citations · 1k across the 43 of their papers we have counts for
4 papers · 1 filter
Avoiding Communication in Logistic Regression
Aditya Devarakonda, James Demmel
Stochastic gradient descent (SGD) is one of the most widely used optimization methods for solving various machine learning problems. SGD solves an optimization problem by iterative…
Training EfficientNets at Supercomputer Scale: 83% ImageNet Top-1 Accuracy in One Hour
Arissa Wongpanich, Hieu Pham, James Demmel +4
EfficientNets are a family of state-of-the-art image classification models based on efficiently scaled convolutional neural networks. Currently, EfficientNets can take on the order…
The Limit of the Batch Size
Yang You, Yuhui Wang, Huan Zhang +3
Large-batch training is an efficient approach for current distributed deep learning systems. It has enabled researchers to reduce the ImageNet/ResNet-50 training from 29 hours to a…
Communication-Optimal Tilings for Projective Nested Loops with Arbitrary Bounds
Grace Dinh, James Demmel
Reducing communication - either between levels of a memory hierarchy or between processors over a network - is a key component of performance optimization (in both time and energy)…