282 citations · 1k across the 43 of their papers we have counts for
6 papers · 1 filter
Auto-Precision Scaling for Distributed Deep Learning
Ruobing Han, James Demmel, Yang You
It has been reported that the communication cost for synchronizing gradients can be a bottleneck, which limits the scalability of distributed deep learning. Using low-precision gra…
An improved analysis and unified perspective on deterministic and randomized low rank matrix approximations
James Demmel, Laura Grigori, Alexander Rusciano
We introduce a Generalized LU-Factorization (\textbf{GLU}) for low-rank matrix approximation. We relate this to past approaches and extensively analyze its approximation properties…
Multitask and Transfer Learning for Autotuning Exascale Applications
Wissam M. Sid-Lakhdar, Mohsen Mahmoudi Aznaveh, Xiaoye S. Li +1
Multitask learning and transfer learning have proven to be useful in the field of machine learning when additional knowledge is available to help a prediction task. We aim at deriv…
Parallel and Communication Avoiding Least Angle Regression
S. Das, J. Demmel, K. Fountoulakis +3
We are interested in parallelizing the Least Angle Regression (LARS) algorithm for fitting linear regression models to high-dimensional data. We consider two parallel and communica…
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
Yang You, Jing Li, Sashank Reddi +7
Training large deep neural networks on massive datasets is computationally very challenging. There has been recent surge in interest in using large batch stochastic optimization me…
Large-Batch Training for LSTM and Beyond
Yang You, Jonathan Hseu, Chris Ying +3
Large-batch training approaches have enabled researchers to utilize large-scale distributed processing and greatly accelerate deep-neural net (DNN) training. For example, by scalin…