282 citations · 1k across the 43 of their papers we have counts for
14 papers · 1 filter
Computron: Serving Distributed Deep Learning Models with Model Parallel Swapping
Daniel Zou, Xinchen Jin, Xueyang Yu +2
Many of the most performant deep learning models today in fields like language and image understanding are fine-tuned models that contain billions of parameters. In anticipation of…
Communication Bounds for Convolutional Neural Networks
Anthony Chen, James Demmel, Grace Dinh +2
Convolutional neural networks (CNNs) are important in a wide variety of machine learning tasks and applications, so optimizing their performance is essential. Moving words of data…
Distributed-Memory Sparse Kernels for Machine Learning
Vivek Bharadwaj, Aydin Buluç, James Demmel
Sampled Dense Times Dense Matrix Multiplication (SDDMM) and Sparse Times Dense Matrix Multiplication (SpMM) appear in diverse settings, such as collaborative filtering, document cl…
Auto-Precision Scaling for Distributed Deep Learning
Ruobing Han, James Demmel, Yang You
It has been reported that the communication cost for synchronizing gradients can be a bottleneck, which limits the scalability of distributed deep learning. Using low-precision gra…
A 3D Parallel Algorithm for QR Decomposition
Grey Ballard, James Demmel, Laura Grigori +2
Interprocessor communication often dominates the runtime of large matrix computations. We present a parallel algorithm for computing QR decompositions whose bandwidth cost (communi…
Accurate, Fast and Scalable Kernel Ridge Regression on Parallel and Distributed Systems
Yang You, James Demmel, Cho-Jui Hsieh +1
We propose two new methods to address the weak scaling problems of KRR: the Balanced KRR (BKRR) and K-means KRR (KKRR). These methods consider alternative ways to partition the inp…