1.8k citations · 9.5k across the 166 of their papers we have counts for
4 papers · 1 filter
RedSync : Reducing Synchronization Traffic for Distributed Deep Learning
Jiarui Fang, Haohuan Fu, Guangwen Yang +1
Data parallelism has become a dominant method to scale Deep Neural Network (DNN) training across multiple nodes. Since synchronizing a large number of gradients of the local model…
Accurate, Fast and Scalable Kernel Ridge Regression on Parallel and Distributed Systems
Yang You, James Demmel, Cho-Jui Hsieh +1
We propose two new methods to address the weak scaling problems of KRR: the Balanced KRR (BKRR) and K-means KRR (KKRR). These methods consider alternative ways to partition the inp…
A Scalable Asynchronous Distributed Algorithm for Topic Modeling
Hsiang-Fu Yu, Cho-Jui Hsieh, Hyokun Yun +2
Learning meaningful topic models with massive document collections which contain millions of documents and billions of tokens is challenging because of two reasons: First, one need…
NOMAD: Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion
Hyokun Yun, Hsiang-Fu Yu, Cho-Jui Hsieh +2
We develop an efficient parallel distributed algorithm for matrix completion, named NOMAD (Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix…