71 citations · 128 across the 4 of their papers we have counts for
4 papers
Priority-based Parameter Propagation for Distributed DNN Training
Anand Jayarajan, Jinliang Wei, Garth Gibson +2
Data parallel training is widely used for scaling distributed deep neural network (DNN) training. However, the performance benefits are often limited by the communication-heavy par…
High-Performance Distributed ML at Scale through Parameter Server Consistency Models
Wei Dai, Abhimanu Kumar, Jinliang Wei +3
As Machine Learning (ML) applications increase in data size and model complexity, practitioners turn to distributed clusters to satisfy the increased computational and memory deman…
Primitives for Dynamic Big Model Parallelism
Seunghak Lee, Jin Kyu Kim, Xun Zheng +3
When training large machine learning models with many variables or parameters, a single machine is often inadequate since the model may be too large to fit in memory, while trainin…
Structure-Aware Dynamic Scheduler for Parallel Machine Learning
Seunghak Lee, Jin Kyu Kim, Qirong Ho +2
Training large machine learning (ML) models with many variables or parameters can take a long time if one employs sequential procedures even with stochastic updates. A natural solu…