71 citations · 251 across the 8 of their papers we have counts for
8 papers
Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads
John Thorpe, Yifan Qiao, Jonathan Eyolfson +8
A graph neural network (GNN) enables deep learning on structured graph data. There are two major GNN training obstacles: 1) it relies on high-end servers with many GPUs which are e…
Priority-based Parameter Propagation for Distributed DNN Training
Anand Jayarajan, Jinliang Wei, Garth Gibson +2
Data parallel training is widely used for scaling distributed deep neural network (DNN) training. However, the performance benefits are often limited by the communication-heavy par…
Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters
Hao Zhang, Zeyu Zheng, Shizhen Xu +7
Deep learning models can take weeks to train on a single GPU-equipped machine, necessitating scaling out DL training to a GPU-cluster. However, current distributed DL implementatio…
Poseidon: A System Architecture for Efficient GPU-based Deep Learning on Multiple Machines
Hao Zhang, Zhiting Hu, Jinliang Wei +4
Deep learning (DL) has achieved notable successes in many machine learning tasks. A number of frameworks have been developed to expedite the process of designing and training deep…
LightLDA: Big Topic Models on Modest Compute Clusters
Jinhui Yuan, Fei Gao, Qirong Ho +6
When building large-scale machine learning (ML) programs, such as big topic models or deep neural nets, one usually assumes such tasks can only be attempted with industrial-sized c…
High-Performance Distributed ML at Scale through Parameter Server Consistency Models
Wei Dai, Abhimanu Kumar, Jinliang Wei +3
As Machine Learning (ML) applications increase in data size and model complexity, practitioners turn to distributed clusters to satisfy the increased computational and memory deman…
Consistent Bounded-Asynchronous Parameter Servers for Distributed ML
Jinliang Wei, Wei Dai, Abhimanu Kumar +3
In distributed ML applications, shared parameters are usually replicated among computing nodes to minimize network overhead. Therefore, proper consistency model must be carefully c…
Petuum: A New Platform for Distributed Machine Learning on Big Data
Eric P. Xing, Qirong Ho, Wei Dai +7
What is a systematic way to efficiently apply a wide spectrum of advanced ML programs to industrial scale problems, using Big Models (up to 100s of billions of parameters) on Big D…