Distributed GraphLab: A Framework for Machine Learning in the Cloud

Abstract

While high-level data parallel frameworks, like MapReduce, simplify the design and implementation of large-scale data processing systems, they do not naturally or efficiently support many important data mining and machine learning algorithms and can lead to inefficient learning systems. To help fill this critical void, we introduced the GraphLab abstraction which naturally expresses asynchronous, dynamic, graph-parallel computation while ensuring data consistency and achieving a high degree of parallel performance in the shared-memory setting. In this paper, we extend the GraphLab framework to the substantially more challenging distributed setting while preserving strong data consistency guarantees. We develop graph based extensions to pipelined locking and data versioning to reduce network congestion and mitigate the effect of network latency. We also introduce fault tolerance to the GraphLab abstraction using the classic Chandy-Lamport snapshot algorithm and demonstrate how it can be easily implemented by exploiting the GraphLab abstraction itself. Finally, we evaluate our distributed implementation of the GraphLab abstraction on a large Amazon EC2 deployment and show 1-2 orders of magnitude performance gains over Hadoop-based implementations.

VLDB2012

References in corpus (11)

• Rejoinder to "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Residual Belief Propagation: Informed Scheduling for Asynchronous Message Passing

Cited by in corpus (11)

• The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey
• Machine Learning and Cloud Computing: Survey of Distributed and SaaS Solutions
• Graph Partitioning via Parallel Submodular Approximation to Accelerate Distributed Machine Learning
• Fast and Efficient Parallel Breadth-First Search with Power-law Graph Transformation
• An Efficient Pruning Process with Locality Aware Exploration and Dynamic Graph Editing for Subgraph Matching
• ThunderRW: An In-Memory Graph Random Walk Engine (Complete Version)
• Technical Report: Developing a Working Data Hub
• Heterogeneous-Reliability Memory: Exploiting Application-Level Memory Error Tolerance
• REPT: A Streaming Algorithm of Approximating Global and Local Triangle Counts in Parallel
• Pipelined Training with Stale Weights of Deep Convolutional Neural Networks
• Robust Lower Bounds for Graph Problems in the Blackboard Model of Communication