GraphLab: A New Framework For Parallel Machine Learning

Abstract

Designing and implementing efficient, provably correct parallel machine learning (ML) algorithms is challenging. Existing high-level parallel abstractions like MapReduce are insufficiently expressive while low-level tools like MPI and Pthreads leave ML experts repeatedly solving the same design challenges. By targeting common patterns in ML, we developed GraphLab, which improves upon abstractions like MapReduce by compactly expressing asynchronous iterative algorithms with sparse computational dependencies while ensuring data consistency and achieving a high degree of parallel performance. We demonstrate the expressiveness of the GraphLab framework by designing and implementing parallel versions of belief propagation, Gibbs sampling, Co-EM, Lasso and Compressed Sensing. We show that using GraphLab we can achieve excellent parallel performance on large scale real-world problems.

Appears in Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (UAI2010)

References in corpus (1)

• Residual Belief Propagation: Informed Scheduling for Asynchronous Message Passing

Cited by in corpus (22)

• NILMTK: An Open Source Toolkit for Non-intrusive Load Monitoring
• Graph Processing on FPGAs: Taxonomy, Survey, Challenges
• CONDENSE: A Reconfigurable Knowledge Acquisition Architecture for Future 5G IoT
• Distributed Edge Partitioning for Graph Processing
• A Survey on Methods and Systems for Graph Compression
• Thinking Like a Vertex: a Survey of Vertex-Centric Frameworks for Distributed Graph Processing
• BigSparse: High-performance external graph analytics
• Fast Differentially Private Matrix Factorization
• Streaming Balanced Graph Partitioning for Random Graphs
• The Family of MapReduce and Large Scale Data Processing Systems
• Identifying Dwarfs Workloads in Big Data Analytics
• Purine: A bi-graph based deep learning framework
• Belief Propagation by Message Passing in Junction Trees: Computing Each Message Faster Using GPU Parallelization
• Spinning Fast Iterative Data Flows
• GraphR: Accelerating Graph Processing Using ReRAM
• Large Scale Estimation in Cyberphysical Systems using Streaming Data: a Case Study with Smartphone Traces
• Pruned Landmark Labeling Meets Vertex Centric Computation: A Surprisingly Happy Marriage!
• Distributed Graph Layout for Scalable Small-world Network Analysis
• Maiter: An Asynchronous Graph Processing Framework for Delta-based Accumulative Iterative Computation
• Utilizing Static Analysis and Code Generation to Accelerate Neural Networks
• Streaming Graph Partitioning in the Planted Partition Model
• Graphyti: A Semi-External Memory Graph Library for FlashGraph