GraphLab: A New Framework for Parallel Machine Learning

Abstract

Designing and implementing efficient, provably correct parallel machine learning (ML) algorithms is challenging. Existing high-level parallel abstractions like MapReduce are insufficiently expressive while low-level tools like MPI and Pthreads leave ML experts repeatedly solving the same design challenges. By targeting common patterns in ML, we developed GraphLab, which improves upon abstractions like MapReduce by compactly expressing asynchronous iterative algorithms with sparse computational dependencies while ensuring data consistency and achieving a high degree of parallel performance. We demonstrate the expressiveness of the GraphLab framework by designing and implementing parallel versions of belief propagation, Gibbs sampling, Co-EM, Lasso and Compressed Sensing. We show that using GraphLab we can achieve excellent parallel performance on large scale real-world problems.

References in corpus (1)

• Residual Belief Propagation: Informed Scheduling for Asynchronous Message Passing

Cited by in corpus (24)

• A Reliable Effective Terascale Linear Learning System
• Theoretically Efficient Parallel Graph Algorithms Can Be Fast and Scalable
• NXgraph: An Efficient Graph Processing System on a Single Machine
• Scaling Datalog for Machine Learning on Big Data
• Fast K-Means with Accurate Bounds
• CONDENSE: A Reconfigurable Knowledge Acquisition Architecture for Future 5G IoT
• xDGP: A Dynamic Graph Processing System with Adaptive Partitioning
• An analysis of the graph processing landscape
• fastFM: A Library for Factorization Machines
• Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads
• Scaling Up Coordinate Descent Algorithms for Large ${\ell }_1$ Regularization Problems
• A Linear Logic Programming Language for Concurrent Programming over Graph Structures
• Mathematical and Algorithmic Analysis of Network and Biological Data
• RankMap: A Platform-Aware Framework for Distributed Learning from Dense Datasets
• S-PowerGraph: Streaming Graph Partitioning for Natural Graphs by Vertex-Cut
• Sparse Allreduce: Efficient Scalable Communication for Power-Law Data
• In-Depth Benchmarking of Graph Database Systems with the Linked Data Benchmark Council (LDBC) Social Network Benchmark (SNB)
• Kaskade: Graph Views for Efficient Graph Analytics
• Relaxed Scheduling for Scalable Belief Propagation
• Accelerating Graph Sampling for Graph Machine Learning using GPUs
• BoPF: Mitigating the Burstiness-Fairness Tradeoff in Multi-Resource Clusters
• Compilation Techniques for Graph Algorithms on GPUs
• An Improved and Optimized Practical Non-Blocking PageRank Algorithm for Massive Graphs
• Distributed Data Processing Frameworks for Big Graph Data