Splash: User-friendly Programming Interface for Parallelizing Stochastic Algorithms
arXiv:1506.07552
Abstract
Stochastic algorithms are efficient approaches to solving machine learning and optimization problems. In this paper, we propose a general framework called Splash for parallelizing stochastic algorithms on multi-node distributed systems. Splash consists of a programming interface and an execution engine. Using the programming interface, the user develops sequential stochastic algorithms without concerning any detail about distributed computing. The algorithm is then automatically parallelized by a communication-efficient execution engine. We provide theoretical justifications on the optimal rate of convergence for parallelizing stochastic gradient descent. Splash is built on top of Apache Spark. The real-data experiments on logistic regression, collaborative filtering and topic modeling verify that Splash yields order-of-magnitude speedup over single-thread stochastic algorithms and over state-of-the-art implementations on Spark.
redo experiments to learn bigger models; compare Splash with state-of-the-art implementations on Spark
References in corpus (7)
- BPR: Bayesian Personalized Ranking from Implicit Feedback
- Expectation Propagation for approximate Bayesian inference
- MLlib: Machine Learning in Apache Spark
- Communication-Efficient Distributed Dual Coordinate Ascent
- MLI: An API for Distributed Machine Learning
- SAME but Different: Fast and High-Quality Gibbs Parameter Estimation
- Petuum: A New Platform for Distributed Machine Learning on Big Data
Cited by in corpus (6)
- Revisiting Distributed Synchronous SGD
- A Survey on Spark Ecosystem for Big Data Processing
- SparkNet: Training Deep Networks in Spark
- Hemingway: Modeling Distributed Optimization Algorithms
- Parallelizing Word2Vec in Shared and Distributed Memory
- SCOPE: Scalable Composite Optimization for Learning on Spark