Declarative Recursive Computation on an RDBMS, or, Why You Should Use a Database For Distributed Machine Learning
arXiv:1904.11121 · doi:10.14778/3317315.3317323
Abstract
A number of popular systems, most notably Google's TensorFlow, have been implemented from the ground up to support machine learning tasks. We consider how to make a very small set of changes to a modern relational database management system (RDBMS) to make it suitable for distributed learning computations. Changes include adding better support for recursion, and optimization and execution of very large compute plans. We also show that there are key advantages to using an RDBMS as a machine learning platform. In particular, learning based on a database management system allows for trivial scaling to large data sets and especially large models, where different computational units operate on different parts of a model that may be too large to fit into RAM.
References in corpus (12)
- Efficient Estimation of Word Representations in Vector Space
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- TensorFlow: A system for large-scale machine learning
- An overview of gradient descent optimization algorithms
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
- One weird trick for parallelizing convolutional neural networks
- Revisiting Distributed Synchronous SGD
- DyNet: The Dynamic Neural Network Toolkit
- Poseidon: A System Architecture for Efficient GPU-based Deep Learning on Multiple Machines
- AMPNet: Asynchronous Model-Parallel Training for Dynamic Neural Networks