An Empirical Analysis of Deep Learning for Cardinality Estimation
arXiv:1905.06425
Abstract
We implement and evaluate deep learning for cardinality estimation by studying the accuracy, space and time trade-offs across several architectures. We find that simple deep learning models can learn cardinality estimations across a variety of datasets (reducing the error by 72% - 98% on average compared to PostgreSQL). In addition, we empirically evaluate the impact of injecting cardinality estimates produced by deep learning models into the PostgreSQL optimizer. In many cases, the estimates from these models lead to better query plans across all datasets, reducing the runtimes by up to 49% on select-project-join workloads. As promising as these models are, we also discuss and address some of the challenges of using them in practice.
References in corpus (10)
- Scikit-learn: Machine Learning in Python
- A Brief Survey of Deep Reinforcement Learning
- State Representation Learning for Control: An Overview
- Narrowing the Gap: Random Forests In Theory and In Practice
- Database Meets Deep Learning: Challenges and Opportunities
- Learning Active Learning from Data
- Learning to Optimize Join Queries With Deep Reinforcement Learning
- Towards a Hands-Free Query Optimizer through Deep Learning
- Multi-Attribute Selectivity Estimation Using Deep Learning
- Unsupervised state representation learning with robotic priors: a robustness benchmark
Cited by in corpus (10)
- Spitz: A Verifiable Database System
- Monotonic Cardinality Estimation of Similarity Selection: A Deep Learning Approach
- Consistent and Flexible Selectivity Estimation for High-Dimensional Data
- Flow-Loss: Learning Cardinality Estimates That Matter
- LMKG: Learned Models for Cardinality Estimation in Knowledge Graphs
- ML-AQP: Query-Driven Approximate Query Processing based on Machine Learning
- Efficient Deep Learning Pipelines for Accurate Cost Estimations Over Large Scale Query Workload
- Online Sketch-based Query Optimization
- Experience-Enhanced Learning: One Size Still does not Fit All in Automatic Database
- Simpli-Squared: A Very Simple Yet Unexpectedly Powerful Join Ordering Algorithm Without Cardinality Estimates