Towards a Hands-Free Query Optimizer through Deep Learning

Abstract

Query optimization remains one of the most important and well-studied problems in database systems. However, traditional query optimizers are complex heuristically-driven systems, requiring large amounts of time to tune for a particular database and requiring even more time to develop and maintain in the first place. In this vision paper, we argue that a new type of query optimizer, based on deep reinforcement learning, can drastically improve on the state-of-the-art. We identify potential complications for future research that integrates deep learning with query optimization, and we describe three novel deep learning based approaches that can lead the way to end-to-end learning-based query optimizers.

arXiv commentPublished in CIDR19

References in corpus (3)

• Deep Learning in Neural Networks: An Overview
• Deep Reinforcement Learning for Join Order Enumeration
• LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations

Cited by in corpus (12)

• Neo: A Learned Query Optimizer
• Database Meets Deep Learning: Challenges and Opportunities
• Learning to Optimize Join Queries With Deep Reinforcement Learning
• An Empirical Analysis of Deep Learning for Cardinality Estimation
• Learning Index Selection with Structured Action Spaces
• LMKG: Learned Models for Cardinality Estimation in Knowledge Graphs
• A Pluggable Learned Index Method via Sampling and Gap Insertion
• Join Query Optimization with Deep Reinforcement Learning Algorithms
• CARMI: A Cache-Aware Learned Index with a Cost-based Construction Algorithm
• Flexible Operator Embeddings via Deep Learning
• Online Sketch-based Query Optimization
• A Comparative Exploration of ML Techniques for Tuning Query Degree of Parallelism