Plan-Structured Deep Neural Network Models for Query Performance Prediction

Abstract

Query performance prediction, the task of predicting the latency of a query, is one of the most challenging problem in database management systems. Existing approaches rely on features and performance models engineered by human experts, but often fail to capture the complex interactions between query operators and input relations, and generally do not adapt naturally to workload characteristics and patterns in query execution plans. In this paper, we argue that deep learning can be applied to the query performance prediction problem, and we introduce a novel neural network architecture for the task: a plan-structured neural network. Our approach eliminates the need for human-crafted feature selection and automatically discovers complex performance models both at the operator and query plan level. Our novel neural network architecture can match the structure of any optimizer-selected query execution plan and predict its latency with high accuracy. We also propose a number of optimizations that reduce training overhead without sacrificing effectiveness. We evaluated our techniques on various workloads and we demonstrate that our plan-structured neural network can outperform the state-of-the-art in query performance prediction.

References in corpus (7)

• Deep Learning in Neural Networks: An Overview
• Neural Discrete Representation Learning
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator
• Deep Reinforcement Learning for Join Order Enumeration
• Database Meets Deep Learning: Challenges and Opportunities
• WiSeDB: A Learning-based Workload Management Advisor for Cloud Databases

Cited by in corpus (9)

• Bao: Learning to Steer Query Optimizers
• Monotonic Cardinality Estimation of Similarity Selection: A Deep Learning Approach
• Fine-Grained Modeling and Optimization for Intelligent Resource Management in Big Data Processing
• Forecasting SQL Query Cost at Twitter
• How Good are Learned Cost Models, Really? Insights from Query Optimization Tasks
• Optimizing Dataflow Systems for Scalable Interactive Visualization
• Phoebe: A Learning-based Checkpoint Optimizer
• Blueprinting the Cloud: Unifying and Automatically Optimizing Cloud Data Infrastructures with BRAD -- Extended Version
• Machine Learning-based Cardinality Estimation in DBMS on Pre-Aggregated Data