Towards Dynamic and Safe Configuration Tuning for Cloud Databases

Abstract

Configuration knobs of database systems are essential to achieve high throughput and low latency. Recently, automatic tuning systems using machine learning methods (ML) have shown to find better configurations compared to experienced database administrators (DBAs). However, there are still gaps to apply the existing systems in production environments, especially in the cloud. First, they conduct tuning for a given workload within a limited time window and ignore the dynamicity of workloads and data. Second, they rely on a copied instance and do not consider the availability of the database when sampling configurations, making the tuning expensive, delayed, and unsafe. To fill these gaps, we propose OnlineTune, which tunes the online databases safely in changing cloud environments. To accommodate the dynamicity, OnlineTune embeds the environmental factors as context feature and adopts contextual Bayesian Optimization with context space partition to optimize the database adaptively and scalably. To pursue safety during tuning, we leverage the black-box and the white-box knowledge to evaluate the safety of configurations and propose a safe exploration strategy via subspace adaptation.%, greatly decreasing the risks of applying bad configurations. We conduct evaluations on dynamic workloads from benchmarks and real-world workloads. Compared with the state-of-the-art methods, OnlineTune achieves 14.4%~165.3% improvement on cumulative performance while reducing 91.0%~99.5% unsafe configuration recommendations.

References in corpus (17)

• Sequence to Sequence Learning with Neural Networks
• Practical Bayesian Optimization of Machine Learning Algorithms
• Neo: A Learned Query Optimizer
• Bayesian Optimization with Unknown Constraints
• BestConfig: Tapping the Performance Potential of Systems via Automatic Configuration Tuning
• Deep Unsupervised Cardinality Estimation
• Learned Cardinalities: Estimating Correlated Joins with Deep Learning
• Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization
• Batched Large-scale Bayesian Optimization in High-dimensional Spaces
• Transfer Learning via Contextual Invariants for One-to-Many Cross-Domain Recommendation
• Black or White? How to Develop an AutoTuner for Memory-based Analytics [Extended Version]
• Towards Dynamic and Safe Configuration Tuning for Cloud Databases
• Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspaces
• NeuroCard: One Cardinality Estimator for All Tables
• DeepDB: Learn from Data, not from Queries!
• DBA bandits: Self-driving index tuning under ad-hoc, analytical workloads with safety guarantees
• CobBO: Coordinate Backoff Bayesian Optimization with Two-Stage Kernels

Cited by in corpus (6)

• Towards Dynamic and Safe Configuration Tuning for Cloud Databases
• Towards General and Efficient Online Tuning for Spark
• NeurDB: An AI-powered Autonomous Data System
• Accuracy Can Lie: On the Impact of Surrogate Model in Configuration Tuning
• TUNA: Tuning Unstable and Noisy Cloud Applications
• A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning