Learning Index Selection with Structured Action Spaces
arXiv:1909.07440
Abstract
Configuration spaces for computer systems can be challenging for traditional and automatic tuning strategies. Injecting task-specific knowledge into the tuner for a task may allow for more efficient exploration of candidate configurations. We apply this idea to the task of index set selection to accelerate database workloads. Index set selection has been amenable to recent applications of vanilla deep RL, but real deployments remain out of reach. In this paper, we explore how learning index selection can be enhanced with task-specific inductive biases, specifically by encoding these inductive biases in better action structures. Index selection-specific action representations arise when the problem is reformulated in terms of permutation learning and we rely on recent work for learning RL policies on permutations. Through this approach, we build an indexing agent that is able to achieve improved indexing and validate its behavior with task-specific statistics. Early experiments reveal that our agent can find configurations that are up to 40% smaller for the same levels of latency as compared with other approaches and indicate more intuitive indexing behavior.
References in corpus (15)
- Efficient Estimation of Word Representations in Vector Space
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Prioritized Experience Replay
- Dueling Network Architectures for Deep Reinforcement Learning
- DARTS: Differentiable Architecture Search
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- Deep Reinforcement Learning: An Overview
- Device Placement Optimization with Reinforcement Learning
- Simple random search provides a competitive approach to reinforcement learning
- Learning Latent Permutations with Gumbel-Sinkhorn Networks
- Ranking via Sinkhorn Propagation
- Towards a Hands-Free Query Optimizer through Deep Learning
- The Case for Automatic Database Administration using Deep Reinforcement Learning
- Learning Permutations with Sinkhorn Policy Gradient
- LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations