Meta-Learning surrogate models for sequential decision making

Abstract

We introduce a unified probabilistic framework for solving sequential decision making problems ranging from Bayesian optimisation to contextual bandits and reinforcement learning. This is accomplished by a probabilistic model-based approach that explains observed data while capturing predictive uncertainty during the decision making process. Crucially, this probabilistic model is chosen to be a Meta-Learning system that allows learning from a distribution of related problems, allowing data efficient adaptation to a target task. As a suitable instantiation of this framework, we explore the use of Neural processes due to statistical and computational desiderata. We apply our framework to a broad range of problem domains, such as control problems, recommender systems and adversarial attacks on RL agents, demonstrating an efficient and general black-box learning approach.

References in corpus (21)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Practical Bayesian Optimization of Machine Learning Algorithms
• Weight Uncertainty in Neural Networks
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Learning to reinforcement learn
• Distributed Distributional Deterministic Policy Gradients
• Federated Meta-Learning with Fast Convergence and Efficient Communication
• Meta-Learning with Latent Embedding Optimization
• Learning to Learn without Gradient Descent by Gradient Descent
• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
• Towards a Neural Statistician
• Unsupervised Predictive Memory in a Goal-Directed Agent
• Meta Reinforcement Learning with Latent Variable Gaussian Processes
• Discriminative k-shot learning using probabilistic models
• Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions
• Guided Meta-Policy Search
• The Variational Homoencoder: Learning to learn high capacity generative models from few examples
• Multi-task Deep Reinforcement Learning with PopArt
• Probabilistic Model-Agnostic Meta-Learning
• VFunc: a Deep Generative Model for Functions

Cited by in corpus (4)

• Probabilistic Active Meta-Learning
• Group Equivariant Conditional Neural Processes
• Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes
• Model-based Meta Reinforcement Learning using Graph Structured Surrogate Models