Adversarial Monte Carlo Meta-Learning of Optimal Prediction Procedures

Abstract

We frame the meta-learning of prediction procedures as a search for an optimal strategy in a two-player game. In this game, Nature selects a prior over distributions that generate labeled data consisting of features and an associated outcome, and the Predictor observes data sampled from a distribution drawn from this prior. The Predictor's objective is to learn a function that maps from a new feature to an estimate of the associated outcome. We establish that, under reasonable conditions, the Predictor has an optimal strategy that is equivariant to shifts and rescalings of the outcome and is invariant to permutations of the observations and to shifts, rescalings, and permutations of the features. We introduce a neural network architecture that satisfies these properties. The proposed strategy performs favorably compared to standard practice in both parametric and nonparametric experiments.

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Scikit-learn: Machine Learning in Python
• Explaining and Harnessing Adversarial Examples
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Matching Networks for One Shot Learning
• On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems
• Equivariance Through Parameter-Sharing
• Deep Models of Interactions Across Sets
• Adversarially Robust Few-Shot Learning: A Meta-Learning Approach
• Convergence of Learning Dynamics in Stackelberg Games
• Adversarial Meta-Learning
• Learning to Learn Neural Networks
• On relatively compact sets in quasi-Banach function spaces