Meta-learning autoencoders for few-shot prediction
arXiv:1807.09912
Abstract
Compared to humans, machine learning models generally require significantly more training examples and fail to extrapolate from experience to solve previously unseen challenges. To help close this performance gap, we augment single-task neural networks with a meta-recognition model which learns a succinct model code via its autoencoder structure, using just a few informative examples. The model code is then employed by a meta-generative model to construct parameters for the task-specific model. We demonstrate that for previously unseen tasks, without additional training, this Meta-Learning Autoencoder (MeLA) framework can build models that closely match the true underlying models, with loss significantly lower than given by fine-tuned baseline networks, and performance that compares favorably with state-of-the-art meta-learning algorithms. MeLA also adds the ability to identify influential training examples and predict which additional data will be most valuable to acquire to improve model prediction.
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Theoretical Models of Learning to Learn
- Matching Networks for One Shot Learning
- Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
- RL: Fast Reinforcement Learning via Slow Reinforcement Learning
- Learning to reinforcement learn
- Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics
- Towards a Neural Statistician
- A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
- Convolution by Evolution: Differentiable Pattern Producing Networks