Decoder Choice Network for Meta-Learning
arXiv:1909.11446
Abstract
Meta-learning has been widely used for implementing few-shot learning and fast model adaptation. One kind of meta-learning methods attempt to learn how to control the gradient descent process in order to make the gradient-based learning have high speed and generalization. This work proposes a method that controls the gradient descent process of the model parameters of a neural network by limiting the model parameters in a low-dimensional latent space. The main challenge of this idea is that a decoder with too many parameters is required. This work designs a decoder with typical structure and shares a part of weights in the decoder to reduce the number of the required parameters. Besides, this work has introduced ensemble learning to work with the proposed approach for improving performance. The results show that the proposed approach is witnessed by the superior performance over the Omniglot classification and the miniImageNet classification tasks.
arXiv comment13papers
References in corpus (13)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
- Wide Residual Networks
- A Simple Neural Attentive Meta-Learner
- Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
- Meta-Learning with Latent Embedding Optimization
- TADAM: Task dependent adaptive metric for improved few-shot learning
- Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments
- Towards a Neural Statistician
- Discriminative k-shot learning using probabilistic models
- Learning to Support: Exploiting Structure Information in Support Sets for One-Shot Learning