Meta-Learning Initializations for Image Segmentation
arXiv:1912.06290
Abstract
We extend first-order model agnostic meta-learning algorithms (including FOMAML and Reptile) to image segmentation, present a novel neural network architecture built for fast learning which we call EfficientLab, and leverage a formal definition of the test error of meta-learning algorithms to decrease error on out of distribution tasks. We show state of the art results on the FSS-1000 dataset by meta-training EfficientLab with FOMAML and using Bayesian optimization to infer the optimal test-time adaptation routine hyperparameters. We also construct a small benchmark dataset, FP-k, for the empirical study of how meta-learning systems perform in both few- and many-shot settings. On the FP-k dataset, we show that meta-learned initializations provide value for canonical few-shot image segmentation but their performance is quickly matched by conventional transfer learning with performance being equal beyond 10 labeled examples. Our code, meta-learned model, and the FP-k dataset are available at https://github.com/ml4ai/mliis .
References in corpus (14)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Practical Bayesian Optimization of Machine Learning Algorithms
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
- Scaling Laws for Neural Language Models
- A Simple Neural Attentive Meta-Learner
- Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
- Random Erasing Data Augmentation
- FSS-1000: A 1000-Class Dataset for Few-Shot Segmentation
- Meta-Learning with Latent Embedding Optimization
- Using Fast Weights to Attend to the Recent Past
- Satellite Imagery Feature Detection using Deep Convolutional Neural Network: A Kaggle Competition
- A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
- How to train your MAML
- Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian