Meta-Learning with Latent Embedding Optimization

Abstract

Gradient-based meta-learning techniques are both widely applicable and proficient at solving challenging few-shot learning and fast adaptation problems. However, they have practical difficulties when operating on high-dimensional parameter spaces in extreme low-data regimes. We show that it is possible to bypass these limitations by learning a data-dependent latent generative representation of model parameters, and performing gradient-based meta-learning in this low-dimensional latent space. The resulting approach, latent embedding optimization (LEO), decouples the gradient-based adaptation procedure from the underlying high-dimensional space of model parameters. Our evaluation shows that LEO can achieve state-of-the-art performance on the competitive miniImageNet and tieredImageNet few-shot classification tasks. Further analysis indicates LEO is able to capture uncertainty in the data, and can perform adaptation more effectively by optimizing in latent space.

References in corpus (17)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning in Neural Networks: An Overview
• Sequence to Sequence Learning with Neural Networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• How transferable are features in deep neural networks?
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
• TADAM: Task dependent adaptive metric for improved few-shot learning
• Using Fast Weights to Attend to the Recent Past
• Deep Meta-Learning: Learning to Learn in the Concept Space
• Dynamic Few-Shot Visual Learning without Forgetting
• Discriminative k-shot learning using probabilistic models
• Probabilistic Model-Agnostic Meta-Learning
• Few-Shot Image Recognition by Predicting Parameters from Activations
• Meta-learning autoencoders for few-shot prediction
• Uncertainty in Multitask Transfer Learning

Cited by in corpus (79)

• Compressed Sensing with Deep Image Prior and Learned Regularization
• Incremental Object Detection via Meta-Learning
• Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
• Generalized Inner Loop Meta-Learning
• FedBABU: Towards Enhanced Representation for Federated Image Classification
• GradNet: Gradient-Guided Network for Visual Object Tracking
• Learning from Few Samples: A Survey
• Leveraging the Feature Distribution in Transfer-based Few-Shot Learning
• Meta-Graph: Few Shot Link Prediction via Meta Learning
• Torchmeta: A Meta-Learning library for PyTorch
• Auto-Meta: Automated Gradient Based Meta Learner Search
• Meta-Learning Symmetries by Reparameterization
• A Baseline for Few-Shot Image Classification
• Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabeling
• Few-Shot Video Classification via Temporal Alignment
• Learning Dynamic Alignment via Meta-filter for Few-shot Learning
• When Does Self-supervision Improve Few-shot Learning?
• Multi-scale Adaptive Task Attention Network for Few-Shot Learning
• Unsupervised Few-shot Learning via Self-supervised Training
• Few-shot Classification via Adaptive Attention
• Reconciling meta-learning and continual learning with online mixtures of tasks
• A Theoretical Analysis of the Number of Shots in Few-Shot Learning
• Selecting Relevant Features from a Multi-domain Representation for Few-shot Classification
• Learning to learn via Self-Critique
• Meta-Learning surrogate models for sequential decision making
• Weak-shot Fine-grained Classification via Similarity Transfer
• Instance Credibility Inference for Few-Shot Learning
• Few-Shot Learning as Domain Adaptation: Algorithm and Analysis
• Is Support Set Diversity Necessary for Meta-Learning?
• Online Structured Meta-learning
• Graph-based Interpolation of Feature Vectors for Accurate Few-Shot Classification
• Learning from the Past: Continual Meta-Learning via Bayesian Graph Modeling
• Task-similarity Aware Meta-learning through Nonparametric Kernel Regression
• The Differentiable Cross-Entropy Method
• Robust Meta-learning for Mixed Linear Regression with Small Batches
• Meta-Learning Initializations for Image Segmentation
• Generalized Adaptation for Few-Shot Learning
• Dizygotic Conditional Variational AutoEncoder for Multi-Modal and Partial Modality Absent Few-Shot Learning
• Guarantees for Tuning the Step Size using a Learning-to-Learn Approach
• MetaDelta: A Meta-Learning System for Few-shot Image Classification
• Chameleon: Learning Model Initializations Across Tasks With Different Schemas
• Unsupervised Partial Point Set Registration via Joint Shape Completion and Registration
• Projective Latent Interventions for Understanding and Fine-tuning Classifiers
• An empirical study of pretrained representations for few-shot classification
• One Shot Learning for Speech Separation
• Spirit Distillation: A Model Compression Method with Multi-domain Knowledge Transfer
• Decoder Choice Network for Meta-Learning
• ReMP: Rectified Metric Propagation for Few-Shot Learning
• Conditional Meta-Learning of Linear Representations
• The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuning
• Self-Supervised Tuning for Few-Shot Segmentation
• Physics-aware Spatiotemporal Modules with Auxiliary Tasks for Meta-Learning
• Improving Few-shot Learning by Spatially-aware Matching and CrossTransformer
• High-order structure preserving graph neural network for few-shot learning
• Meta-learning for mixed linear regression
• Task-Adaptive Feature Transformer for Few-Shot Segmentation
• Learning Meta-class Memory for Few-Shot Semantic Segmentation
• Single-View 3D Object Reconstruction from Shape Priors in Memory
• Hierarchical Meta Learning
• Task Affinity with Maximum Bipartite Matching in Few-Shot Learning
• Learning Classifiers for Domain Adaptation, Zero and Few-Shot Recognition Based on Learning Latent Semantic Parts
• Will Multi-modal Data Improves Few-shot Learning?
• A Few Shot Adaptation of Visual Navigation Skills to New Observations using Meta-Learning
• Annotation-Efficient Untrimmed Video Action Recognition
• Variable-Shot Adaptation for Online Meta-Learning
• A Meta-learning Approach to Reservoir Computing: Time Series Prediction with Limited Data
• Training few-shot classification via the perspective of minibatch and pretraining
• Adaptation-Agnostic Meta-Training
• Fairness-Aware Online Meta-learning
• One-Shot Object Affordance Detection in the Wild
• Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learning
• Mutual-Information Based Few-Shot Classification
• Multi-accent Speech Separation with One Shot Learning
• Domain Agnostic Few-Shot Learning For Document Intelligence
• MetaMix: Improved Meta-Learning with Interpolation-based Consistency Regularization
• Meta-Learning to Improve Pre-Training
• Few-shot Learning via Dependency Maximization and Instance Discriminant Analysis
• Fair Meta-Learning For Few-Shot Classification
• Diversity Transfer Network for Few-Shot Learning