On sensitivity of meta-learning to support data
arXiv:2110.13953
Abstract
Meta-learning algorithms are widely used for few-shot learning. For example, image recognition systems that readily adapt to unseen classes after seeing only a few labeled examples. Despite their success, we show that modern meta-learning algorithms are extremely sensitive to the data used for adaptation, i.e. support data. In particular, we demonstrate the existence of (unaltered, in-distribution, natural) images that, when used for adaptation, yield accuracy as low as 4\% or as high as 95\% on standard few-shot image classification benchmarks. We explain our empirical findings in terms of class margins, which in turn suggests that robust and safe meta-learning requires larger margins than supervised learning.
arXiv commentAccepted at NeurIPS 2021
References in corpus (15)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- Towards Deep Learning Models Resistant to Adversarial Attacks
- Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
- Matching Networks for One Shot Learning
- On First-Order Meta-Learning Algorithms
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- TADAM: Task dependent adaptive metric for improved few-shot learning
- Generalized Inner Loop Meta-Learning
- Adversarially Robust Few-Shot Learning: A Meta-Learning Approach
- learn2learn: A Library for Meta-Learning Research
- Adversarial Meta-Learning
- Is Support Set Diversity Necessary for Meta-Learning?
- Meta learning Framework for Automated Driving
- Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks