Interventional Few-Shot Learning

Abstract

We uncover an ever-overlooked deficiency in the prevailing Few-Shot Learning (FSL) methods: the pre-trained knowledge is indeed a confounder that limits the performance. This finding is rooted from our causal assumption: a Structural Causal Model (SCM) for the causalities among the pre-trained knowledge, sample features, and labels. Thanks to it, we propose a novel FSL paradigm: Interventional Few-Shot Learning (IFSL). Specifically, we develop three effective IFSL algorithmic implementations based on the backdoor adjustment, which is essentially a causal intervention towards the SCM of many-shot learning: the upper-bound of FSL in a causal view. It is worth noting that the contribution of IFSL is orthogonal to existing fine-tuning and meta-learning based FSL methods, hence IFSL can improve all of them, achieving a new 1-/5-shot state-of-the-art on \textit{mini}ImageNet, \textit{tiered}ImageNet, and cross-domain CUB. Code is released at https://github.com/yue-zhongqi/ifsl.

arXiv commentAccepted by NeurIPS 2020

References in corpus (16)

• Distilling the Knowledge in a Neural Network
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Prototypical Networks for Few-shot Learning
• How transferable are features in deep neural networks?
• Data Augmentation Generative Adversarial Networks
• What makes ImageNet good for transfer learning?
• SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning
• Causal Confusion in Imitation Learning
• Cross Attention Network for Few-shot Classification
• An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning
• Learning Independent Causal Mechanisms
• Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions
• LGM-Net: Learning to Generate Matching Networks for Few-Shot Learning
• An empirical study of pretrained representations for few-shot classification
• Transfer learning from language models to image caption generators: Better models may not transfer better

Cited by in corpus (13)

• Causal Intervention for Weakly-Supervised Semantic Segmentation
• Should Graph Convolution Trust Neighbors? A Simple Causal Inference Method
• Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect
• Rectifying the Shortcut Learning of Background for Few-Shot Learning
• MHFC: Multi-Head Feature Collaboration for Few-Shot Learning
• Assaying Out-Of-Distribution Generalization in Transfer Learning
• Counterfactual VQA: A Cause-Effect Look at Language Bias
• Benign Shortcut for Debiasing: Fair Visual Recognition via Intervention with Shortcut Features
• Conterfactual Generative Zero-Shot Semantic Segmentation
• Counterfactual Variable Control for Robust and Interpretable Question Answering
• Confounder Identification-free Causal Visual Feature Learning
• Improving Weakly-supervised Object Localization via Causal Intervention
• Towards Unbiased Visual Emotion Recognition via Causal Intervention