Transductive Information Maximization For Few-Shot Learning

Abstract

We introduce Transductive Infomation Maximization (TIM) for few-shot learning. Our method maximizes the mutual information between the query features and their label predictions for a given few-shot task, in conjunction with a supervision loss based on the support set. Furthermore, we propose a new alternating-direction solver for our mutual-information loss, which substantially speeds up transductive-inference convergence over gradient-based optimization, while yielding similar accuracy. TIM inference is modular: it can be used on top of any base-training feature extractor. Following standard transductive few-shot settings, our comprehensive experiments demonstrate that TIM outperforms state-of-the-art methods significantly across various datasets and networks, while used on top of a fixed feature extractor trained with simple cross-entropy on the base classes, without resorting to complex meta-learning schemes. It consistently brings between 2% and 5% improvement in accuracy over the best performing method, not only on all the well-established few-shot benchmarks but also on more challenging scenarios,with domain shifts and larger numbers of classes.

NeurIPS 2020. Code available at https://github.com/mboudiaf/TIM

References in corpus (12)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Learning deep representations by mutual information estimation and maximization
• A Simple Neural Attentive Meta-Learner
• Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
• On First-Order Meta-Learning Algorithms
• Learning Discrete Representations via Information Maximizing Self-Augmented Training
• SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning
• Learning to Self-Train for Semi-Supervised Few-Shot Classification
• Cross Attention Network for Few-shot Classification
• Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
• Laplacian Regularized Few-Shot Learning

Cited by in corpus (8)

• MHFC: Multi-Head Feature Collaboration for Few-Shot Learning
• GCT: Graph Co-Training for Semi-Supervised Few-Shot Learning
• PTransIPs: Identification of phosphorylation sites enhanced by protein PLM embeddings
• Transductive Few-Shot Learning: Clustering is All You Need?
• A Strong Baseline for Semi-Supervised Incremental Few-Shot Learning
• MDFM: Multi-Decision Fusing Model for Few-Shot Learning
• Transductive image segmentation: Self-training and effect of uncertainty estimation
• Few-shot Learning via Dependency Maximization and Instance Discriminant Analysis