ReFine: Re-randomization before Fine-tuning for Cross-domain Few-shot Learning

Abstract

Cross-domain few-shot learning (CD-FSL), where there are few target samples under extreme differences between source and target domains, has recently attracted huge attention. Recent studies on CD-FSL generally focus on transfer learning based approaches, where a neural network is pre-trained on popular labeled source domain datasets and then transferred to target domain data. Although the labeled datasets may provide suitable initial parameters for the target data, the domain difference between the source and target might hinder fine-tuning on the target domain. This paper proposes a simple yet powerful method that re-randomizes the parameters fitted on the source domain before adapting to the target data. The re-randomization resets source-specific parameters of the source pre-trained model and thus facilitates fine-tuning on the target domain, improving few-shot performance.

arXiv commentCIKM 2022 Short; 5 pages, 3 figures, 4 tables

References in corpus (19)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Generalizing from a Few Examples: A Survey on Few-Shot Learning
• Meta-Learning for Semi-Supervised Few-Shot Classification
• Transfusion: Understanding Transfer Learning for Medical Imaging
• Fast Context Adaptation via Meta-Learning
• Are All Layers Created Equal?
• Meta-Learning with Warped Gradient Descent
• BOIL: Towards Representation Change for Few-shot Learning
• What Do Neural Networks Learn When Trained With Random Labels?
• Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficulty
• Multi-level Metric Learning for Few-shot Image Recognition
• Backbone Can Not be Trained at Once: Rolling Back to Pre-trained Network for Person Re-Identification
• The Impact of Reinitialization on Generalization in Convolutional Neural Networks
• HIDRA: Head Initialization across Dynamic targets for Robust Architectures