Enabling the Network to Surf the Internet
arXiv:2102.12205
Abstract
Few-shot learning is challenging due to the limited data and labels. Existing algorithms usually resolve this problem by pre-training the model with a considerable amount of annotated data which shares knowledge with the target domain. Nevertheless, large quantities of homogenous data samples are not always available. To tackle this issue, we develop a framework that enables the model to surf the Internet, which implies that the model can collect and annotate data without manual effort. Since the online data is virtually limitless and continues to be generated, the model can thus be empowered to constantly obtain up-to-date knowledge from the Internet. Additionally, we observe that the generalization ability of the learned representation is crucial for self-supervised learning. To present its importance, a naive yet efficient normalization strategy is proposed. Consequentially, this strategy boosts the accuracy of the model significantly (20.46% at most). We demonstrate the superiority of the proposed framework with experiments on miniImageNet, tieredImageNet and Omniglot. The results indicate that our method has surpassed previous unsupervised counterparts by a large margin (more than 10%) and obtained performance comparable with the supervised ones.
References in corpus (12)
- Prototypical Networks for Few-shot Learning
- Bootstrap your own latent: A new approach to self-supervised Learning
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- A Simple Neural Attentive Meta-Learner
- Big Self-Supervised Models are Strong Semi-Supervised Learners
- Conceptual Inadequacy of the Shannon Information in Quantum Measurements
- Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?
- Infinite Mixture Prototypes for Few-Shot Learning
- Self-supervised Knowledge Distillation for Few-shot Learning
- Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer
- Few-Shot Learning with Global Class Representations
- Are Fewer Labels Possible for Few-shot Learning?