SelfMatch: Combining Contrastive Self-Supervision and Consistency for Semi-Supervised Learning

Abstract

This paper introduces SelfMatch, a semi-supervised learning method that combines the power of contrastive self-supervised learning and consistency regularization. SelfMatch consists of two stages: (1) self-supervised pre-training based on contrastive learning and (2) semi-supervised fine-tuning based on augmentation consistency regularization. We empirically demonstrate that SelfMatch achieves the state-of-the-art results on standard benchmark datasets such as CIFAR-10 and SVHN. For example, for CIFAR-10 with 40 labeled examples, SelfMatch achieves 93.19% accuracy that outperforms the strong previous methods such as MixMatch (52.46%), UDA (70.95%), ReMixMatch (80.9%), and FixMatch (86.19%). We note that SelfMatch can close the gap between supervised learning (95.87%) and semi-supervised learning (93.19%) by using only a few labels for each class.

4 pages, NeurIPS 2020 Workshop: Self-Supervised Learning - Theory and Practice

References in corpus (5)

• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Learning deep representations by mutual information estimation and maximization
• Unsupervised Data Augmentation for Consistency Training
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Big Self-Supervised Models are Strong Semi-Supervised Learners

Cited by in corpus (5)

• Graph Convolutional Networks based on Manifold Learning for Semi-Supervised Image Classification
• Active Self-Semi-Supervised Learning for Few Labeled Samples
• Robust Mutual Learning for Semi-supervised Semantic Segmentation
• Shuffle & Divide: Contrastive Learning for Long Text
• ContraCluster: Learning to Classify without Labels by Contrastive Self-Supervision and Prototype-Based Semi-Supervision