Understand and Improve Contrastive Learning Methods for Visual Representation: A Review
arXiv:2106.03259
Abstract
Traditional supervised learning methods are hitting a bottleneck because of their dependency on expensive manually labeled data and their weaknesses such as limited generalization ability and vulnerability to adversarial attacks. A promising alternative, self-supervised learning, as a type of unsupervised learning, has gained popularity because of its potential to learn effective data representations without manual labeling. Among self-supervised learning algorithms, contrastive learning has achieved state-of-the-art performance in several fields of research. This literature review aims to provide an up-to-date analysis of the efforts of researchers to understand the key components and the limitations of self-supervised learning.
arXiv comment12 pages, 5 figures
References in corpus (24)
- A Simple Framework for Contrastive Learning of Visual Representations
- mixup: Beyond Empirical Risk Minimization
- Bootstrap your own latent: A new approach to self-supervised Learning
- Improved Baselines with Momentum Contrastive Learning
- Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
- Learning Representations by Maximizing Mutual Information Across Views
- What Makes for Good Views for Contrastive Learning?
- Contrastive Multiview Coding
- Manifold Mixup: Better Representations by Interpolating Hidden States
- Big Self-Supervised Models are Strong Semi-Supervised Learners
- Exploring Simple Siamese Representation Learning
- Unsupervised Visual Representation Learning by Context Prediction
- Hard Negative Mixing for Contrastive Learning
- Unsupervised Learning of Visual Representations using Videos
- Supervised Contrastive Learning
- Learning Representations for Automatic Colorization
- Spatiotemporal Contrastive Video Representation Learning
- A Mutual Information Maximization Perspective of Language Representation Learning
- Subject-Aware Contrastive Learning for Biosignals
- Intriguing Properties of Contrastive Losses
- Contrastive Learning with Adversarial Examples
- AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations rather than Data
- Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Prediction
- Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Prediction