Towards Good Practices in Self-supervised Representation Learning
arXiv:2012.00868
Abstract
Self-supervised representation learning has seen remarkable progress in the last few years. More recently, contrastive instance learning has shown impressive results compared to its supervised learning counterparts. However, even with the ever increased interest in contrastive instance learning, it is still largely unclear why these methods work so well. In this paper, we aim to unravel some of the mysteries behind their success, which are the good practices. Through an extensive empirical analysis, we hope to not only provide insights but also lay out a set of best practices that led to the success of recent work in self-supervised representation learning.
References in corpus (7)
- Bootstrap your own latent: A new approach to self-supervised Learning
- Improved Regularization of Convolutional Neural Networks with Cutout
- Improved Baselines with Momentum Contrastive Learning
- Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
- Adversarial Feature Learning
- Contrastive Multiview Coding
- Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases