Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Prediction

Abstract

State-of-the-art methods for self-supervised learning (SSL) build representations by maximizing the similarity between different transformed "views" of a sample. Without sufficient diversity in the transformations used to create views, however, it can be difficult to overcome nuisance variables in the data and build rich representations. This motivates the use of the dataset itself to find similar, yet distinct, samples to serve as views for one another. In this paper, we introduce Mine Your Own vieW (MYOW), a new approach for self-supervised learning that looks within the dataset to define diverse targets for prediction. The idea behind our approach is to actively mine views, finding samples that are neighbors in the representation space of the network, and then predict, from one sample's latent representation, the representation of a nearby sample. After showing the promise of MYOW on benchmarks used in computer vision, we highlight the power of this idea in a novel application in neuroscience where SSL has yet to be applied. When tested on multi-unit neural recordings, we find that MYOW outperforms other self-supervised approaches in all examples (in some cases by more than 10%), and often surpasses the supervised baseline. With MYOW, we show that it is possible to harness the diversity of the data to build rich views and leverage self-supervision in new domains where augmentations are limited or unknown.

References in corpus (31)

• A Simple Framework for Contrastive Learning of Visual Representations
• mixup: Beyond Empirical Risk Minimization
• Bootstrap your own latent: A new approach to self-supervised Learning
• The Effectiveness of Data Augmentation in Image Classification using Deep Learning
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Contrastive Representation Learning: A Framework and Review
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• What Makes for Good Views for Contrastive Learning?
• Contrastive Multiview Coding
• Exploring Simple Siamese Representation Learning
• Deep Graph Contrastive Representation Learning
• Hard Negative Mixing for Contrastive Learning
• Self-supervised Learning for ECG-based Emotion Recognition
• Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases
• Dropout as data augmentation
• Local Aggregation for Unsupervised Learning of Visual Embeddings
• Subject-Aware Contrastive Learning for Biosignals
• Data augmentation using learned transformations for one-shot medical image segmentation
• Learning by Association - A versatile semi-supervised training method for neural networks
• Time-Contrastive Networks: Self-Supervised Learning from Video
• Data-Efficient Reinforcement Learning with Self-Predictive Representations
• Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE
• Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning
• Whitening for Self-Supervised Representation Learning
• Contrastive Learning with Stronger Augmentations
• Understanding Self-supervised Learning with Dual Deep Networks
• BYOL works even without batch statistics
• Learning Actionable Representations from Visual Observations
• Joint Contrastive Learning with Infinite Possibilities
• Multi-label Contrastive Predictive Coding
• Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning

Cited by in corpus (7)

• Drop, Swap, and Generate: A Self-Supervised Approach for Generating Neural Activity
• Understand and Improve Contrastive Learning Methods for Visual Representation: A Review
• Unsupervised Object-Level Representation Learning from Scene Images
• Cross-Trajectory Representation Learning for Zero-Shot Generalization in RL
• On Feature Decorrelation in Self-Supervised Learning
• HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation
• MIC: Model-agnostic Integrated Cross-channel Recommenders