Improvements to context based self-supervised learning

Abstract

We develop a set of methods to improve on the results of self-supervised learning using context. We start with a baseline of patch based arrangement context learning and go from there. Our methods address some overt problems such as chromatic aberration as well as other potential problems such as spatial skew and mid-level feature neglect. We prevent problems with testing generalization on common self-supervised benchmark tests by using different datasets during our development. The results of our methods combined yield top scores on all standard self-supervised benchmarks, including classification and detection on PASCAL VOC 2007, segmentation on PASCAL VOC 2012, and "linear tests" on the ImageNet and CSAIL Places datasets. We obtain an improvement over our baseline method of between 4.0 to 7.1 percentage points on transfer learning classification tests. We also show results on different standard network architectures to demonstrate generalization as well as portability. All data, models and programs are available at: https://gdo-datasci.llnl.gov/selfsupervised/.

arXiv commentAccepted paper at CVPR 2018

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Deep Residual Learning for Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Unsupervised Visual Representation Learning by Context Prediction
• High-Performance Neural Networks for Visual Object Classification
• Unsupervised Learning by Predicting Noise
• Unsupervised Learning of Visual Representations using Videos
• Data-dependent Initializations of Convolutional Neural Networks
• Learning to See by Moving
• Colorization as a Proxy Task for Visual Understanding

Cited by in corpus (38)

• Models Genesis
• Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
• Self-supervised Domain Adaptation for Computer Vision Tasks
• Self-Supervised Learning for Videos: A Survey
• Self-Supervised Spatiotemporal Feature Learning via Video Rotation Prediction
• A critical analysis of self-supervision, or what we can learn from a single image
• Self-labelling via simultaneous clustering and representation learning
• High-Fidelity Image Generation With Fewer Labels
• Large-scale Unsupervised Semantic Segmentation
• Revisiting Self-Supervised Visual Representation Learning
• Small Sample Learning in Big Data Era
• Contrastive Learning with Adversarial Examples
• Automatic Shortcut Removal for Self-Supervised Representation Learning
• Are we pretraining it right? Digging deeper into visio-linguistic pretraining
• Self-Supervised Video Representation Learning with Space-Time Cubic Puzzles
• Self-Supervised Features Improve Open-World Learning
• Self-Supervised GANs via Auxiliary Rotation Loss
• Image Enhanced Rotation Prediction for Self-Supervised Learning
• Online Deep Clustering for Unsupervised Representation Learning
• Self-supervised Motion Learning from Static Images
• Self-Supervised Learning via Conditional Motion Propagation
• Deep Self-Supervised Representation Learning for Free-Hand Sketch
• Learning Visual Representations for Transfer Learning by Suppressing Texture
• Jigsaw Clustering for Unsupervised Visual Representation Learning
• Aggregative Self-Supervised Feature Learning from a Limited Sample
• GraphCode2Vec: Generic Code Embedding via Lexical and Program Dependence Analyses
• Dissecting Image Crops
• Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learning
• Self-supervised Learning with Fully Convolutional Networks
• Improving Image Clustering through Sample Ranking and Its Application to remote--sensing images
• Exploring Set Similarity for Dense Self-supervised Representation Learning
• Self-supervised Regularization for Text Classification
• TreeGAN: Incorporating Class Hierarchy into Image Generation
• Metric-based Regularization and Temporal Ensemble for Multi-task Learning using Heterogeneous Unsupervised Tasks
• Exploit Clues from Views: Self-Supervised and Regularized Learning for Multiview Object Recognition
• Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision
• When Better Features Mean Greater Risks: The Performance-Privacy Trade-Off in Contrastive Learning
• Self-supervised Learning with Local Attention-Aware Feature