Local Aggregation for Unsupervised Learning of Visual Embeddings

Abstract

Unsupervised approaches to learning in neural networks are of substantial interest for furthering artificial intelligence, both because they would enable the training of networks without the need for large numbers of expensive annotations, and because they would be better models of the kind of general-purpose learning deployed by humans. However, unsupervised networks have long lagged behind the performance of their supervised counterparts, especially in the domain of large-scale visual recognition. Recent developments in training deep convolutional embeddings to maximize non-parametric instance separation and clustering objectives have shown promise in closing this gap. Here, we describe a method that trains an embedding function to maximize a metric of local aggregation, causing similar data instances to move together in the embedding space, while allowing dissimilar instances to separate. This aggregation metric is dynamic, allowing soft clusters of different scales to emerge. We evaluate our procedure on several large-scale visual recognition datasets, achieving state-of-the-art unsupervised transfer learning performance on object recognition in ImageNet, scene recognition in Places 205, and object detection in PASCAL VOC.

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Recent Trends in Deep Learning Based Natural Language Processing
• Adversarial Feature Learning
• Data-dependent Initializations of Convolutional Neural Networks
• Building high-level features using large scale unsupervised learning
• Texture Synthesis with Spatial Generative Adversarial Networks

Cited by in corpus (28)

• Data-Efficient Image Recognition with Contrastive Predictive Coding
• What Makes for Good Views for Contrastive Learning?
• Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
• Contrastive Multiview Coding
• CLEAR: Contrastive Learning for Sentence Representation
• Memory-augmented Dense Predictive Coding for Video Representation Learning
• Are we done with ImageNet?
• Deep Robust Clustering by Contrastive Learning
• Whitening for Self-Supervised Representation Learning
• Parameter Efficient Multimodal Transformers for Video Representation Learning
• The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models
• Understanding the Behaviour of Contrastive Loss
• Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Prediction
• Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination
• AdCo: Adversarial Contrast for Efficient Learning of Unsupervised Representations from Self-Trained Negative Adversaries
• Unsupervised Learning from Video with Deep Neural Embeddings
• Exploring Instance Relations for Unsupervised Feature Embedding
• Distilling Localization for Self-Supervised Representation Learning
• How Useful is Self-Supervised Pretraining for Visual Tasks?
• CO2: Consistent Contrast for Unsupervised Visual Representation Learning
• Temperature as Uncertainty in Contrastive Learning
• Multi-label Contrastive Predictive Coding
• Semantic-Aware Generation for Self-Supervised Visual Representation Learning
• Spatial Ensemble: a Novel Model Smoothing Mechanism for Student-Teacher Framework
• EqCo: Equivalent Rules for Self-supervised Contrastive Learning
• Self-Supervised Learning of a Biologically-Inspired Visual Texture Model
• A Unified Mixture-View Framework for Unsupervised Representation Learning
• Temporal Knowledge Consistency for Unsupervised Visual Representation Learning