Improved Baselines with Momentum Contrastive Learning

Abstract

Contrastive unsupervised learning has recently shown encouraging progress, e.g., in Momentum Contrast (MoCo) and SimCLR. In this note, we verify the effectiveness of two of SimCLR's design improvements by implementing them in the MoCo framework. With simple modifications to MoCo---namely, using an MLP projection head and more data augmentation---we establish stronger baselines that outperform SimCLR and do not require large training batches. We hope this will make state-of-the-art unsupervised learning research more accessible. Code will be made public.

Tech report, 2 pages + references

References in corpus (8)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Learning deep representations by mutual information estimation and maximization
• Momentum Contrast for Unsupervised Visual Representation Learning
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Learning Representations by Maximizing Mutual Information Across Views
• Contrastive Multiview Coding

Cited by in corpus (428)

• Learning Transferable Visual Models From Natural Language Supervision
• A Survey on Visual Transformer
• Transformers in Vision: A Survey
• Bootstrap your own latent: A new approach to self-supervised Learning
• On the Opportunities and Risks of Foundation Models
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Recent advances and clinical applications of deep learning in medical image analysis
• Momentum Contrast for Unsupervised Visual Representation Learning
• BEiT: BERT Pre-Training of Image Transformers
• SpectralGPT: Spectral Remote Sensing Foundation Model
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• What Makes for Good Views for Contrastive Learning?
• ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision
• Prototypical Contrastive Learning of Unsupervised Representations
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Exploring Simple Siamese Representation Learning
• Self-Supervised Speech Representation Learning: A Review
• Contrastive Learning of Medical Visual Representations from Paired Images and Text
• CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
• Self-supervised Co-training for Video Representation Learning
• Hard Negative Mixing for Contrastive Learning
• Self-supervised learning methods and applications in medical imaging analysis: A survey
• Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
• An Empirical Study of Remote Sensing Pretraining
• A survey on Semi-, Self- and Unsupervised Learning for Image Classification
• Self-supervised representation learning from 12-lead ECG data
• Momentum Contrastive Learning for Few-Shot COVID-19 Diagnosis from Chest CT Images
• Global and Local Contrastive Self-Supervised Learning for Semantic Segmentation of HR Remote Sensing Images
• Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
• Unsupervised Point Cloud Representation Learning with Deep Neural Networks: A Survey
• Twin Contrastive Learning for Online Clustering
• AutoNovel: Automatically Discovering and Learning Novel Visual Categories
• DeepSMILE: Contrastive self-supervised pre-training benefits MSI and HRD classification directly from H&E whole-slide images in colorectal and breast cancer
• One Million Scenes for Autonomous Driving: ONCE Dataset
• Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases
• Learning Representation for Clustering via Prototype Scattering and Positive Sampling
• Contrastive Code Representation Learning
• Disentangling Long and Short-Term Interests for Recommendation
• Semantics-Guided Contrastive Network for Zero-Shot Object detection
• A Comprehensive Study of Deep Video Action Recognition
• Self-Supervised Learning with Swin Transformers
• RCPS: Rectified Contrastive Pseudo Supervision for Semi-Supervised Medical Image Segmentation
• CMID: A Unified Self-Supervised Learning Framework for Remote Sensing Image Understanding
• Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works
• Self-Supervised Multisensor Change Detection
• What Should Not Be Contrastive in Contrastive Learning
• Vector-quantized Image Modeling with Improved VQGAN
• Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data
• Large-scale Unsupervised Semantic Segmentation
• Efficient Self-supervised Vision Transformers for Representation Learning
• What makes instance discrimination good for transfer learning?
• Self-Supervised Representation Learning for Astronomical Images
• Towards Label-efficient Automatic Diagnosis and Analysis: A Comprehensive Survey of Advanced Deep Learning-based Weakly-supervised, Semi-supervised and Self-supervised Techniques in Histopathological Image Analysis
• SEED: Self-supervised Distillation For Visual Representation
• Are we done with ImageNet?
• A Framework For Contrastive Self-Supervised Learning And Designing A New Approach
• Self-EMD: Self-Supervised Object Detection without ImageNet
• Self-supervised learning through the eyes of a child
• Partial success in closing the gap between human and machine vision
• A Systematic Survey of Regularization and Normalization in GANs
• Video Representation Learning with Visual Tempo Consistency
• A Review of Predictive and Contrastive Self-supervised Learning for Medical Images
• Aligning Pretraining for Detection via Object-Level Contrastive Learning
• i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning
• MixCo: Mix-up Contrastive Learning for Visual Representation
• Semantic-aware Dense Representation Learning for Remote Sensing Image Change Detection
• MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models
• On Mutual Information in Contrastive Learning for Visual Representations
• Self-Supervision Closes the Gap Between Weak and Strong Supervision in Histology
• Self-Supervised RF Signal Representation Learning for NextG Signal Classification with Deep Learning
• Self-Supervised Pre-Training for Transformer-Based Person Re-Identification
• CCGL: Contrastive Cascade Graph Learning
• Image Classification with Small Datasets: Overview and Benchmark
• Whitening for Self-Supervised Representation Learning
• SpectralEarth: Training Hyperspectral Foundation Models at Scale
• Solo-learn: A Library of Self-supervised Methods for Visual Representation Learning
• Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
• Learning Invariance from Generated Variance for Unsupervised Person Re-identification
• Distilling Knowledge by Mimicking Features
• ReSSL: Relational Self-Supervised Learning with Weak Augmentation
• Dense Contrastive Learning for Self-Supervised Visual Pre-Training
• Representation Learning via Invariant Causal Mechanisms
• LogME: Practical Assessment of Pre-trained Models for Transfer Learning
• Open-Set Recognition: a Good Closed-Set Classifier is All You Need?
• DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting
• Multi-Label Contrastive Learning for Abstract Visual Reasoning
• Contrastive Learning with Hard Negative Samples
• Watching the World Go By: Representation Learning from Unlabeled Videos
• Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation
• Unsupervised Outlier Detection using Memory and Contrastive Learning
• Contrastive Learning with Stronger Augmentations
• AP-10K: A Benchmark for Animal Pose Estimation in the Wild
• Understanding self-supervised Learning Dynamics without Contrastive Pairs
• Learning Contrastive Self-Distillation for Ultra-Fine-Grained Visual Categorization Targeting Limited Samples
• Understanding Self-supervised Learning with Dual Deep Networks
• MoPro: Webly Supervised Learning with Momentum Prototypes
• Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations
• P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understanding
• MaxMatch: Semi-Supervised Learning with Worst-Case Consistency
• Cross-Modal Contrastive Learning for Text-to-Image Generation
• VIMPAC: Video Pre-Training via Masked Token Prediction and Contrastive Learning
• Semi-TCL: Semi-Supervised Track Contrastive Representation Learning
• Bridging the Gap Between Patient-specific and Patient-independent Seizure Prediction via Knowledge Distillation
• SODA10M: A Large-Scale 2D Self/Semi-Supervised Object Detection Dataset for Autonomous Driving
• Pixel-Level Clustering Network for Unsupervised Image Segmentation
• UP-DETR: Unsupervised Pre-training for Object Detection with Transformers
• Do Different Tracking Tasks Require Different Appearance Models?
• Context-self contrastive pretraining for crop type semantic segmentation
• PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers
• The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models
• Data-Efficient Instance Generation from Instance Discrimination
• See through Gradients: Image Batch Recovery via GradInversion
• Inherit with Distillation and Evolve with Contrast: Exploring Class Incremental Semantic Segmentation Without Exemplar Memory
• Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning
• Semi-Supervised Contrastive Learning with Generalized Contrastive Loss and Its Application to Speaker Recognition
• SelfFed: Self-Supervised Federated Learning for Data Heterogeneity and Label Scarcity in Medical Images
• FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding
• MST: Masked Self-Supervised Transformer for Visual Representation
• Improving COVID-19 CT Classification of CNNs by Learning Parameter-Efficient Representation
• MedAug: Contrastive learning leveraging patient metadata improves representations for chest X-ray interpretation
• Parametric Instance Classification for Unsupervised Visual Feature Learning
• Conditional Negative Sampling for Contrastive Learning of Visual Representations
• Incremental False Negative Detection for Contrastive Learning
• Uncertainty-Aware Pseudo-Label Filtering for Source-Free Unsupervised Domain Adaptation
• Understanding the Behaviour of Contrastive Loss
• Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning
• Evaluating Contrastive Models for Instance-based Image Retrieval
• Improving Text-to-Image Synthesis Using Contrastive Learning
• Unsupervised Degradation Representation Learning for Blind Super-Resolution
• Self-Supervised Training Enhances Online Continual Learning
• Weakly Contrastive Learning via Batch Instance Discrimination and Feature Clustering for Small Sample SAR ATR
• A Systematic Benchmarking Analysis of Transfer Learning for Medical Image Analysis
• Self-Supervised Learning with Kernel Dependence Maximization
• Viewmaker Networks: Learning Views for Unsupervised Representation Learning
• Hybrid Discriminative-Generative Training via Contrastive Learning
• Sample-efficient Reinforcement Learning Representation Learning with Curiosity Contrastive Forward Dynamics Model
• Towards the Generalization of Contrastive Self-Supervised Learning
• ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation
• GraSS: Contrastive Learning with Gradient Guided Sampling Strategy for Remote Sensing Image Semantic Segmentation
• Unsupervised Representation Learning by InvariancePropagation
• EDMAE: An Efficient Decoupled Masked Autoencoder for Standard View Identification in Pediatric Echocardiography
• BYOL works even without batch statistics
• Semi-Supervised Visual Representation Learning for Fashion Compatibility
• A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning
• 3D Human Action Representation Learning via Cross-View Consistency Pursuit
• On the surprising similarities between supervised and self-supervised models
• LoCo: Local Contrastive Representation Learning
• Poisoning and Backdooring Contrastive Learning
• Contrastive Syn-to-Real Generalization
• Contrastive Attention for Automatic Chest X-ray Report Generation
• BenthicNet: A global compilation of seafloor images for deep learning applications
• Contrastive Visual-Linguistic Pretraining
• Data Compression and Inference in Cosmology with Self-Supervised Machine Learning
• Augmented Skeleton Based Contrastive Action Learning with Momentum LSTM for Unsupervised Action Recognition
• Multi-queue Momentum Contrast for Microvideo-Product Retrieval
• Multimodal Contrastive Training for Visual Representation Learning
• A Self Supervised StyleGAN for Image Annotation and Classification with Extremely Limited Labels
• Debiased Contrastive Learning
• Self-supervised Cross-silo Federated Neural Architecture Search
• Prototypical Partial Optimal Transport for Universal Domain Adaptation
• Self-supervised Pre-training with Hard Examples Improves Visual Representations
• Object-aware Contrastive Learning for Debiased Scene Representation
• Clinically Labeled Contrastive Learning for OCT Biomarker Classification
• Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotation
• Universum-inspired Supervised Contrastive Learning
• Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptation
• Multi-modal Self-supervised Pre-training for Regulatory Genome Across Cell Types
• Boosting Discriminative Visual Representation Learning with Scenario-Agnostic Mixup
• Few-Shot Segmentation with Global and Local Contrastive Learning
• Cross-domain Contrastive Learning for Unsupervised Domain Adaptation
• Exploring Active Learning for Semiconductor Defect Segmentation
• Self-Damaging Contrastive Learning
• Energy Models for Better Pseudo-Labels: Improving Semi-Supervised Classification with the 1-Laplacian Graph Energy
• Dynamic Bottleneck for Robust Self-Supervised Exploration
• Look at What I'm Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videos
• Self-supervised Image Clustering from Multiple Incomplete Views via Constrastive Complementary Generation
• Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination
• RadBERT-CL: Factually-Aware Contrastive Learning For Radiology Report Classification
• ICE: Inter-instance Contrastive Encoding for Unsupervised Person Re-identification
• Continual Contrastive Learning for Image Classification
• AdCo: Adversarial Contrast for Efficient Learning of Unsupervised Representations from Self-Trained Negative Adversaries
• Self-supervised Neural Architecture Search
• Joint Contrastive Learning with Infinite Possibilities
• Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs
• Distilling Localization for Self-Supervised Representation Learning
• Terrain-Informed Self-Supervised Learning: Enhancing Building Footprint Extraction from LiDAR Data with Limited Annotations
• Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast
• Learning the Relation between Similarity Loss and Clustering Loss in Self-Supervised Learning
• Momentum^2 Teacher: Momentum Teacher with Momentum Statistics for Self-Supervised Learning
• Self-Supervised Features Improve Open-World Learning
• Category Contrast for Unsupervised Domain Adaptation in Visual Tasks
• ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training
• Deep Clustering by Semantic Contrastive Learning
• Joint Generative and Contrastive Learning for Unsupervised Person Re-identification
• Temperature as Uncertainty in Contrastive Learning
• Spatially Consistent Representation Learning
• Towards Good Practices in Self-supervised Representation Learning
• Self-Paced Contrastive Learning for Semi-supervised Medical Image Segmentation with Meta-labels
• Rethinking Self-Supervised Learning: Small is Beautiful
• CO2: Consistent Contrast for Unsupervised Visual Representation Learning
• INTERN: A New Learning Paradigm Towards General Vision
• Run Away From your Teacher: Understanding BYOL by a Novel Self-Supervised Approach
• Multi-Dataset Benchmarks for Masked Identification using Contrastive Representation Learning
• Understand and Improve Contrastive Learning Methods for Visual Representation: A Review
• Multi-label Contrastive Predictive Coding
• Joint Self-Supervised and Supervised Contrastive Learning for Multimodal MRI Data: Towards Predicting Abnormal Neurodevelopment
• RegionCL: Can Simple Region Swapping Contribute to Contrastive Learning?
• Gradient Regularized Contrastive Learning for Continual Domain Adaptation
• Geometric Understanding of Discriminability and Transferability for Visual Domain Adaptation
• Improving Transferability of Representations via Augmentation-Aware Self-Supervision
• A vector quantized masked autoencoder for audiovisual speech emotion recognition
• Rethinking Supervised Pre-training for Better Downstream Transferring
• SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learning
• Improving Transformation Invariance in Contrastive Representation Learning
• Self-supervised Mean Teacher for Semi-supervised Chest X-ray Classification
• PointSmile: Point Self-supervised Learning via Curriculum Mutual Information
• DimCL: Dimensional Contrastive Learning For Improving Self-Supervised Learning
• Learning Bidirectional Translation between Descriptions and Actions with Small Paired Data
• HyperInverter: Improving StyleGAN Inversion via Hypernetwork
• Training GANs with Stronger Augmentations via Contrastive Discriminator
• Instance Localization for Self-supervised Detection Pretraining
• Pretext-Contrastive Learning: Toward Good Practices in Self-supervised Video Representation Leaning
• Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning
• Supporting Clustering with Contrastive Learning
• MixSiam: A Mixture-based Approach to Self-supervised Representation Learning
• DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning
• A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classification
• Understanding Hyperbolic Metric Learning through Hard Negative Sampling
• OpenCoS: Contrastive Semi-supervised Learning for Handling Open-set Unlabeled Data
• Semantic-Aware Generation for Self-Supervised Visual Representation Learning
• A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation
• Self-Supervision, Remote Sensing and Abstraction: Representation Learning Across 3 Million Locations
• Does a Face Mask Protect my Privacy?: Deep Learning to Predict Protected Attributes from Masked Face Images
• Self-supervised Semi-supervised Learning for Data Labeling and Quality Evaluation
• Augmentation-aware Self-supervised Learning with Conditioned Projector
• Unsupervised Object-Level Representation Learning from Scene Images
• For self-supervised learning, Rationality implies generalization, provably
• Self supervised learning improves dMMR/MSI detection from histology slides across multiple cancers
• Compound Batch Normalization for Long-tailed Image Classification
• Self-Supervised Learning by Estimating Twin Class Distributions
• Self-Supervised Multi-View Learning via Auto-Encoding 3D Transformations
• Representation Learning via Consistent Assignment of Views to Clusters
• Estimating Galactic Distances From Images Using Self-supervised Representation Learning
• A Multi-Stage Attentive Transfer Learning Framework for Improving COVID-19 Diagnosis
• Positional Contrastive Learning for Volumetric Medical Image Segmentation
• Simple Contrastive Representation Adversarial Learning for NLP Tasks
• What Is Considered Complete for Visual Recognition?
• Do We Really Need to Learn Representations from In-domain Data for Outlier Detection?
• Auxiliary Tasks Speed Up Learning PointGoal Navigation
• Complex QA and language models hybrid architectures, Survey
• INoD: Injected Noise Discriminator for Self-Supervised Representation Learning in Agricultural Fields
• ExpPoint-MAE: Better interpretability and performance for self-supervised point cloud transformers
• TransformCode: A Contrastive Learning Framework for Code Embedding via Subtree Transformation
• Towards Deviation-Robust Agent Navigation via Perturbation-Aware Contrastive Learning
• Less is More: Selective Reduction of CT Data for Self-Supervised Pre-Training of Deep Learning Models with Contrastive Learning Improves Downstream Classification Performance
• Robust and Explainable Framework to Address Data Scarcity in Diagnostic Imaging
• Evaluating the Label Efficiency of Contrastive Self-Supervised Learning for Multi-Resolution Satellite Imagery
• Hybrid Generative-Contrastive Representation Learning
• Winning the CVPR'2021 Kinetics-GEBD Challenge: Contrastive Learning Approach
• A Theory-Driven Self-Labeling Refinement Method for Contrastive Representation Learning
• Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection
• Unsupervised Pre-training for Person Re-identification
• Unsupervised domain adaptation via coarse-to-fine feature alignment method using contrastive learning
• Embedding Transfer with Label Relaxation for Improved Metric Learning
• Self-supervised Video Representation Learning by Context and Motion Decoupling
• All you need are a few pixels: semantic segmentation with PixelPick
• Distilling Audio-Visual Knowledge by Compositional Contrastive Learning
• Heterogeneous Contrastive Learning: Encoding Spatial Information for Compact Visual Representations
• On Feature Decorrelation in Self-Supervised Learning
• Prototypical Cross-domain Knowledge Transfer for Cervical Dysplasia Visual Inspection
• A Simple Framework for Uncertainty in Contrastive Learning
• Unsupervised Visual Attention and Invariance for Reinforcement Learning
• Center-wise Local Image Mixture For Contrastive Representation Learning
• The Power of Contrast for Feature Learning: A Theoretical Analysis
• 8-bit Optimizers via Block-wise Quantization
• Cross-Domain Sentiment Classification with In-Domain Contrastive Learning
• Improving Contrastive Learning by Visualizing Feature Transformation
• MSVQ: Self-Supervised Learning with Multiple Sample Views and Queues
• Revisiting the Transferability of Supervised Pretraining: an MLP Perspective
• Semantically-Conditioned Negative Samples for Efficient Contrastive Learning
• Dense Semantic Contrast for Self-Supervised Visual Representation Learning
• Self-supervised Visual Attribute Learning for Fashion Compatibility
• Do Explicit Alignments Robustly Improve Multilingual Encoders?
• An Empirical Study and Analysis on Open-Set Semi-Supervised Learning
• Contrastive Unsupervised Learning for Audio Fingerprinting
• EqCo: Equivalent Rules for Self-supervised Contrastive Learning
• Imbalance-Aware Self-Supervised Learning for 3D Radiomic Representations
• S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration
• Improved Meta-Learning Training for Speaker Verification
• Multi-label Iterated Learning for Image Classification with Label Ambiguity
• Jigsaw Clustering for Unsupervised Visual Representation Learning
• Improving Few-Shot Learning with Auxiliary Self-Supervised Pretext Tasks
• Deep Probability Estimation
• Unleashing the Potential of Unsupervised Pre-Training with Intra-Identity Regularization for Person Re-Identification
• Trapped in texture bias? A large scale comparison of deep instance segmentation
• Contrastive Attraction and Contrastive Repulsion for Representation Learning
• Geographical Knowledge-driven Representation Learning for Remote Sensing Images
• Hierarchical Multi-Label Classification with Missing Information for Benthic Habitat Imagery
• Consistency Regularization for Deep Face Anti-Spoofing
• Pri3D: Can 3D Priors Help 2D Representation Learning?
• The Animal ID Problem: Continual Curation
• Towards Fine-grained Visual Representations by Combining Contrastive Learning with Image Reconstruction and Attention-weighted Pooling
• Towards Demystifying Representation Learning with Non-contrastive Self-supervision
• Motion-aware Contrastive Video Representation Learning via Foreground-background Merging
• Polyphone Disambiguation in Mandarin Chinese with Semi-Supervised Learning
• Are Fewer Labels Possible for Few-shot Learning?
• Online Exemplar Fine-Tuning for Image-to-Image Translation
• Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning
• A Unified Mixture-View Framework for Unsupervised Representation Learning
• USCL: Pretraining Deep Ultrasound Image Diagnosis Model through Video Contrastive Representation Learning
• CLOUD: Contrastive Learning of Unsupervised Dynamics
• Contrast-reconstruction Representation Learning for Self-supervised Skeleton-based Action Recognition
• Shot in the Dark: Few-Shot Learning with No Base-Class Labels
• Self-Supervised Pretraining on Satellite Imagery: a Case Study on Label-Efficient Vehicle Detection
• Integrating Categorical Semantics into Unsupervised Domain Translation
• Don't miss the Mismatch: Investigating the Objective Function Mismatch for Unsupervised Representation Learning
• Beneficial Perturbations Network for Defending Adversarial Examples
• Multi-source Few-shot Domain Adaptation
• A Fast Knowledge Distillation Framework for Visual Recognition
• Leveraging Self-Supervised Vision Transformers for Segmentation-based Transfer Function Design
• One-bit Supervision for Image Classification: Problem, Solution, and Beyond
• PoseContrast: Class-Agnostic Object Viewpoint Estimation in the Wild with Pose-Aware Contrastive Learning
• Learning Temporally-Consistent Representations for Data-Efficient Reinforcement Learning
• Unsupervised Image Classification for Deep Representation Learning
• Enhancing Sound Source Localization via False Negative Elimination
• Meta Clustering Learning for Large-scale Unsupervised Person Re-identification
• Contrastive Learning of Global-Local Video Representations
• Semi-supervised Semantic Segmentation with Directional Context-aware Consistency
• Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift
• Compressive Visual Representations
• Staying in Shape: Learning Invariant Shape Representations using Contrastive Learning
• Self-Supervised Learning from Unlabeled Fundus Photographs Improves Segmentation of the Retina
• Long-Short Temporal Contrastive Learning of Video Transformers
• IDE-Net: Interactive Driving Event and Pattern Extraction from Human Data
• Practical Assessment of Generalization Performance Robustness for Deep Networks via Contrastive Examples
• Sensor Data Augmentation by Resampling for Contrastive Learning in Human Activity Recognition
• A Multi-level Alignment Training Scheme for Video-and-Language Grounding
• Addressing Feature Suppression in Unsupervised Visual Representations
• Explicitly Modeling the Discriminability for Instance-Aware Visual Object Tracking
• Progressive Stage-wise Learning for Unsupervised Feature Representation Enhancement
• Temporal Knowledge Consistency for Unsupervised Visual Representation Learning
• Video Contrastive Learning with Global Context
• Few Shot Learning With No Labels
• Multi-level Knowledge Distillation via Knowledge Alignment and Correlation
• Exploiting Spatial-Temporal Semantic Consistency for Video Scene Parsing
• D2C: Diffusion-Denoising Models for Few-shot Conditional Generation
• K-Shot Contrastive Learning of Visual Features with Multiple Instance Augmentations
• You Never Cluster Alone
• Unsupervised Visual Representation Learning by Online Constrained K-Means
• Exploring Set Similarity for Dense Self-supervised Representation Learning
• UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning
• Dissecting Supervised Contrastive Learning
• Train a One-Million-Way Instance Classifier for Unsupervised Visual Representation Learning
• What Can You Learn from Your Muscles? Learning Visual Representation from Human Interactions
• Unsupervised Object Detection with LiDAR Clues
• MP2: A Momentum Contrast Approach for Recommendation with Pointwise and Pairwise Learning
• CoDiM: Learning with Noisy Labels via Contrastive Semi-Supervised Learning
• Self-supervised Learning for Semi-supervised Temporal Language Grounding
• On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy
• SCALP -- Supervised Contrastive Learning for Cardiopulmonary Disease Classification and Localization in Chest X-rays using Patient Metadata
• Parametric Contrastive Learning
• Dynamic Convolution for 3D Point Cloud Instance Segmentation
• Personalizing Pre-trained Models
• Unsupervised Embedding Learning from Uncertainty Momentum Modeling
• Low Budget Active Learning via Wasserstein Distance: An Integer Programming Approach
• THAT: Two Head Adversarial Training for Improving Robustness at Scale
• On the robustness of self-supervised representations for multi-view object classification
• Multi-dataset Pretraining: A Unified Model for Semantic Segmentation
• OBoW: Online Bag-of-Visual-Words Generation for Self-Supervised Learning
• Towards noise robust trigger-word detection with contrastive learning pre-task for fast on-boarding of new trigger-words
• MIC: Model-agnostic Integrated Cross-channel Recommenders
• Semi-supervised Learning for Dense Object Detection in Retail Scenes
• Improve Unsupervised Pretraining for Few-label Transfer
• Online Unsupervised Learning of Visual Representations and Categories
• Attribute Guided Sparse Tensor-Based Model for Person Re-Identification
• Deep Bregman Divergence for Contrastive Learning of Visual Representations
• Unsupervised Local Discrimination for Medical Images
• Knowledge accumulating: The general pattern of learning
• Segmentation of VHR EO Images using Unsupervised Learning
• Self-Supervised Pretraining and Controlled Augmentation Improve Rare Wildlife Recognition in UAV Images
• Self-Supervised Visual Representations Learning by Contrastive Mask Prediction
• Contrastive Representations for Label Noise Require Fine-Tuning
• TransAug: Translate as Augmentation for Sentence Embeddings
• Aligning Cross-lingual Sentence Representations with Dual Momentum Contrast
• On-target Adaptation
• Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering
• Self Supervision to Distillation for Long-Tailed Visual Recognition
• MIO : Mutual Information Optimization using Self-Supervised Binary Contrastive Learning
• Mitigating Sampling Bias and Improving Robustness in Active Learning
• Seeking an Optimal Approach for Computer-Aided Pulmonary Embolism Detection
• Homography augumented momentum constrastive learning for SAR image retrieval
• A Survey on Green Deep Learning
• Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency
• CrossCLR: Cross-modal Contrastive Learning For Multi-modal Video Representations
• HoughCL: Finding Better Positive Pairs in Dense Self-supervised Learning
• Self-Supervised Visual Representation Learning Using Lightweight Architectures
• Understanding the Role of Self-Supervised Learning in Out-of-Distribution Detection Task
• Semi-Supervised Learning with Taxonomic Labels
• Distilling Visual Priors from Self-Supervised Learning
• Active Learning at the ImageNet Scale
• Improved Detection of Face Presentation Attacks Using Image Decomposition
• Axiomatic Explanations for Visual Search, Retrieval, and Similarity Learning
• Adaptive Hierarchical Similarity Metric Learning with Noisy Labels
• Recent Advancements in Self-Supervised Paradigms for Visual Feature Representation
• Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations
• UBoCo : Unsupervised Boundary Contrastive Learning for Generic Event Boundary Detection
• Self-supervised similarity search for large scientific datasets
• Dual Prototypical Contrastive Learning for Few-shot Semantic Segmentation
• SLADE: A Self-Training Framework For Distance Metric Learning
• PreViTS: Contrastive Pretraining with Video Tracking Supervision
• Fighting Fire with Fire: Contrastive Debiasing without Bias-free Data via Generative Bias-transformation
• PCPs: Patient Cardiac Prototypes
• Contrast R-CNN for Continual Learning in Object Detection
• FROST: Faster and more Robust One-shot Semi-supervised Training
• Super-Selfish: Self-Supervised Learning on Images with PyTorch
• Atlas Based Representation and Metric Learning on Manifolds
• CleftNet: Augmented Deep Learning for Synaptic Cleft Detection from Brain Electron Microscopy
• Using contrastive learning to improve the performance of steganalysis schemes
• Training Efficiency and Robustness in Deep Learning
• Self-Supervised Learning for Large-Scale Unsupervised Image Clustering
• Langevin Cooling for Domain Translation
• Contrastive Forward-Forward: A Training Algorithm of Vision Transformer
• Label-free Monitoring of Self-Supervised Learning Progress
• Joint Learning of Neural Transfer and Architecture Adaptation for Image Recognition
• On Equivariant and Invariant Learning of Object Landmark Representations
• GenURL: A General Framework for Unsupervised Representation Learning
• Contrastive Active Inference
• Pedestrian Detection by Exemplar-Guided Contrastive Learning