Exploiting Unlabeled Data in CNNs by Self-supervised Learning to Rank

Abstract

For many applications the collection of labeled data is expensive laborious. Exploitation of unlabeled data during training is thus a long pursued objective of machine learning. Self-supervised learning addresses this by positing an auxiliary task (different, but related to the supervised task) for which data is abundantly available. In this paper, we show how ranking can be used as a proxy task for some regression problems. As another contribution, we propose an efficient backpropagation technique for Siamese networks which prevents the redundant computation introduced by the multi-branch network architecture. We apply our framework to two regression problems: Image Quality Assessment (IQA) and Crowd Counting. For both we show how to automatically generate ranked image sets from unlabeled data. Our results show that networks trained to regress to the ground truth targets for labeled data and to simultaneously learn to rank unlabeled data obtain significantly better, state-of-the-art results for both IQA and crowd counting. In addition, we show that measuring network uncertainty on the self-supervised proxy task is a good measure of informativeness of unlabeled data. This can be used to drive an algorithm for active learning and we show that this reduces labeling effort by up to 50%.

Accepted at TPAMI. (Keywords: Learning from rankings, image quality assessment, crowd counting, active learning). arXiv admin note: text overlap with arXiv:1803.03095

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• Caffe: Convolutional Architecture for Fast Feature Embedding
• A Survey of Recent Advances in CNN-based Single Image Crowd Counting and Density Estimation
• dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairs
• On the Use of Deep Learning for Blind Image Quality Assessment
• CrowdNet: A Deep Convolutional Network for Dense Crowd Counting
• Crowd Counting by Adapting Convolutional Neural Networks with Side Information

Cited by in corpus (35)

• NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization
• Image Quality Assessment using Contrastive Learning
• TransCrowd: weakly-supervised crowd counting with transformers
• Self-supervised Contrastive Representation Learning for Semi-supervised Time-Series Classification
• Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning
• CNN-based Density Estimation and Crowd Counting: A Survey
• Redesigning Multi-Scale Neural Network for Crowd Counting
• CCTrans: Simplifying and Improving Crowd Counting with Transformer
• TreeFormer: a Semi-Supervised Transformer-based Framework for Tree Counting from a Single High Resolution Image
• CrowdFormer: Weakly-supervised Crowd counting with Improved Generalizability
• Locate, Size and Count: Accurately Resolving People in Dense Crowds via Detection
• Consolidated Dataset and Metrics for High-Dynamic-Range Image Quality
• BBA-net: A bi-branch attention network for crowd counting
• Deep Rank-Consistent Pyramid Model for Enhanced Crowd Counting
• Rethinking Global Context in Crowd Counting
• A strong baseline for image and video quality assessment
• Towards Using Count-level Weak Supervision for Crowd Counting
• AutoScale: Learning to Scale for Crowd Counting and Localization
• Set-to-Sequence Methods in Machine Learning: a Review
• Joint Data and Feature Augmentation for Self-Supervised Representation Learning on Point Clouds
• Distribution Estimation to Automate Transformation Policies for Self-Supervision
• Controlling biases and diversity in diverse image-to-image translation
• Multi-Level Attentive Convoluntional Neural Network for Crowd Counting
• Fine-grained Domain Adaptive Crowd Counting via Point-derived Segmentation
• Controllable List-wise Ranking for Universal No-reference Image Quality Assessment
• Reducing Spatial Labeling Redundancy for Semi-supervised Crowd Counting
• Data Transformation Insights in Self-supervision with Clustering Tasks
• Crowd Counting by Self-supervised Transfer Colorization Learning and Global Prior Classification
• Counting People by Estimating People Flows
• IM-META: Influence Maximization Using Node Metadata in Networks With Unknown Topology
• Towards Robust Pattern Recognition: A Review
• Self-Supervised Contextual Bandits in Computer Vision
• Active Crowd Counting with Limited Supervision
• VisDrone-CC2020: The Vision Meets Drone Crowd Counting Challenge Results
• Self-supervised Deep Learning for Reading Activity Classification