Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning

Abstract

Effective convolutional neural networks are trained on large sets of labeled data. However, creating large labeled datasets is a very costly and time-consuming task. Semi-supervised learning uses unlabeled data to train a model with higher accuracy when there is a limited set of labeled data available. In this paper, we consider the problem of semi-supervised learning with convolutional neural networks. Techniques such as randomized data augmentation, dropout and random max-pooling provide better generalization and stability for classifiers that are trained using gradient descent. Multiple passes of an individual sample through the network might lead to different predictions due to the non-deterministic behavior of these techniques. We propose an unsupervised loss function that takes advantage of the stochastic nature of these methods and minimizes the difference between the predictions of multiple passes of a training sample through the network. We evaluate the proposed method on several benchmark datasets.

9 pages, 2 figures, 5 tables

References in corpus (5)

• Improving neural networks by preventing co-adaptation of feature detectors
• Semi-Supervised Learning with Ladder Networks
• Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition
• Spatially-sparse convolutional neural networks
• Batch-normalized Maxout Network in Network

Cited by in corpus (50)

• Training Generative Adversarial Networks with Limited Data
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• AutoAugment: Learning Augmentation Policies from Data
• Learning Discrete Representations via Information Maximizing Self-Augmented Training
• Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation
• Semi-Supervised Medical Image Segmentation via Learning Consistency under Transformations
• SPICE: Semantic Pseudo-labeling for Image Clustering
• Adversarially Robust Generalization Just Requires More Unlabeled Data
• Compete to Win: Enhancing Pseudo Labels for Barely-supervised Medical Image Segmentation
• Generative Artificial Intelligence Meets Synthetic Aperture Radar: A Survey
• Data Distillation: Towards Omni-Supervised Learning
• Unsupervised and semi-supervised learning with Categorical Generative Adversarial Networks assisted by Wasserstein distance for dermoscopy image Classification
• A Shape Transformation-based Dataset Augmentation Framework for Pedestrian Detection
• Deep Co-Training for Semi-Supervised Image Recognition
• Smooth Neighbors on Teacher Graphs for Semi-supervised Learning
• Learning Loss Functions for Semi-supervised Learning via Discriminative Adversarial Networks
• Vicinal and categorical domain adaptation
• Entanglement Verification with Deep Semi-supervised Machine Learning
• Cross-Referencing Self-Training Network for Sound Event Detection in Audio Mixtures
• Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation
• SSFL: Tackling Label Deficiency in Federated Learning via Personalized Self-Supervision
• Exploring Self-Supervised Regularization for Supervised and Semi-Supervised Learning
• Semi-Supervised Learning using Siamese Networks
• Robust Semi-Supervised Learning for Self-learning Open-World Classes
• Semi-Supervised Medical Image Segmentation with Co-Distribution Alignment
• SST: Self-training with Self-adaptive Thresholding for Semi-supervised Learning
• Advances in Medical Image Segmentation: A Comprehensive Survey with a Focus on Lumbar Spine Applications
• Supervision by Denoising for Medical Image Segmentation
• OpenCoS: Contrastive Semi-supervised Learning for Handling Open-set Unlabeled Data
• Leveraging Unlabelled Data in Multiple-Instance Learning Problems for Improved Detection of Parkinsonian Tremor in Free-Living Conditions
• Deep Low-Density Separation for Semi-Supervised Classification
• An analysis of over-sampling labeled data in semi-supervised learning with FixMatch
• Towards Few-Annotation Learning for Object Detection: Are Transformer-based Models More Efficient ?
• GuidedMix-Net: Learning to Improve Pseudo Masks Using Labeled Images as Reference
• Enhancing Dual Network Based Semi-Supervised Medical Image Segmentation with Uncertainty-Guided Pseudo-Labeling
• DRo: A data-scarce mechanism to revolutionize the performance of Deep Learning based Security Systems
• Learning Invariant Representation with Consistency and Diversity for Semi-supervised Source Hypothesis Transfer
• Adversarial Transformations for Semi-Supervised Learning
• Temporal Self-Ensembling Teacher for Semi-Supervised Object Detection
• Transfer Learning with Pre-trained Conditional Generative Models
• HybridNet: Classification and Reconstruction Cooperation for Semi-Supervised Learning
• SaaS: Speed as a Supervisor for Semi-supervised Learning
• Towards Robust Pattern Recognition: A Review
• ContraCluster: Learning to Classify without Labels by Contrastive Self-Supervision and Prototype-Based Semi-Supervision
• On Diversity in Discriminative Neural Networks
• Investigating Semi-Supervised Learning Algorithms in Text Datasets
• Minimizing Labeling Effort for Tree Skeleton Segmentation using an Automated Iterative Training Methodology
• Credal Self-Supervised Learning
• Discriminate-and-Rectify Encoders: Learning from Image Transformation Sets
• End-To-End Graph-based Deep Semi-Supervised Learning