EnAET: A Self-Trained framework for Semi-Supervised and Supervised Learning with Ensemble Transformations

Abstract

Deep neural networks have been successfully applied to many real-world applications. However, such successes rely heavily on large amounts of labeled data that is expensive to obtain. Recently, many methods for semi-supervised learning have been proposed and achieved excellent performance. In this study, we propose a new EnAET framework to further improve existing semi-supervised methods with self-supervised information. To our best knowledge, all current semi-supervised methods improve performance with prediction consistency and confidence ideas. We are the first to explore the role of {\bf self-supervised} representations in {\bf semi-supervised} learning under a rich family of transformations. Consequently, our framework can integrate the self-supervised information as a regularization term to further improve {\it all} current semi-supervised methods. In the experiments, we use MixMatch, which is the current state-of-the-art method on semi-supervised learning, as a baseline to test the proposed EnAET framework. Across different datasets, we adopt the same hyper-parameters, which greatly improves the generalization ability of the EnAET framework. Experiment results on different datasets demonstrate that the proposed EnAET framework greatly improves the performance of current semi-supervised algorithms. Moreover, this framework can also improve {\bf supervised learning} by a large margin, including the extremely challenging scenarios with only 10 images per class. The code and experiment records are available in \url{https://github.com/maple-research-lab/EnAET}.

arXiv comment10 pages, 3 figures, conference

References in corpus (14)

• Improved Regularization of Convolutional Neural Networks with Cutout
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Temporal Ensembling for Semi-Supervised Learning
• Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition
• Interpolation Consistency Training for Semi-Supervised Learning
• AutoAugment: Learning Augmentation Policies from Data
• CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
• Semi-Supervised Learning with Ladder Networks
• Shake-Shake regularization
• Auxiliary Deep Generative Models
• Stacked What-Where Auto-encoders
• Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules
• There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average
• Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks

Cited by in corpus (9)

• Ensemble deep learning: A review
• A survey on Semi-, Self- and Unsupervised Learning for Image Classification
• Neural Architecture Transfer
• Contrastive Learning with Stronger Augmentations
• NSGANetV2: Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Search
• Self-Supervised Learning for Semi-Supervised Temporal Action Proposal
• Pseudo-Representation Labeling Semi-Supervised Learning
• Covariate Distribution Aware Meta-learning
• LiDAM: Semi-Supervised Learning with Localized Domain Adaptation and Iterative Matching