Learning by Association - A versatile semi-supervised training method for neural networks

Abstract

In many real-world scenarios, labeled data for a specific machine learning task is costly to obtain. Semi-supervised training methods make use of abundantly available unlabeled data and a smaller number of labeled examples. We propose a new framework for semi-supervised training of deep neural networks inspired by learning in humans. "Associations" are made from embeddings of labeled samples to those of unlabeled ones and back. The optimization schedule encourages correct association cycles that end up at the same class from which the association was started and penalizes wrong associations ending at a different class. The implementation is easy to use and can be added to any existing end-to-end training setup. We demonstrate the capabilities of learning by association on several data sets and show that it can improve performance on classification tasks tremendously by making use of additionally available unlabeled data. In particular, for cases with few labeled data, our training scheme outperforms the current state of the art on SVHN.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Natural Language Processing (almost) from Scratch
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Going Deeper with Convolutions
• Distributional Smoothing with Virtual Adversarial Training
• Unsupervised Visual Representation Learning by Context Prediction
• Auxiliary Deep Generative Models
• Stacked What-Where Auto-encoders
• Multi-Task Cross-Lingual Sequence Tagging from Scratch
• Building high-level features using large scale unsupervised learning
• Early Visual Concept Learning with Unsupervised Deep Learning

Cited by in corpus (25)

• Learn the new, keep the old: Extending pretrained models with new anatomy and images
• Associative Domain Adaptation
• Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings
• Semi-Supervised Learning by Label Gradient Alignment
• Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Prediction
• Unsupervised Learning Facial Parameter Regressor for Action Unit Intensity Estimation via Differentiable Renderer
• Stretching Domain Adaptation: How far is too far?
• Be Consistent! Improving Procedural Text Comprehension using Label Consistency
• Exploiting Contextual Information with Deep Neural Networks
• MUSCLE: Strengthening Semi-Supervised Learning Via Concurrent Unsupervised Learning Using Mutual Information Maximization
• SaaS: Speed as a Supervisor for Semi-supervised Learning
• Imaginative Walks: Generative Random Walk Deviation Loss for Improved Unseen Learning Representation
• Consistency Regularization with Generative Adversarial Networks for Semi-Supervised Learning
• Local Clustering with Mean Teacher for Semi-supervised Learning
• Y^2Seq2Seq: Cross-Modal Representation Learning for 3D Shape and Text by Joint Reconstruction and Prediction of View and Word Sequences
• Unsupervised Facial Action Unit Intensity Estimation via Differentiable Optimization
• Towards Robust Pattern Recognition: A Review
• Reliable Label Bootstrapping for Semi-Supervised Learning
• Regularization with Latent Space Virtual Adversarial Training
• End-To-End Graph-based Deep Semi-Supervised Learning
• Using Side Channel Information and Artificial Intelligence for Malware Detection
• Smart-Inspect: Micro Scale Localization and Classification of Smartphone Glass Defects for Industrial Automation
• Dynamic Adaptation on Non-Stationary Visual Domains
• Semi-supervised Learning with Contrastive Predicative Coding
• Iterative Self-Learning: Semi-Supervised Improvement to Dataset Volumes and Model Accuracy