Semi-Supervised Learning Enabled by Multiscale Deep Neural Network Inversion
arXiv:1802.10172
Abstract
Deep Neural Networks (DNNs) provide state-of-the-art solutions in several difficult machine perceptual tasks. However, their performance relies on the availability of a large set of labeled training data, which limits the breadth of their applicability. Hence, there is a need for new {\em semi-supervised learning} methods for DNNs that can leverage both (a small amount of) labeled and unlabeled training data. In this paper, we develop a general loss function enabling DNNs of any topology to be trained in a semi-supervised manner without extra hyper-parameters. As opposed to current semi-supervised techniques based on topology-specific or unstable approaches, ours is both robust and general. We demonstrate that our approach reaches state-of-the-art performance on the SVHN ( test error, with labels and wide Resnet) and CIFAR10 (16.38% test error, with 8000 labels and sigmoid convolutional neural network) data sets.
References in corpus (14)
- Adam: A Method for Stochastic Optimization
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- ADADELTA: An Adaptive Learning Rate Method
- Empirical Evaluation of Rectified Activations in Convolutional Network
- Wide Residual Networks
- Temporal Ensembling for Semi-Supervised Learning
- Improved Techniques for Training GANs
- On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
- Distributional Smoothing with Virtual Adversarial Training
- Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
- Auxiliary Deep Generative Models
- Good Semi-supervised Learning that Requires a Bad GAN
- Triple Generative Adversarial Nets
- The Impact of Local Geometry and Batch Size on Stochastic Gradient Descent for Nonconvex Problems