Semi-Supervised Learning with IPM-based GANs: an Empirical Study
arXiv:1712.02505
Abstract
We present an empirical investigation of a recent class of Generative Adversarial Networks (GANs) using Integral Probability Metrics (IPM) and their performance for semi-supervised learning. IPM-based GANs like Wasserstein GAN, Fisher GAN and Sobolev GAN have desirable properties in terms of theoretical understanding, training stability, and a meaningful loss. In this work we investigate how the design of the critic (or discriminator) influences the performance in semi-supervised learning. We distill three key take-aways which are important for good SSL performance: (1) the K+1 formulation, (2) avoiding batch normalization in the critic and (3) avoiding gradient penalty constraints on the classification layer.
Appeared at NIPS 2017 Workshop: Deep Learning: Bridging Theory and Practice
References in corpus (9)
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Temporal Ensembling for Semi-Supervised Learning
- Improved Training of Wasserstein GANs
- MMD GAN: Towards Deeper Understanding of Moment Matching Network
- Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
- Good Semi-supervised Learning that Requires a Bad GAN
- Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
- On integral probability metrics, ϕ-divergences and binary classification
- McGan: Mean and Covariance Feature Matching GAN