Good Semi-supervised Learning that Requires a Bad GAN

Abstract

Semi-supervised learning methods based on generative adversarial networks (GANs) obtained strong empirical results, but it is not clear 1) how the discriminator benefits from joint training with a generator, and 2) why good semi-supervised classification performance and a good generator cannot be obtained at the same time. Theoretically, we show that given the discriminator objective, good semisupervised learning indeed requires a bad generator, and propose the definition of a preferred generator. Empirically, we derive a novel formulation based on our analysis that substantially improves over feature matching GANs, obtaining state-of-the-art results on multiple benchmark datasets.

NIPS 2017 camera ready

References in corpus (7)

• Semi-Supervised Classification with Graph Convolutional Networks
• Temporal Ensembling for Semi-Supervised Learning
• Energy-based Generative Adversarial Network
• Triple Generative Adversarial Nets
• Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• Amortised MAP Inference for Image Super-resolution
• Semi-Supervised QA with Generative Domain-Adaptive Nets

Cited by in corpus (40)

• Data-Efficient Image Recognition with Contrastive Predictive Coding
• A DIRT-T Approach to Unsupervised Domain Adaptation
• Adversarial Reciprocal Points Learning for Open Set Recognition
• Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Generative Adversarial Networks and Conditional Random Fields for Hyperspectral Image Classification
• Diffusion Models, Image Super-Resolution And Everything: A Survey
• Towards Label-efficient Automatic Diagnosis and Analysis: A Comprehensive Survey of Advanced Deep Learning-based Weakly-supervised, Semi-supervised and Self-supervised Techniques in Histopathological Image Analysis
• Semi-supervised Domain Adaptation via Minimax Entropy
• A review of Generative Adversarial Networks for Electronic Health Records: applications, evaluation measures and data sources
• Low-Shot Learning from Imaginary Data
• Real Risks of Fake Data: Synthetic Data, Diversity-Washing and Consent Circumvention
• SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training
• Novelty Detection with GAN
• Pulsar Candidate Identification Using Semi-Supervised Generative Adversarial Networks
• SEAL: Semi-supervised Adversarial Active Learning on Attributed Graphs
• Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation
• Smooth Neighbors on Teacher Graphs for Semi-supervised Learning
• Generative Adversarial Networks: A Survey Towards Private and Secure Applications
• Semi-supervised Rare Disease Detection Using Generative Adversarial Network
• Not to Cry Wolf: Distantly Supervised Multitask Learning in Critical Care
• A Comparative Review of Recent Few-Shot Object Detection Algorithms
• One-Class Adversarial Nets for Fraud Detection
• Imbalanced Data Learning by Minority Class Augmentation using Capsule Adversarial Networks
• Boundary-Refined Prototype Generation: A General End-to-End Paradigm for Semi-Supervised Semantic Segmentation
• Deep Low-Density Separation for Semi-Supervised Classification
• Semi-Supervised Learning Enabled by Multiscale Deep Neural Network Inversion
• Global versus Localized Generative Adversarial Nets
• ScarceGAN: Discriminative Classification Framework for Rare Class Identification for Longitudinal Data with Weak Prior
• Joint Learning of Generative Translator and Classifier for Visually Similar Classes
• Sparsely Grouped Multi-task Generative Adversarial Networks for Facial Attribute Manipulation
• Modeling EEG data distribution with a Wasserstein Generative Adversarial Network to predict RSVP Events
• Energy-Based Models with Applications to Speech and Language Processing
• Consistency Regularization with Generative Adversarial Networks for Semi-Supervised Learning
• Generative-Discriminative Complementary Learning
• Semi-Supervised Learning with IPM-based GANs: an Empirical Study
• Online Detection of Action Start in Untrimmed, Streaming Videos
• ROI Regularization for Semi-supervised and Supervised Learning
• End-To-End Graph-based Deep Semi-Supervised Learning
• Sample Complexity of Nonparametric Semi-Supervised Learning