Learning Loss Functions for Semi-supervised Learning via Discriminative Adversarial Networks

Abstract

We propose discriminative adversarial networks (DAN) for semi-supervised learning and loss function learning. Our DAN approach builds upon generative adversarial networks (GANs) and conditional GANs but includes the key differentiator of using two discriminators instead of a generator and a discriminator. DAN can be seen as a framework to learn loss functions for predictors that also implements semi-supervised learning in a straightforward manner. We propose instantiations of DAN for two different prediction tasks: classification and ranking. Our experimental results on three datasets of different tasks demonstrate that DAN is a promising framework for both semi-supervised learning and learning loss functions for predictors. For all tasks, the semi-supervised capability of DAN can significantly boost the predictor performance for small labeled sets with minor architecture changes across tasks. Moreover, the loss functions automatically learned by DANs are very competitive and usually outperform the standard pairwise and negative log-likelihood loss functions for both semi-supervised and supervised learning.

11 pages

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Conditional Generative Adversarial Nets
• Improved Techniques for Training GANs
• Convolutional Neural Networks for Sentence Classification
• Adversarially Learned Inference
• Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning
• Semi-Supervised Learning with Generative Adversarial Networks
• Dynamic Coattention Networks For Question Answering
• Machine Comprehension Using Match-LSTM and Answer Pointer
• Attentive Pooling Networks
• Auxiliary Deep Generative Models
• A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
• Bilateral Multi-Perspective Matching for Natural Language Sentences
• Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
• Open Question Answering with Weakly Supervised Embedding Models
• ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs

Cited by in corpus (10)

• Semi-supervised Learning with GANs: Manifold Invariance with Improved Inference
• Yes, we GAN: Applying Adversarial Techniques for Autonomous Driving
• Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information
• Multi-Task Handwritten Document Layout Analysis
• Learning Context-Sensitive Convolutional Filters for Text Processing
• VIABLE: Fast Adaptation via Backpropagating Learned Loss
• S4-Net: Geometry-Consistent Semi-Supervised Semantic Segmentation
• Matching Embeddings for Domain Adaptation
• Stateless actor-critic for instance segmentation with high-level priors
• A Unified Framework of Surrogate Loss by Refactoring and Interpolation