Max-margin Deep Generative Models

Abstract

Deep generative models (DGMs) are effective on learning multilayered representations of complex data and performing inference of input data by exploring the generative ability. However, little work has been done on examining or empowering the discriminative ability of DGMs on making accurate predictions. This paper presents max-margin deep generative models (mmDGMs), which explore the strongly discriminative principle of max-margin learning to improve the discriminative power of DGMs, while retaining the generative capability. We develop an efficient doubly stochastic subgradient algorithm for the piecewise linear objective. Empirical results on MNIST and SVHN datasets demonstrate that (1) max-margin learning can significantly improve the prediction performance of DGMs and meanwhile retain the generative ability; and (2) mmDGMs are competitive to the state-of-the-art fully discriminative networks by employing deep convolutional neural networks (CNNs) as both recognition and generative models.

References in corpus (3)

• Semi-Supervised Learning with Deep Generative Models
• Theano: new features and speed improvements
• Stochastic Pooling for Regularization of Deep Convolutional Neural Networks

Cited by in corpus (10)

• Variational Autoencoder for Deep Learning of Images, Labels and Captions
• Variational Temporal Deep Generative Model for Radar HRRP Target Recognition
• Conditional Generative Moment-Matching Networks
• Learning Kernel for Conditional Moment-Matching Discrepancy-based Image Classification
• Learning to Generate with Memory
• A Deep Generative Deconvolutional Image Model
• Triple Generative Adversarial Networks
• Multi-objects Generation with Amortized Structural Regularization
• Deep Autoencoding Topic Model with Scalable Hybrid Bayesian Inference
• Semi-supervised Learning with Contrastive Predicative Coding