Semi-Supervised Generation with Cluster-aware Generative Models

Abstract

Deep generative models trained with large amounts of unlabelled data have proven to be powerful within the domain of unsupervised learning. Many real life data sets contain a small amount of labelled data points, that are typically disregarded when training generative models. We propose the Cluster-aware Generative Model, that uses unlabelled information to infer a latent representation that models the natural clustering of the data, and additional labelled data points to refine this clustering. The generative performances of the model significantly improve when labelled information is exploited, obtaining a log-likelihood of -79.38 nats on permutation invariant MNIST, while also achieving competitive semi-supervised classification accuracies. The model can also be trained fully unsupervised, and still improve the log-likelihood performance with respect to related methods.

References in corpus (12)

• Auto-Encoding Variational Bayes
• Categorical Reparameterization with Gumbel-Softmax
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Variational Inference with Normalizing Flows
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Theano: new features and speed improvements
• Semi-Supervised Learning with Ladder Networks
• Distributional Smoothing with Virtual Adversarial Training
• Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
• Auxiliary Deep Generative Models
• Deep Kalman Filters
• The Variational Gaussian Process

Cited by in corpus (3)

• BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling
• Capturing Label Characteristics in VAEs
• Adversarially Learned Mixture Model