Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders

Abstract

We study a variant of the variational autoencoder model (VAE) with a Gaussian mixture as a prior distribution, with the goal of performing unsupervised clustering through deep generative models. We observe that the known problem of over-regularisation that has been shown to arise in regular VAEs also manifests itself in our model and leads to cluster degeneracy. We show that a heuristic called minimum information constraint that has been shown to mitigate this effect in VAEs can also be applied to improve unsupervised clustering performance with our model. Furthermore we analyse the effect of this heuristic and provide an intuition of the various processes with the help of visualizations. Finally, we demonstrate the performance of our model on synthetic data, MNIST and SVHN, showing that the obtained clusters are distinct, interpretable and result in achieving competitive performance on unsupervised clustering to the state-of-the-art results.

12 pages, 6 figures, Under review as a conference paper at ICLR 2017

References in corpus (1)

• Stochastic Backpropagation through Mixture Density Distributions

Cited by in corpus (15)

• Topic-Guided Variational Autoencoders for Text Generation
• Deep Discriminative Clustering Analysis
• Deep Unsupervised Clustering Using Mixture of Autoencoders
• GAP: Generalizable Approximate Graph Partitioning Framework
• Variational Memory Addressing in Generative Models
• X-ray Study of Spatial Structures in Tycho's Supernova Remnant Using Unsupervised Deep Learning
• On the Necessity and Effectiveness of Learning the Prior of Variational Auto-Encoder
• Multi-objects Generation with Amortized Structural Regularization
• Learning Disentangled Representations of Timbre and Pitch for Musical Instrument Sounds Using Gaussian Mixture Variational Autoencoders
• Continual Learning of New Sound Classes using Generative Replay
• Deep Clustering With Intra-class Distance Constraint for Hyperspectral Images
• Automatic dimensionality selection for principal component analysis models with the ignorance score
• WiSE-ALE: Wide Sample Estimator for Approximate Latent Embedding
• Generating the support with extreme value losses
• Coarse Grained Exponential Variational Autoencoders