Learning Latent Representations in Neural Networks for Clustering through Pseudo Supervision and Graph-based Activity Regularization

Abstract

In this paper, we propose a novel unsupervised clustering approach exploiting the hidden information that is indirectly introduced through a pseudo classification objective. Specifically, we randomly assign a pseudo parent-class label to each observation which is then modified by applying the domain specific transformation associated with the assigned label. Generated pseudo observation-label pairs are subsequently used to train a neural network with Auto-clustering Output Layer (ACOL) that introduces multiple softmax nodes for each pseudo parent-class. Due to the unsupervised objective based on Graph-based Activity Regularization (GAR) terms, softmax duplicates of each parent-class are specialized as the hidden information captured through the help of domain specific transformations is propagated during training. Ultimately we obtain a k-means friendly latent representation. Furthermore, we demonstrate how the chosen transformation type impacts performance and helps propagate the latent information that is useful in revealing unknown clusters. Our results show state-of-the-art performance for unsupervised clustering tasks on MNIST, SVHN and USPS datasets, with the highest accuracies reported to date in the literature.

arXiv commentTo appear in proceedings of the International Conference on Learning Representations 2018

References in corpus (4)

• Adam: A Method for Stochastic Optimization
• Natural Language Processing (almost) from Scratch
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders

Cited by in corpus (6)

• Gaussian Mixture Generative Adversarial Networks for Diverse Datasets, and the Unsupervised Clustering of Images
• Unsupervised Semantic Aggregation and Deformable Template Matching for Semi-Supervised Learning
• Disentangling to Cluster: Gaussian Mixture Variational Ladder Autoencoders
• Streaming Adaptive Nonparametric Variational Autoencoder
• Geographical Knowledge-driven Representation Learning for Remote Sensing Images
• Deep Goal-Oriented Clustering