Cluster Analysis with Deep Embeddings and Contrastive Learning
arXiv:2109.12714
Abstract
Unsupervised disentangled representation learning is a long-standing problem in computer vision. This work proposes a novel framework for performing image clustering from deep embeddings by combining instance-level contrastive learning with a deep embedding based cluster center predictor. Our approach jointly learns representations and predicts cluster centers in an end-to-end manner. This is accomplished via a three-pronged approach that combines a clustering loss, an instance-wise contrastive loss, and an anchor loss. Our fundamental intuition is that using an ensemble loss that incorporates instance-level features and a clustering procedure focusing on semantic similarity reinforces learning better representations in the latent space. We observe that our method performs exceptionally well on popular vision datasets when evaluated using standard clustering metrics such as Normalized Mutual Information (NMI), in addition to producing geometrically well-separated cluster embeddings as defined by the Euclidean distance. Our framework performs on par with widely accepted clustering methods and outperforms the state-of-the-art contrastive learning method on the CIFAR-10 dataset with an NMI score of 0.772, a 7-8% improvement on the strong baseline.
References in corpus (14)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Deep Residual Learning for Image Recognition
- Bootstrap your own latent: A new approach to self-supervised Learning
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- Barlow Twins: Self-Supervised Learning via Redundancy Reduction
- Contrastive Multiview Coding
- Prototypical Contrastive Learning of Unsupervised Representations
- Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination
- Joint Unsupervised Learning of Deep Representations and Image Clusters
- Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases
- Unsupervised Deep Embedding for Clustering Analysis
- SpectralNet: Spectral Clustering using Deep Neural Networks
- Deep Discriminative Clustering Analysis
- Supporting Clustering with Contrastive Learning