MiCE: Mixture of Contrastive Experts for Unsupervised Image Clustering
arXiv:2105.01899
Abstract
We present Mixture of Contrastive Experts (MiCE), a unified probabilistic clustering framework that simultaneously exploits the discriminative representations learned by contrastive learning and the semantic structures captured by a latent mixture model. Motivated by the mixture of experts, MiCE employs a gating function to partition an unlabeled dataset into subsets according to the latent semantics and multiple experts to discriminate distinct subsets of instances assigned to them in a contrastive learning manner. To solve the nontrivial inference and learning problems caused by the latent variables, we further develop a scalable variant of the Expectation-Maximization (EM) algorithm for MiCE and provide proof of the convergence. Empirically, we evaluate the clustering performance of MiCE on four widely adopted natural image datasets. MiCE achieves significantly better results than various previous methods and a strong contrastive learning baseline.
arXiv commentInternational Conference on Learning Representations (ICLR) 2021
References in corpus (18)
- Auto-Encoding Variational Bayes
- Distilling the Knowledge in a Neural Network
- A Simple Framework for Contrastive Learning of Visual Representations
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- Momentum Contrast for Unsupervised Visual Representation Learning
- Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering
- Contrastive Multiview Coding
- Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
- Prototypical Contrastive Learning of Unsupervised Representations
- Learning Discrete Representations via Information Maximizing Self-Augmented Training
- Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
- Deep Unsupervised Clustering Using Mixture of Autoencoders
- Max-Mahalanobis Linear Discriminant Analysis Networks
- Graphical Generative Adversarial Networks
- GATCluster: Self-Supervised Gaussian-Attention Network for Image Clustering
- Countering Noisy Labels By Learning From Auxiliary Clean Labels
- DHOG: Deep Hierarchical Object Grouping