Joint Unsupervised Learning of Deep Representations and Image Clusters

Abstract

In this paper, we propose a recurrent framework for Joint Unsupervised LEarning (JULE) of deep representations and image clusters. In our framework, successive operations in a clustering algorithm are expressed as steps in a recurrent process, stacked on top of representations output by a Convolutional Neural Network (CNN). During training, image clusters and representations are updated jointly: image clustering is conducted in the forward pass, while representation learning in the backward pass. Our key idea behind this framework is that good representations are beneficial to image clustering and clustering results provide supervisory signals to representation learning. By integrating two processes into a single model with a unified weighted triplet loss and optimizing it end-to-end, we can obtain not only more powerful representations, but also more precise image clusters. Extensive experiments show that our method outperforms the state-of-the-art on image clustering across a variety of image datasets. Moreover, the learned representations generalize well when transferred to other tasks.

19 pages, 11 figures, 14 tables, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Unsupervised Learning of Visual Representations using Videos
• Building high-level features using large scale unsupervised learning
• Deep Learning with Nonparametric Clustering
• Spike-and-Slab Sparse Coding for Unsupervised Feature Discovery
• Graph Degree Linkage: Agglomerative Clustering on a Directed Graph
• Learning A Task-Specific Deep Architecture For Clustering

Cited by in corpus (35)

• Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering
• Clustering with Deep Learning: Taxonomy and New Methods
• Discovering New Intents with Deep Aligned Clustering
• Efficient Self-supervised Vision Transformers for Representation Learning
• Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reduction
• Deep Continuous Clustering
• Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization
• Deep Discriminative Clustering Analysis
• Automatically Discovering and Learning New Visual Categories with Ranking Statistics
• Deep Sparse Subspace Clustering
• Probabilistic Attribute Tree in Convolutional Neural Networks for Facial Expression Recognition
• Discriminatively Boosted Image Clustering with Fully Convolutional Auto-Encoders
• Improving Image Clustering With Multiple Pretrained CNN Feature Extractors
• Elastic-InfoGAN: Unsupervised Disentangled Representation Learning in Class-Imbalanced Data
• Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination
• Deep Discriminative Latent Space for Clustering
• Exploring Instance Relations for Unsupervised Feature Embedding
• GATCluster: Self-Supervised Gaussian-Attention Network for Image Clustering
• Deep Density-based Image Clustering
• A Framework for Deep Constrained Clustering -- Algorithms and Advances
• Deep Image Clustering with Category-Style Representation
• Unsupervised Joint Mining of Deep Features and Image Labels for Large-scale Radiology Image Categorization and Scene Recognition
• MixNMatch: Multifactor Disentanglement and Encoding for Conditional Image Generation
• PQk-means: Billion-scale Clustering for Product-quantized Codes
• Disentangling to Cluster: Gaussian Mixture Variational Ladder Autoencoders
• Cluster Analysis with Deep Embeddings and Contrastive Learning
• Universal Perceptual Grouping
• Unsupervised Continual Learning Via Pseudo Labels
• Weakly supervised discriminative feature learning with state information for person identification
• Towards Robust Pattern Recognition: A Review
• Deep Transformation-Invariant Clustering
• Image Disguise based on Generative Model
• From Time Series to Euclidean Spaces: On Spatial Transformations for Temporal Clustering
• Interpretable Deep Neural Networks for Facial Expression and Dimensional Emotion Recognition in-the-wild
• Information based Deep Clustering: An experimental study