Improving Image Clustering With Multiple Pretrained CNN Feature Extractors

Abstract

For many image clustering problems, replacing raw image data with features extracted by a pretrained convolutional neural network (CNN), leads to better clustering performance. However, the specific features extracted, and, by extension, the selected CNN architecture, can have a major impact on the clustering results. In practice, this crucial design choice is often decided arbitrarily due to the impossibility of using cross-validation with unsupervised learning problems. However, information contained in the different pretrained CNN architectures may be complementary, even when pretrained on the same data. To improve clustering performance, we rephrase the image clustering problem as a multi-view clustering (MVC) problem that considers multiple different pretrained feature extractors as different "views" of the same data. We then propose a multi-input neural network architecture that is trained end-to-end to solve the MVC problem effectively. Our experimental results, conducted on three different natural image datasets, show that: 1. using multiple pretrained CNNs jointly as feature extractors improves image clustering; 2. using an end-to-end approach improves MVC; and 3. combining both produces state-of-the-art results for the problem of image clustering.

arXiv comment13 pages, 3 figures, 4 tables. Poster presentation at BMVC 2018 (29.9% acceptance)

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• CNN Features off-the-shelf: an Astounding Baseline for Recognition
• Xception: Deep Learning with Depthwise Separable Convolutions
• Learning Discrete Representations via Information Maximizing Self-Augmented Training
• Clustering with Deep Learning: Taxonomy and New Methods
• Joint Unsupervised Learning of Deep Representations and Image Clusters
• Iterative Views Agreement: An Iterative Low-Rank based Structured Optimization Method to Multi-View Spectral Clustering
• A Survey on Multi-View Clustering
• Unsupervised Joint Mining of Deep Features and Image Labels for Large-scale Radiology Image Categorization and Scene Recognition