Convolutional Clustering for Unsupervised Learning

Abstract

The task of labeling data for training deep neural networks is daunting and tedious, requiring millions of labels to achieve the current state-of-the-art results. Such reliance on large amounts of labeled data can be relaxed by exploiting hierarchical features via unsupervised learning techniques. In this work, we propose to train a deep convolutional network based on an enhanced version of the k-means clustering algorithm, which reduces the number of correlated parameters in the form of similar filters, and thus increases test categorization accuracy. We call our algorithm convolutional k-means clustering. We further show that learning the connection between the layers of a deep convolutional neural network improves its ability to be trained on a smaller amount of labeled data. Our experiments show that the proposed algorithm outperforms other techniques that learn filters unsupervised. Specifically, we obtained a test accuracy of 74.1% on STL-10 and a test error of 0.5% on MNIST.

11 pages

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Improving neural networks by preventing co-adaptation of feature detectors
• Going Deeper with Convolutions
• Stacked What-Where Auto-encoders

Cited by in corpus (15)

• What Do We Understand About Convolutional Networks?
• Identifying Strong Lenses with Unsupervised Machine Learning using Convolutional Autoencoder
• Reversible Architectures for Arbitrarily Deep Residual Neural Networks
• Whitening for Self-Supervised Representation Learning
• Self-supervised learning of visual features through embedding images into text topic spaces
• Autoencoder-based time series clustering with energy applications
• Self-Supervised Feature Learning by Learning to Spot Artifacts
• CSNNs: Unsupervised, Backpropagation-free Convolutional Neural Networks for Representation Learning
• Leveraging Random Label Memorization for Unsupervised Pre-Training
• TextTopicNet - Self-Supervised Learning of Visual Features Through Embedding Images on Semantic Text Spaces
• Deep Clustering With Intra-class Distance Constraint for Hyperspectral Images
• MUSCLE: Strengthening Semi-Supervised Learning Via Concurrent Unsupervised Learning Using Mutual Information Maximization
• Self-Supervised Visual Representations for Cross-Modal Retrieval
• Deep Micro-Dictionary Learning and Coding Network
• Diversity Regularized Adversarial Learning