Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering

Abstract

Most learning approaches treat dimensionality reduction (DR) and clustering separately (i.e., sequentially), but recent research has shown that optimizing the two tasks jointly can substantially improve the performance of both. The premise behind the latter genre is that the data samples are obtained via linear transformation of latent representations that are easy to cluster; but in practice, the transformation from the latent space to the data can be more complicated. In this work, we assume that this transformation is an unknown and possibly nonlinear function. To recover the `clustering-friendly' latent representations and to better cluster the data, we propose a joint DR and K-means clustering approach in which DR is accomplished via learning a deep neural network (DNN). The motivation is to keep the advantages of jointly optimizing the two tasks, while exploiting the deep neural network's ability to approximate any nonlinear function. This way, the proposed approach can work well for a broad class of generative models. Towards this end, we carefully design the DNN structure and the associated joint optimization criterion, and propose an effective and scalable algorithm to handle the formulated optimization problem. Experiments using different real datasets are employed to showcase the effectiveness of the proposed approach.

Final ICML2017 version. Main paper: 10 pages; Supplementary material: 4 pages

References in corpus (4)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Joint Unsupervised Learning of Deep Representations and Image Clusters
• Fast Conical Hull Algorithms for Near-separable Non-negative Matrix Factorization
• Learning From Hidden Traits: Joint Factor Analysis and Latent Clustering

Cited by in corpus (107)

• Prototypical Contrastive Learning of Unsupervised Representations
• Deep Embedded Multi-view Clustering with Collaborative Training
• Clustering with Deep Learning: Taxonomy and New Methods
• SPICE: Semantic Pseudo-labeling for Image Clustering
• Machine Learning for Intrusion Detection in Industrial Control Systems: Applications, Challenges, and Recommendations
• SpectralNet: Spectral Clustering using Deep Neural Networks
• Deep Nearest Neighbor Anomaly Detection
• Deep Temporal Clustering : Fully Unsupervised Learning of Time-Domain Features
• Classification-Based Anomaly Detection for General Data
• Discovering New Intents with Deep Aligned Clustering
• Deep Continuous Clustering
• AutoEmbedder: A semi-supervised DNN embedding system for clustering
• Distributed Traffic Synthesis and Classification in Edge Networks: A Federated Self-supervised Learning Approach
• Open-World Semi-Supervised Learning
• Deep Transparent Prediction through Latent Representation Analysis
• CGC: Contrastive Graph Clustering for Community Detection and Tracking
• RAT: Reinforcement-Learning-Driven and Adaptive Testing for Vulnerability Discovery in Web Application Firewalls
• Clustering-friendly Representation Learning via Instance Discrimination and Feature Decorrelation
• GAP: Generalizable Approximate Graph Partitioning Framework
• Automatically Discovering and Learning New Visual Categories with Ranking Statistics
• TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognition
• Temporal Phenotyping using Deep Predictive Clustering of Disease Progression
• A deep learning approach to clustering visual arts
• Continual Learning via Inter-Task Synaptic Mapping
• Learning to Discover Novel Visual Categories via Deep Transfer Clustering
• Deep Autoencoder-based Fuzzy C-Means for Topic Detection
• Image Clustering using an Augmented Generative Adversarial Network and Information Maximization
• Unsupervised Semantic Aggregation and Deformable Template Matching for Semi-Supervised Learning
• Centroid Transformers: Learning to Abstract with Attention
• Towards better Validity: Dispersion based Clustering for Unsupervised Person Re-identification
• Deep Comprehensive Correlation Mining for Image Clustering
• Self-supervised Representation Learning With Path Integral Clustering For Speaker Diarization
• RDEC: Integrating Regularization into Deep Embedded Clustering for Imbalanced Datasets
• Discovering New Intents via Constrained Deep Adaptive Clustering with Cluster Refinement
• Geodesic Clustering in Deep Generative Models
• Deep Discriminative Latent Space for Clustering
• Towards Fair Deep Clustering With Multi-State Protected Variables
• GATCluster: Self-Supervised Gaussian-Attention Network for Image Clustering
• MiCE: Mixture of Contrastive Experts for Unsupervised Image Clustering
• Deep Clustering by Semantic Contrastive Learning
• Differentiable Deep Clustering with Cluster Size Constraints
• A Framework for Deep Constrained Clustering -- Algorithms and Advances
• Improving Unsupervised Image Clustering With Robust Learning
• Towards a Near Universal Time Series Data Mining Tool: Introducing the Matrix Profile
• CARL-G: Clustering-Accelerated Representation Learning on Graphs
• Unsupervised Multiple Person Tracking using AutoEncoder-Based Lifted Multicuts
• Zooming Into the Darknet: Characterizing Internet Background Radiation and its Structural Changes
• Improving Deep Image Clustering With Spatial Transformer Layers
• Scattering Transform Based Image Clustering using Projection onto Orthogonal Complement
• Towards a Flexible Embedding Learning Framework
• Self-supervised Document Clustering Based on BERT with Data Augment
• Unsupervised Cyberbullying Detection via Time-Informed Gaussian Mixture Model
• Learning to Cluster Faces via Transformer
• Deep clustering with concrete k-means
• Multi-Facet Clustering Variational Autoencoders
• Shared Generative Latent Representation Learning for Multi-view Clustering
• Quantifying Learnability and Describability of Visual Concepts Emerging in Representation Learning
• Deep Fair Discriminative Clustering
• Learning Algorithms for Coarsening Uncertainty Space and Applications to Multiscale Simulations
• Meta-Learning to Cluster
• Efficient Clustering for Stretched Mixtures: Landscape and Optimality
• Learning for Multi-Model and Multi-Type Fitting
• Clustering-Based Representation Learning through Output Translation and Its Application to Remote--Sensing Images
• Dissimilarity Mixture Autoencoder for Deep Clustering
• Definition-independent Formalization of Soundscapes: Towards a Formal Methodology
• Dialog Intent Induction with Deep Multi-View Clustering
• Unsupervised Embedding of Hierarchical Structure in Euclidean Space
• Deep NMF Topic Modeling
• Hierarchically Clustered Representation Learning
• Disentanglement Then Reconstruction: Learning Compact Features for Unsupervised Domain Adaptation
• CEIL: A General Classification-Enhanced Iterative Learning Framework for Text Clustering
• Unsupervised deep clustering for predictive texture pattern discovery in medical images
• A Framework for Joint Unsupervised Learning of Cluster-Aware Embedding for Heterogeneous Networks
• Double cycle-consistent generative adversarial network for unsupervised conditional generation
• Multi-level Feature Learning on Embedding Layer of Convolutional Autoencoders and Deep Inverse Feature Learning for Image Clustering
• Balanced Order Batching with Task-Oriented Graph Clustering
• You Never Cluster Alone
• Smoothed Multi-View Subspace Clustering
• Clustering of Time Series Data with Prior Geographical Information
• Signal Clustering with Class-independent Segmentation
• Learning Robust Representation for Clustering through Locality Preserving Variational Discriminative Network
• A Framework for Deep Constrained Clustering
• Picking Apart Story Salads
• A Binary Optimization Approach for Constrained K-Means Clustering
• A Transfer Learning based Feature-Weak-Relevant Method for Image Clustering
• ThetA -- fast and robust clustering via a distance parameter
• An unsupervised deep learning framework via integrated optimization of representation learning and GMM-based modeling
• Unsupervised Image Segmentation by Mutual Information Maximization and Adversarial Regularization
• Generalized Clustering by Learning to Optimize Expected Normalized Cuts
• METEOR: Learning Memory and Time Efficient Representations from Multi-modal Data Streams
• Separating Boundary Points via Structural Regularization for Very Compact Clusters
• Relation-Guided Representation Learning
• Enhancing the Analysis of Software Failures in Cloud Computing Systems with Deep Learning
• Clustering with Similarity Preserving
• Representation Learning by Reconstructing Neighborhoods
• Scalable Deep $k$-Subspace Clustering
• DICE: Deep Significance Clustering for Outcome-Aware Stratification
• Recurrent Deep Divergence-based Clustering for simultaneous feature learning and clustering of variable length time series
• Detecting Anomalies Through Contrast in Heterogeneous Data
• Deep Clustering with Measure Propagation
• Label-Removed Generative Adversarial Networks Incorporating with K-Means
• From Time Series to Euclidean Spaces: On Spatial Transformations for Temporal Clustering
• Deep Embedded K-Means Clustering
• A Self-supervised Learning System for Object Detection in Videos Using Random Walks on Graphs
• UCSL : A Machine Learning Expectation-Maximization framework for Unsupervised Clustering driven by Supervised Learning
• Learning for Multi-Type Subspace Clustering
• Instance segmentation with the number of clusters incorporated in embedding learning