Neural Clustering Processes

Abstract

Probabilistic clustering models (or equivalently, mixture models) are basic building blocks in countless statistical models and involve latent random variables over discrete spaces. For these models, posterior inference methods can be inaccurate and/or very slow. In this work we introduce deep network architectures trained with labeled samples from any generative model of clustered datasets. At test time, the networks generate approximate posterior samples of cluster labels for any new dataset of arbitrary size. We develop two complementary approaches to this task, requiring either O(N) or O(K) network forward passes per dataset, where N is the dataset size and K the number of clusters. Unlike previous approaches, our methods sample the labels of all the data points from a well-defined posterior, and can learn nonparametric Bayesian posteriors since they do not limit the number of mixture components. As a scientific application, we present a novel approach to neural spike sorting for high-density multielectrode arrays.

References in corpus (21)

• Sequence to Sequence Learning with Neural Networks
• Categorical Reparameterization with Gumbel-Softmax
• Expectation Propagation for approximate Bayesian inference
• Sequence Transduction with Recurrent Neural Networks
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications
• Clustering with Deep Learning: Taxonomy and New Methods
• Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
• Towards a Neural Statistician
• Equivariance Through Parameter-Sharing
• Non-parametric Bayesian modeling of complex networks
• Probabilistic symmetries and invariant neural networks
• Permutation-equivariant neural networks applied to dynamics prediction
• An Alternative Prior Process for Nonparametric Bayesian Clustering
• Neural Processes
• Deep Amortized Inference for Probabilistic Programs
• Deep Amortized Clustering
• Unimodal clustering using isotonic regression: ISO-SPLIT
• Reducing over-clustering via the powered Chinese restaurant process
• PINE: Universal Deep Embedding for Graph Nodes via Partial Permutation Invariant Set Functions
• Amortized Bayesian inference for clustering models

Cited by in corpus (2)

• Deep Amortized Clustering
• Disentangled Sticky Hierarchical Dirichlet Process Hidden Markov Model