Cluster and Feature Modeling from Combinatorial Stochastic Processes

Abstract

One of the focal points of the modern literature on Bayesian nonparametrics has been the problem of clustering, or partitioning, where each data point is modeled as being associated with one and only one of some collection of groups called clusters or partition blocks. Underlying these Bayesian nonparametric models are a set of interrelated stochastic processes, most notably the Dirichlet process and the Chinese restaurant process. In this paper we provide a formal development of an analogous problem, called feature modeling, for associating data points with arbitrary nonnegative integer numbers of groups, now called features or topics. We review the existing combinatorial stochastic process representations for the clustering problem and develop analogous representations for the feature modeling problem. These representations include the beta process and the Indian buffet process as well as new representations that provide insight into the connections between these processes. We thereby bring the same level of completeness to the treatment of Bayesian nonparametric feature modeling that has previously been achieved for Bayesian nonparametric clustering.

Published in at http://dx.doi.org/10.1214/13-STS434 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (5)

• Model Selection in Linear Mixed Models
• Beta-Negative Binomial Process and Poisson Factor Analysis
• Defining Predictive Probability Functions for Species Sampling Models
• Gibbs fragmentation trees
• Combinatorial clustering and the beta negative binomial process

Cited by in corpus (15)

• Edge-exchangeable graphs and sparsity
• A Bayesian feature allocation model for tumor heterogeneity
• Bayesian Inference for Latent Biologic Structure with Determinantal Point Processes (DPP)
• Scaling the Indian Buffet Process via Submodular Maximization
• The combinatorial structure of beta negative binomial processes
• Diffusions on a space of interval partitions: Poisson-Dirichlet stationary distributions
• Scaled subordinators and generalizations of the Indian buffet process
• Feature allocations, probability functions, and paintboxes
• Bayesian nonparametric Principal Component Analysis
• Bayesian Inference for Tumor Subclones Accounting for Sequencing and Structural Variants
• Bayesian Double Feature Allocation for Phenotyping with Electronic Health Records
• Edge-exchangeable graphs and sparsity (NIPS 2016)
• Consensus Monte Carlo for Random Subsets using Shared Anchors
• Particle-Gibbs Sampling For Bayesian Feature Allocation Models
• The pilgrim process