Independent finite approximations for Bayesian nonparametric inference
arXiv:2009.10780 · doi:10.1214/23-BA1385
Abstract
Completely random measures (CRMs) and their normalizations (NCRMs) offer flexible models in Bayesian nonparametrics. But their infinite dimensionality presents challenges for inference. Two popular finite approximations are truncated finite approximations (TFAs) and independent finite approximations (IFAs). While the former have been well-studied, IFAs lack similarly general bounds on approximation error, and there has been no systematic comparison between the two options. In the present work, we propose a general recipe to construct practical finite-dimensional approximations for homogeneous CRMs and NCRMs, in the presence or absence of power laws. We call our construction the automated independent finite approximation (AIFA). Relative to TFAs, we show that AIFAs facilitate more straightforward derivations and use of parallel computing in approximate inference. We upper bound the approximation error of AIFAs for a wide class of common CRMs and NCRMs -- and thereby develop guidelines for choosing the approximation level. Our lower bounds in key cases suggest that our upper bounds are tight. We prove that, for worst-case choices of observation likelihoods, TFAs are more efficient than AIFAs. Conversely, we find that in real-data experiments with standard likelihoods, AIFAs and TFAs perform similarly. Moreover, we demonstrate that AIFAs can be used for hyperparameter estimation even when other potential IFA options struggle or do not apply.
arXiv commentThe paper has been accepted for publication in Bayesian Analysis. Currently, it is posted on Bayesian Analysis Advance Publication
References in corpus (15)
- SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
- Auto-Encoding Variational Bayes
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
- Programming with models: writing statistical algorithms for general model structures with NIMBLE
- Pyro: Deep Universal Probabilistic Programming
- Automatic Differentiation Variational Inference
- Beta-Negative Binomial Process and Poisson Factor Analysis
- Bayesian Nonparametric Hidden Semi-Markov Models
- An Infinite Latent Attribute Model for Network Data
- Nonparametric Bayesian Factor Analysis for Dynamic Count Matrices
- Stochastic approximations to the Pitman-Yor process
- Truncated Random Measures
- Gamma Processes, Stick-Breaking, and Variational Inference
- A unified construction for series representations and finite approximations of completely random measures