Dependent Multinomial Models Made Easy: Stick Breaking with the Pólya-Gamma Augmentation
arXiv:1506.05843
Abstract
Many practical modeling problems involve discrete data that are best represented as draws from multinomial or categorical distributions. For example, nucleotides in a DNA sequence, children's names in a given state and year, and text documents are all commonly modeled with multinomial distributions. In all of these cases, we expect some form of dependency between the draws: the nucleotide at one position in the DNA strand may depend on the preceding nucleotides, children's names are highly correlated from year to year, and topics in text may be correlated and dynamic. These dependencies are not naturally captured by the typical Dirichlet-multinomial formulation. Here, we leverage a logistic stick-breaking representation and recent innovations in Pólya-gamma augmentation to reformulate the multinomial distribution in terms of latent variables with jointly Gaussian likelihoods, enabling us to take advantage of a host of Bayesian inference techniques for Gaussian models with minimal overhead.
References in corpus (3)
Cited by in corpus (20)
- Recurrent switching linear dynamical systems
- A New Spatial Count Data Model with Bayesian Additive Regression Trees for Accident Hot Spot Identification
- GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning
- Personalized Federated Learning with Gaussian Processes
- Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentation
- Bayesian Few-Shot Classification with One-vs-Each Pólya-Gamma Augmented Gaussian Processes
- A Bayesian Markov model with Pólya-Gamma sampling for estimating individual behavior transition probabilities from accelerometer classifications
- Latent Dirichlet Analysis of Categorical Survey Responses
- Efficient Bayesian Inference of Sigmoidal Gaussian Cox Processes
- Bayesian Inference for Gamma Models
- Computationally Efficient Bayesian Unit-Level Models for Non-Gaussian Data Under Informative Sampling
- An Asymptotically Efficient Metropolis-Hastings Sampler for Bayesian Inference in Large-Scale Educational Measuremen
- Scalable Cross Validation Losses for Gaussian Process Models
- Gamma Belief Networks
- Latent variable modeling with random features
- Bayesian Hierarchical Models for High-Dimensional Mediation Analysis with Coordinated Selection of Correlated Mediators
- A Simple Correction Procedure for High-Dimensional Generalized Linear Models with Measurement Error
- Bayesian Classifier Fusion with an Explicit Model of Correlation
- Fast Bayesian inference for large occupancy data sets, using the Polya-Gamma scheme
- Computationally Efficient Deep Bayesian Unit-Level Modeling of Survey Data under Informative Sampling for Small Area Estimation