Robust Bayesian Inference for Discrete Outcomes with the Total Variation Distance

Abstract

Models of discrete-valued outcomes are easily misspecified if the data exhibit zero-inflation, overdispersion or contamination. Without additional knowledge about the existence and nature of this misspecification, model inference and prediction are adversely affected. Here, we introduce a robust discrepancy-based Bayesian approach using the Total Variation Distance (TVD). In the process, we address and resolve two challenges: First, we study convergence and robustness properties of a computationally efficient estimator for the TVD between a parametric model and the data-generating mechanism. Second, we provide an efficient inference method adapted from Lyddon et al. (2019) which corresponds to formulating an uninformative nonparametric prior directly over the data-generating mechanism. Lastly, we empirically demonstrate that our approach is robust and significantly improves predictive performance on a range of simulated and real world data.

arXiv comment16p., 7 figs.; authors contributed equally & author order determined by coin flip

References in corpus (9)

• Principles of Bayesian Inference using General Divergence Criteria
• Generalized Variational Inference: Three arguments for deriving new Posteriors
• A Primer on PAC-Bayesian Learning
• Statistical Inference for Generative Models with Maximum Mean Discrepancy
• Nonparametric learning from Bayesian models with randomized objective functions
• Asymptotic normality, concentration, and coverage of generalized posteriors
• Scalable Nonparametric Sampling from Multimodal Posteriors with the Posterior Bootstrap
• Concentration Inequalities for the Empirical Distribution
• Frequentist Consistency of Generalized Variational Inference