Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy

Abstract

Probabilistic (Bayesian) modeling has experienced a surge of applications in almost all quantitative sciences and industrial areas. This development is driven by a combination of several factors, including better probabilistic estimation algorithms, flexible software, increased computing power, and a growing awareness of the benefits of probabilistic learning. However, a principled Bayesian model building workflow is far from complete and many challenges remain. To aid future research and applications of a principled Bayesian workflow, we ask and provide answers for what we perceive as two fundamental questions of Bayesian modeling, namely (a) "What actually is a Bayesian model?" and (b) "What makes a good Bayesian model?". As an answer to the first question, we propose the PAD model taxonomy that defines four basic kinds of Bayesian models, each representing some combination of the assumed joint distribution of all (known or unknown) variables (P), a posterior approximator (A), and training data (D). As an answer to the second question, we propose ten utility dimensions according to which we can evaluate Bayesian models holistically, namely, (1) causal consistency, (2) parameter recoverability, (3) predictive performance, (4) fairness, (5) structural faithfulness, (6) parsimony, (7) interpretability, (8) convergence, (9) estimation speed, and (10) robustness. Further, we propose two example utility decision trees that describe hierarchies and trade-offs between utilities depending on the inferential goals that drive model building and testing.

References in corpus (86)

• Adam: A Method for Stochastic Optimization
• Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
• Variational Inference: A Review for Statisticians
• Towards A Rigorous Science of Interpretable Machine Learning
• To Explain or to Predict?
• Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory
• The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
• Rank-normalization, folding, and localization: An improved $\hat{R}$ for assessing convergence of MCMC
• Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Expectation Propagation for approximate Bayesian inference
• Normalizing Flows: An Introduction and Review of Current Methods
• Visualization in Bayesian workflow
• Superstatistics
• The frontier of simulation-based inference
• A Survey on the Explainability of Supervised Machine Learning
• Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users
• Deep Neural Networks as Gaussian Processes
• Using stacking to average Bayesian predictive distributions
• Sparsity information and regularization in the horseshoe and other shrinkage priors
• The prior can generally only be understood in the context of the likelihood
• Normalizing Flows for Probabilistic Modeling and Inference
• Comparison of Bayesian predictive methods for model selection
• Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis
• A General Framework for Updating Belief Distributions
• Bayesian Inference with Optimal Maps
• Markov Chain Monte Carlo: Can We Trust the Third Significant Figure?
• The Frontiers of Fairness in Machine Learning
• Validating Bayesian Inference Algorithms with Simulation-Based Calibration
• Pareto Smoothed Importance Sampling
• Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs
• Masked Autoregressive Flow for Density Estimation
• Essentially No Barriers in Neural Network Energy Landscape
• Misspecification in infinite-dimensional Bayesian statistics
• Analyzing Inverse Problems with Invertible Neural Networks
• Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis
• Approximate leave-future-out cross-validation for Bayesian time series models
• Flexible statistical inference for mechanistic models of neural dynamics
• Exposing the Probabilistic Causal Structure of Discrimination
• Likelihood-free inference with emulator networks
• Automatic Posterior Transformation for Likelihood-Free Inference
• CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training
• Simulation Intelligence: Towards a New Generation of Scientific Methods
• Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows
• Learning Summary Statistic for Approximate Bayesian Computation via Deep Neural Network
• Expectation propagation as a way of life: A framework for Bayesian inference on partitioned data
• Effective Degrees of Freedom: A Flawed Metaphor
• Efficient leave-one-out cross-validation for Bayesian non-factorized normal and Student-t models
• Inference Networks for Sequential Monte Carlo in Graphical Models
• Weak Informativity and the Information in One Prior Relative to Another
• Model Selection in Bayesian Neural Networks via Horseshoe Priors
• On the Expressiveness of Approximate Inference in Bayesian Neural Networks
• Graphical Test for Discrete Uniformity and its Applications in Goodness of Fit Evaluation and Multiple Sample Comparison
• Markov Chain Monte Carlo Estimation of Quantiles
• Implicitly Adaptive Importance Sampling
• OutbreakFlow: Model-based Bayesian inference of disease outbreak dynamics with invertible neural networks and its application to the COVID-19 pandemics in Germany
• Improving multilevel regression and poststratification with structured priors
• Inference Compilation and Universal Probabilistic Programming
• Likelihood-free MCMC with Amortized Approximate Ratio Estimators
• Exploring the Nonlinear Cloud and Rain Equation
• Truncated proposals for scalable and hassle-free simulation-based inference
• Intuitive Joint Priors for Bayesian Linear Multilevel Models: The R2D2M2 prior
• Calibrating Model-Based Inferences and Decisions
• Projection Predictive Inference for Generalized Linear and Additive Multilevel Models
• Prepaid parameter estimation without likelihoods
• Neural Approximate Sufficient Statistics for Implicit Models
• On Contrastive Learning for Likelihood-free Inference
• A multiscale strategy for Bayesian inference using transport maps
• Bayesian Model Selection, the Marginal Likelihood, and Generalization
• An Evolutionary Approach to Dynamic Introduction of Tasks in Large-scale Multitask Learning Systems
• A fully Bayesian sparse polynomial chaos expansion approach with joint priors on the coefficients and global selection of terms
• Learning under Model Misspecification: Applications to Variational and Ensemble methods
• Robust, Accurate Stochastic Optimization for Variational Inference
• Detecting Model Misspecification in Amortized Bayesian Inference with Neural Networks
• JAGS, NIMBLE, Stan: a detailed comparison among Bayesian MCMC software
• Robust Neural Posterior Estimation and Statistical Model Criticism
• Global Non-convex Optimization with Discretized Diffusions
• Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoods
• Sequential Neural Posterior and Likelihood Approximation
• JANA: Jointly Amortized Neural Approximation of Complex Bayesian Models
• Likelihood-Free Inference with Generative Neural Networks via Scoring Rule Minimization
• A Framework for Improving the Reliability of Black-box Variational Inference
• Physics-informed dynamic mode decomposition (piDMD)
• Measuring the non-asymptotic convergence of sequential Monte Carlo samplers using probabilistic programming
• Amortized Bayesian Inference for Models of Cognition
• Locally Adaptive Shrinkage Priors for Trends and Breaks in Count Time Series

Cited by in corpus (2)

• Uncertainty Quantification and Propagation in Surrogate-based Bayesian Inference
• Gaussian distributional structural equation models: A framework for modeling latent heteroscedasticity