Nonparametric learning from Bayesian models with randomized objective functions

Abstract

Bayesian learning is built on an assumption that the model space contains a true reflection of the data generating mechanism. This assumption is problematic, particularly in complex data environments. Here we present a Bayesian nonparametric approach to learning that makes use of statistical models, but does not assume that the model is true. Our approach has provably better properties than using a parametric model and admits a Monte Carlo sampling scheme that can afford massive scalability on modern computer architectures. The model-based aspect of learning is particularly attractive for regularizing nonparametric inference when the sample size is small, and also for correcting approximate approaches such as variational Bayes (VB). We demonstrate the approach on a number of examples including VB classifiers and Bayesian random forests.

Cited by in corpus (10)

• Scalable Nonparametric Sampling from Multimodal Posteriors with the Posterior Bootstrap
• Machine learning and AI research for Patient Benefit: 20 Critical Questions on Transparency, Replicability, Ethics and Effectiveness
• Learning under Model Misspecification: Applications to Variational and Ensemble methods
• Random weighting in LASSO regression
• Robust Inference and Model Criticism Using Bagged Posteriors
• Introducing prior information in Weighted Likelihood Bootstrap with applications to model misspecification
• Asymptotics of cut distributions and robust modular inference using Posterior Bootstrap
• Bayesian Bootstrap Spike-and-Slab LASSO
• Robust Bayesian Inference for Discrete Outcomes with the Total Variation Distance
• Nonparametric posterior learning for emission tomography with multimodal data