Bayesian Coresets: Revisiting the Nonconvex Optimization Perspective

Abstract

Bayesian coresets have emerged as a promising approach for implementing scalable Bayesian inference. The Bayesian coreset problem involves selecting a (weighted) subset of the data samples, such that the posterior inference using the selected subset closely approximates the posterior inference using the full dataset. This manuscript revisits Bayesian coresets through the lens of sparsity constrained optimization. Leveraging recent advances in accelerated optimization methods, we propose and analyze a novel algorithm for coreset selection. We provide explicit convergence rate guarantees and present an empirical evaluation on a variety of benchmark datasets to highlight our proposed algorithm's superior performance compared to state-of-the-art on speed and accuracy.

arXiv commentAISTATS 2021 (Oral)

References in corpus (11)

• Sampling Can Be Faster Than Optimization
• Streaming Variational Bayes
• Coresets for Scalable Bayesian Logistic Regression
• Bayesian Posterior Sampling via Stochastic Gradient Fisher Scoring
• Automated Scalable Bayesian Inference via Hilbert Coresets
• Bayesian Batch Active Learning as Sparse Subset Approximation
• Firefly Monte Carlo: Exact MCMC with Subsets of Data
• IHT dies hard: Provable accelerated Iterative Hard Thresholding
• Boosting Variational Inference: an Optimization Perspective
• Variational consensus Monte Carlo
• Learning Sparse Distributions using Iterative Hard Thresholding