Optimally-Weighted Herding is Bayesian Quadrature

Abstract

Herding and kernel herding are deterministic methods of choosing samples which summarise a probability distribution. A related task is choosing samples for estimating integrals using Bayesian quadrature. We show that the criterion minimised when selecting samples in kernel herding is equivalent to the posterior variance in Bayesian quadrature. We then show that sequential Bayesian quadrature can be viewed as a weighted version of kernel herding which achieves performance superior to any other weighted herding method. We demonstrate empirically a rate of convergence faster than O(1/N). Our results also imply an upper bound on the empirical error of the Bayesian quadrature estimate.

Accepted as an oral presentation at Uncertainty in Artificial Intelligence 2012. Updated to fix several typos

References in corpus (1)

• On the Equivalence between Herding and Conditional Gradient Algorithms

Cited by in corpus (14)

• Kernel Mean Embedding of Distributions: A Review and Beyond
• Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
• Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees
• Compressed Monte Carlo with application in particle filtering
• Sampling Permutations for Shapley Value Estimation
• Metrizing Weak Convergence with Maximum Mean Discrepancies
• Sampling based approximation of linear functionals in Reproducing Kernel Hilbert Spaces
• Optimal quantisation of probability measures using maximum mean discrepancy
• Sparse Variational Inference: Bayesian Coresets from Scratch
• Positively Weighted Kernel Quadrature via Subsampling
• Sparser Kernel Herding with Pairwise Conditional Gradients without Swap Steps
• Kernel quadrature by applying a point-wise gradient descent method to discrete energies
• Estimating Rényi's $\alpha$-Cross-Entropies in a Matrix-Based Way
• Kernel quadrature with DPPs