Optimally-Weighted Herding is Bayesian Quadrature

Abstract

Herding and kernel herding are deterministic methods of choosing samples which summarise a probability distribution. A related task is choosing samples for estimating integrals using Bayesian quadrature. We show that the criterion minimised when selecting samples in kernel herding is equivalent to the posterior variance in Bayesian quadrature. We then show that sequential Bayesian quadrature can be viewed as a weighted version of kernel herding which achieves performance superior to any other weighted herding method. We demonstrate empirically a rate of convergence faster than O(1/N). Our results also imply an upper bound on the empirical error of the Bayesian quadrature estimate.

Appears in Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence (UAI2012). This copy was withdrawn since it's a duplicate of arXiv:1204.1664

References in corpus (1)

• On the Equivalence between Herding and Conditional Gradient Algorithms

Cited by in corpus (13)

• Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences
• Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernels
• On two ways to use determinantal point processes for Monte Carlo integration
• Sampling based approximation of linear functionals in Reproducing Kernel Hilbert Spaces
• Control functionals for Monte Carlo integration
• Convergence Analysis of Deterministic Kernel-Based Quadrature Rules in Misspecified Settings
• Kernel interpolation with continuous volume sampling
• On the Sampling Problem for Kernel Quadrature
• Data-driven Random Fourier Features using Stein Effect
• Bayesian quadrature and energy minimization for space-filling design
• Probabilistic Models for Integration Error in the Assessment of Functional Cardiac Models
• Herding Generalizes Diverse M -Best Solutions
• Compressed particle methods for expensive models with application in Astronomy and Remote Sensing