Variational Representations of Annealing Paths: Bregman Information under Monotonic Embedding
arXiv:2209.07481 · doi:10.1007/s41884-023-00129-6
Abstract
Markov Chain Monte Carlo methods for sampling from complex distributions and estimating normalization constants often simulate samples from a sequence of intermediate distributions along an annealing path, which bridges between a tractable initial distribution and a target density of interest. Prior works have constructed annealing paths using quasi-arithmetic means, and interpreted the resulting intermediate densities as minimizing an expected divergence to the endpoints. To analyze these variational representations of annealing paths, we extend known results showing that the arithmetic mean over arguments minimizes the expected Bregman divergence to a single representative point. In particular, we obtain an analogous result for quasi-arithmetic means, when the inputs to the Bregman divergence are transformed under a monotonic embedding function. Our analysis highlights the interplay between quasi-arithmetic means, parametric families, and divergence functionals using the rho-tau representational Bregman divergence framework, and associates common divergence functionals with intermediate densities along an annealing path.
arXiv commentPublished in Information Geometry (Info. Geo. 2024)
References in corpus (10)
- A nonequilibrium equality for free energy differences
- Rényi Divergence and Kullback-Leibler Divergence
- Equilibrium free energy differences from nonequilibrium measurements: a master equation approach
- Parallel Tempering: Theory, Applications, and New Perspectives
- Estimating divergence functionals and the likelihood ratio by convex risk minimization
- An elementary introduction to information geometry
- The Burbea-Rao and Bhattacharyya centroids
- A new one parameter deformation of the exponential function
- A simple probabilistic construction yielding generalized entropies and divergences, escort distributions and q-Gaussians
- Dual Connections in Nonparametric Classical Information Geometry