Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents

Abstract

In this chapter, we identify fundamental geometric structures that underlie the problems of sampling, optimisation, inference and adaptive decision-making. Based on this identification, we derive algorithms that exploit these geometric structures to solve these problems efficiently. We show that a wide range of geometric theories emerge naturally in these fields, ranging from measure-preserving processes, information divergences, Poisson geometry, and geometric integration. Specifically, we explain how (i) leveraging the symplectic geometry of Hamiltonian systems enable us to construct (accelerated) sampling and optimisation methods, (ii) the theory of Hilbertian subspaces and Stein operators provides a general methodology to obtain robust estimators, (iii) preserving the information geometry of decision-making yields adaptive agents that perform active inference. Throughout, we emphasise the rich connections between these fields; e.g., inference draws on sampling and optimisation, and adaptive decision-making assesses decisions by inferring their counterfactual consequences. Our exposition provides a conceptual overview of underlying ideas, rather than a technical discussion, which can be found in the references herein.

arXiv comment30 pages, 4 figures; 42 pages including table of contents and references

References in corpus (77)

• Variational Inference: A Review for Statisticians
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Stochastic Gradient Hamiltonian Monte Carlo
• A Variational Perspective on Accelerated Methods in Optimization
• Kernel Mean Embedding of Distributions: A Review and Beyond
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
• Stochastic gradient descent on Riemannian manifolds
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
• Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
• Optimal control as a graphical model inference problem
• Elliptical slice sampling
• An elementary introduction to information geometry
• Active inference on discrete state-spaces: a synthesis
• Hilbert space embeddings and metrics on probability measures
• A Complete Recipe for Stochastic Gradient MCMC
• Training generative neural networks via Maximum Mean Discrepancy optimization
• Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy
• The free energy principle made simpler but not too simple
• Active inference: demystified and compared
• The Zig-Zag Process and Super-Efficient Sampling for Bayesian Analysis of Big Data
• Rejection-free Monte-Carlo sampling for general potentials
• Optimal non-reversible linear drift for the convergence to equilibrium of a diffusion
• Variance Reduction using Nonreversible Langevin Samplers
• Accelerating diffusions
• Convergence of Numerical Time-Averaging and Stationary Measures via Poisson Equations
• Explicit symplectic approximation of nonseparable Hamiltonians: algorithm and long time performance
• On the Geometric Ergodicity of Hamiltonian Monte Carlo
• Geometric integrators and the Hamiltonian Monte Carlo method
• Curvature, concentration and error estimates for Markov chain Monte Carlo
• An introduction to Lie group integrators -- basics, new developments and applications
• Universality, Characteristic Kernels and RKHS Embedding of Measures
• Measuring Sample Quality with Kernels
• Proper local scoring rules
• Piecewise-Deterministic Markov Chain Monte Carlo
• Irreversible Langevin samplers and variance reduction: a large deviation approach
• Large sample analysis of the median heuristic
• Numerical integrators for the Hybrid Monte Carlo method
• Active Inference in Robotics and Artificial Agents: Survey and Challenges
• Compressible Generalized Hybrid Monte Carlo
• Improving Asymptotic Variance of MCMC Estimators: Non-reversible Chains are Better
• An empirical evaluation of active inference in multi-armed bandits
• ADMM and Accelerated ADMM as Continuous Dynamical Systems
• On the convergence of Hamiltonian Monte Carlo
• Modified Hamiltonian Monte Carlo for Bayesian inference
• Information-geometric Markov Chain Monte Carlo methods using Diffusions
• Is There an Analog of Nesterov Acceleration for MCMC?
• Statistical Inference for Generative Models with Maximum Mean Discrepancy
• Shadow Hamiltonians, Poisson Brackets, and Gauge Theories
• Extra Chance Generalized Hybrid Monte Carlo
• Identifying the Optimal Integration Time in Hamiltonian Monte Carlo
• Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families
• A Nonsmooth Dynamical Systems Perspective on Accelerated Extensions of ADMM
• Adaptive multi-stage integrators for optimal energy conservation in molecular simulations
• Geometric Generalisations of SHAKE and RATTLE
• Using Perturbed Underdamped Langevin Dynamics to Efficiently Sample from Probability Distributions
• Deep Active Inference for Partially Observable MDPs
• On dissipative symplectic integration with applications to gradient-based optimization
• Improving dynamical lattice QCD simulations through integrator tuning using Poisson brackets and a force-gradient integrator
• Adaptivity and Computation-Statistics Tradeoffs for Kernel and Distance based High Dimensional Two Sample Testing
• Bayesian Mechanics for Stationary Processes
• Palindromic 3-stage splitting integrators, a roadmap
• Conformal Symplectic and Relativistic Optimization
• Introducing an Explicit Symplectic Integration Scheme for Riemannian Manifold Hamiltonian Monte Carlo
• Metrizing Weak Convergence with Maximum Mean Discrepancies
• Gradient flows and proximal splitting methods: A unified view on accelerated and stochastic optimization
• Multi-stage splitting integrators for sampling with modified Hamiltonian Monte Carlo methods
• Minimum Stein Discrepancy Estimators
• The reproducing Stein kernel approach for post-hoc corrected sampling
• Momentum Improves Optimization on Riemannian Manifolds
• Probabilistic morphisms and Bayesian nonparametrics
• Optimal friction matrix for underdamped Langevin sampling
• Multiple projection MCMC algorithms on submanifolds
• Measuring the Irreversibility of Numerical Schemes for Reversible Stochastic Differential Equations
• Optimization on manifolds: A symplectic approach
• Control as Hybrid Inference
• Geometric integration of non-autonomous Hamiltonian problems

Cited by in corpus (2)

• On Bayesian Mechanics: A Physics of and by Beliefs
• A Worked Example of the Bayesian Mechanics of Classical Objects