Analysis of Bayesian Inference Algorithms by the Dynamical Functional Approach

Abstract

We analyze the dynamics of an algorithm for approximate inference with large Gaussian latent variable models in a student-teacher scenario. To model nontrivial dependencies between the latent variables, we assume random covariance matrices drawn from rotation invariant ensembles. For the case of perfect data-model matching, the knowledge of static order parameters derived from the replica method allows us to obtain efficient algorithmic updates in terms of matrix-vector multiplications with a fixed matrix. Using the dynamical functional approach, we obtain an exact effective stochastic process in the thermodynamic limit for a single node. From this, we obtain closed-form expressions for the rate of the convergence. Analytical results are excellent agreement with simulations of single instances of large models.

arXiv comment25 pages, 2 figures

References in corpus (15)

• Message Passing Algorithms for Compressed Sensing
• The dynamics of message passing on dense graphs, with applications to compressed sensing
• Universality in polytope phase transitions and message passing algorithms
• Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models
• Introduction to Random Matrices - Theory and Practice
• Statistical physics-based reconstruction in compressed sensing
• Mean-Field Equations for Spin Models with Orthogonal Interaction Matrices
• Vector Precoding for Wireless MIMO Systems: A Replica Analysis
• A Theory of Solving TAP Equations for Ising Models with General Invariant Random Matrices
• Inference from correlated patterns: a unified theory for perceptron learning and linear vector channels
• Memory-free dynamics for the TAP equations of Ising models with arbitrary rotation invariant ensembles of random coupling matrices
• New scaling of Itzykson-Zuber integrals
• A Deterministic and Generalized Framework for Unsupervised Learning with Restricted Boltzmann Machines
• Integration of invariant matrices and application to statistics
• Statistical mechanical analysis of the linear vector channel in digital communication

Cited by in corpus (2)

• A Dynamical Mean-Field Theory for Learning in Restricted Boltzmann Machines
• Exact solution to the random sequential dynamics of a message passing algorithm