Macroscopic Analysis of Vector Approximate Message Passing in a Model Mismatch Setting

Abstract

Vector approximate message passing (VAMP) is an efficient approximate inference algorithm used for generalized linear models. Although VAMP exhibits excellent performance, particularly when measurement matrices are sampled from rotationally invariant ensembles, existing convergence and performance analyses have been limited mostly to cases in which the correct posterior distribution is available. Here, we extend the analyses for cases in which the correct posterior distribution is not used in the inference stage. We derive state evolution equations, which macroscopically describe the dynamics of VAMP, and show that their fixed point is consistent with the replica symmetric solution obtained by the replica method of statistical mechanics. We also show that the fixed point of VAMP can exhibit a microscopic instability, the critical condition of which agrees with that for breaking the replica symmetry. The results of numerical experiments support our findings.

arXiv comment22pages, 2 figures

References in corpus (18)

• Expectation Propagation for approximate Bayesian inference
• The dynamics of message passing on dense graphs, with applications to compressed sensing
• The structural glass transition and the entropy of the metastable states
• Statistical physics of inference: Thresholds and algorithms
• Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models
• Turbo Compressed Sensing with Partial DFT Sensing Matrix
• The Nishimori line and Bayesian Statistics
• Phase transitions and sample complexity in Bayes-optimal matrix factorization
• Tractable approximations for probabilistic models: The adaptive TAP mean field approach
• A Theory of Solving TAP Equations for Ising Models with General Invariant Random Matrices
• The Mutual Information in Random Linear Estimation Beyond i.i.d. Matrices
• Inference from correlated patterns: a unified theory for perceptron learning and linear vector channels
• Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (or : How to Prove Kabashima's Replica Formula)
• Perceptron capacity revisited: classification ability for correlated patterns
• Memory-free dynamics for the TAP equations of Ising models with arbitrary rotation invariant ensembles of random coupling matrices
• Phase retrieval in high dimensions: Statistical and computational phase transitions
• Learning of correlated patterns by simple perceptrons
• A Dynamical Mean-Field Theory for Learning in Restricted Boltzmann Machines

Cited by in corpus (4)

• Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (or : How to Prove Kabashima's Replica Formula)
• Statistical physics of deep learning: Optimal learning of a multi-layer perceptron near interpolation
• Statistical mechanics of extensive-width Bayesian neural networks near interpolation
• Optimal generalisation and learning transition in extensive-width shallow neural networks near interpolation