Some observations on the ambivalent role of symmetries in Bayesian inference problems

Abstract

We collect in this note some observations on the role of symmetries in Bayesian inference problems, that can be useful or detrimental depending on the way they act on the signal and on the observations. We emphasize in particular the need to gauge away unobservable invariances in the definition of a distance between a signal and its estimator, and the consequences this implies for the statistical mechanics treatment of such models, taking as a motivating example the extensive rank matrix factorization problem.

14 pages

References in corpus (18)

• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising
• Statistical limits of dictionary learning: random matrix theory and the spectral replica method
• Learning with invariances in random features and kernel models
• Matrix factorization with neural networks
• Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances
• Singular Vectors of Sums of Rectangular Random Matrices and Optimal Estimators of High-Rank Signals: The Extensive Spike Model
• Equivalence of Approximate Message Passing and Low-Degree Polynomials in Rank-One Matrix Estimation
• Optimal denoising of rotationally invariant rectangular matrices
• A multiscale cavity method for sublinear-rank symmetric matrix factorization
• Rectangular Rotational Invariant Estimator for General Additive Noise Matrices
• The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents
• Tensor cumulants for statistical inference on invariant distributions
• Complexity of Robust Orbit Problems for Torus Actions and the abc-conjecture
• Bayesian Extensive-Rank Matrix Factorization with Rotational Invariant Priors
• On the phase diagram of extensive-rank symmetric matrix denoising beyond rotational invariance
• The Decimation Scheme for Symmetric Matrix Factorization
• Multivariate Priors and the Linearity of Optimal Bayesian Estimators under Gaussian Noise