Some observations on the ambivalent role of symmetries in Bayesian inference problems

Abstract

We collect in this note some observations on the role of symmetries in Bayesian inference problems, that can be useful or detrimental depending on the way they act on the signal and on the observations. We emphasize in particular the need to gauge away unobservable invariances in the definition of a distance between a signal and its estimator, and the consequences this implies for the statistical mechanics treatment of such models, taking as a motivating example the extensive rank matrix factorization problem.

arXiv comment14 pages

References in corpus (34)

• Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• The phase transition in inhomogeneous random graphs
• Statistical physics of inference: Thresholds and algorithms
• Maximum Weight Matching via Max-Product Belief Propagation
• Mutual information for symmetric rank-one matrix estimation: A proof of the replica formula
• Phase transitions and sample complexity in Bayes-optimal matrix factorization
• Constrained Low-rank Matrix Estimation: Phase Transitions, Approximate Message Passing and Applications
• Rotational invariant estimator for general noisy matrices
• Estimation under group actions: recovering orbits from invariants
• The Computer Science and Physics of Community Detection: Landscapes, Phase Transitions, and Hardness
• Computational Barriers to Estimation from Low-Degree Polynomials
• Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference
• Statistical limits of dictionary learning: random matrix theory and the spectral replica method
• Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising
• Learning with invariances in random features and kernel models
• Statistical Mechanics of Dictionary Learning
• Stochastic optimization by message passing
• Matrix factorization with neural networks
• Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances
• Online stochastic gradient descent on non-convex losses from high-dimensional inference
• Singular Vectors of Sums of Rectangular Random Matrices and Optimal Estimators of High-Rank Signals: The Extensive Spike Model
• Equivalence of Approximate Message Passing and Low-Degree Polynomials in Rank-One Matrix Estimation
• Optimal denoising of rotationally invariant rectangular matrices
• Adversarial Satisfiability Problem
• A multiscale cavity method for sublinear-rank symmetric matrix factorization
• Rectangular Rotational Invariant Estimator for General Additive Noise Matrices
• The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents
• Tensor cumulants for statistical inference on invariant distributions
• Complexity of Robust Orbit Problems for Torus Actions and the abc-conjecture
• Bayesian Extensive-Rank Matrix Factorization with Rotational Invariant Priors
• On the phase diagram of extensive-rank symmetric matrix denoising beyond rotational invariance
• The Decimation Scheme for Symmetric Matrix Factorization
• Multivariate Priors and the Linearity of Optimal Bayesian Estimators under Gaussian Noise