Diagrammatics of free energies with fixed variance for high-dimensional data

Abstract

Systems with many interacting stochastic constituents are fully characterized by their free energy. Computing this quantity is therefore the objective of various approaches, notably perturbative expansions, which are applied in problems ranging from high-dimensional statistics to complex systems. However, a lot of these techniques are complicated to apply in practice because they lack a sufficient organization of the terms of the perturbative series. In this manuscript, we tackle this problem by using Feynman diagrams, extending a framework introduced earlier to the case of free energies at fixed variances. This diagrammatics do not require the theory to expand around to be Gaussian, which allows its application to the free energy of a spin system studied to derive message-passing algorithms by Maillard et al. 2019. We complete their perturbative derivation of the free energy in the thermodynamic limit. Furthermore, we derive resummations to estimate the entropies of poorly sampled systems requiring only limited statistics and we revisit earlier approaches to compute the free energy of the Ising model, revealing new insights due to the extension of our framework to the free energy at fixed variances. We expect our approach to be particularly useful for problems of high-dimensional statistics, like matrix factorization, and the study of complex networks.

arXiv commentEquivalent to published version. 24 pages, 3 figures. Comments welcome!

References in corpus (15)

• Networks beyond pairwise interactions: structure and dynamics
• Statistical physics of inference: Thresholds and algorithms
• Small-correlation expansions for the inverse Ising problem
• Phase transitions and sample complexity in Bayes-optimal matrix factorization
• Mean-Field Equations for Spin Models with Orthogonal Interaction Matrices
• Eigenstate Thermalization Hypothesis and Free Probability
• High-temperature Expansions and Message Passing Algorithms
• Mean-field theory for a spin-glass model of neural networks: TAP free energy and paramagnetic to spin-glass transition
• A robust generalization of the Legendre transform for QFT
• Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising
• Expansion of the effective action around non-Gaussian theories
• Resummed mean-field inference for strongly coupled data
• On the Structure of QFT in the Particle Picture of the Path Integral Formulation
• Diagrammatics for the Inverse Problem in Spin Systems and Simple Liquids
• Diagrammatic expansion for the mutual-information rate in the realm of limited statistics