Bayesian Renormalization

Abstract

In this note we present a fully information theoretic approach to renormalization inspired by Bayesian statistical inference, which we refer to as Bayesian Renormalization. The main insight of Bayesian Renormalization is that the Fisher metric defines a correlation length that plays the role of an emergent RG scale quantifying the distinguishability between nearby points in the space of probability distributions. This RG scale can be interpreted as a proxy for the maximum number of unique observations that can be made about a given system during a statistical inference experiment. The role of the Bayesian Renormalization scheme is subsequently to prepare an effective model for a given system up to a precision which is bounded by the aforementioned scale. In applications of Bayesian Renormalization to physical systems, the emergent information theoretic scale is naturally identified with the maximum energy that can be probed by current experimental apparatus, and thus Bayesian Renormalization coincides with ordinary renormalization. However, Bayesian Renormalization is sufficiently general to apply even in circumstances in which an immediate physical scale is absent, and thus provides an ideal approach to renormalization in data science contexts. To this end, we provide insight into how the Bayesian Renormalization scheme relates to existing methods for data compression and data generation such as the information bottleneck and the diffusion learning paradigm. We conclude by designing an explicit form of Bayesian Renormalization inspired by Wilson's momentum shell renormalization scheme in Quantum Field Theory. We apply this Bayesian Renormalization scheme to a simple Neural Network and verify the sense in which it organizes the parameters of the model according to a hierarchy of information theoretic importance.

20 pages, no figures. V2: Citation format fixed, references added. V3: Journal accepted version, new Section 4 includes fully worked implementation of Bayesian Renormalization to a Neural Network, 30 pages, 2 tables, 5 figures

References in corpus (13)

• Hierarchical Text-Conditional Image Generation with CLIP Latents
• Score-Based Generative Modeling through Stochastic Differential Equations
• Zero-Shot Text-to-Image Generation
• Seeing a c-theorem with holography
• A c-theorem for the entanglement entropy
• Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise
• Manifestly Gauge Invariant QCD
• The Exact Renormalization Group and Higher-spin Holography
• The holographic map as a conditional expectation
• The Vacuum Energy Density and Gravitational Entropy
• Renormalization group and diffusion equation
• Large $N$ von Neumann algebras and the renormalization of Newton's constant
• Exact Renormalization of Wave Functionals yields Continuous MERA

Cited by in corpus (11)

• Network Renormalization
• The Inverse of Exact Renormalization Group Flows as Statistical Inference
• On learning higher-order cumulants in diffusion models
• Bayesian RG Flow in Neural Network Field Theories
• Wilsonian Renormalization of Neural Network Gaussian Processes
• NCoder -- A Quantum Field Theory approach to encoding data
• A Lindbladian for exact renormalization of density operators in QFT
• Quantum error correction realized by the renormalization group in scalar field theories
• Learning 3-Manifold Triangulations
• Journey from the Wilson exact RG towards the Wegner-Morris Fokker-Planck RG and the Carosso field-coarsening via Langevin stochastic processes
• AInstein: Numerical Einstein Metrics via Machine Learning