Bayesian Renormalization

Abstract

In this note we present a fully information theoretic approach to renormalization inspired by Bayesian statistical inference, which we refer to as Bayesian Renormalization. The main insight of Bayesian Renormalization is that the Fisher metric defines a correlation length that plays the role of an emergent RG scale quantifying the distinguishability between nearby points in the space of probability distributions. This RG scale can be interpreted as a proxy for the maximum number of unique observations that can be made about a given system during a statistical inference experiment. The role of the Bayesian Renormalization scheme is subsequently to prepare an effective model for a given system up to a precision which is bounded by the aforementioned scale. In applications of Bayesian Renormalization to physical systems, the emergent information theoretic scale is naturally identified with the maximum energy that can be probed by current experimental apparatus, and thus Bayesian Renormalization coincides with ordinary renormalization. However, Bayesian Renormalization is sufficiently general to apply even in circumstances in which an immediate physical scale is absent, and thus provides an ideal approach to renormalization in data science contexts. To this end, we provide insight into how the Bayesian Renormalization scheme relates to existing methods for data compression and data generation such as the information bottleneck and the diffusion learning paradigm. We conclude by designing an explicit form of Bayesian Renormalization inspired by Wilson's momentum shell renormalization scheme in Quantum Field Theory. We apply this Bayesian Renormalization scheme to a simple Neural Network and verify the sense in which it organizes the parameters of the model according to a hierarchy of information theoretic importance.

arXiv comment20 pages, no figures. V2: Citation format fixed, references added. V3: Journal accepted version, new Section 4 includes fully worked implementation of Bayesian Renormalization to a Neural Network, 30 pages, 2 tables, 5 figures

References in corpus (52)

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• The information bottleneck method
• Hierarchical Text-Conditional Image Generation with CLIP Latents
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• Score-Based Generative Modeling through Stochastic Differential Equations
• Zero-Shot Text-to-Image Generation
• The Exact Renormalisation Group and Approximate Solutions
• Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users
• Entanglement Renormalization and Holography
• Philosophy and the practice of Bayesian statistics
• Exact Renormalization Group Equations. An Introductory Review
• Seeing a c-theorem with holography
• Why does deep and cheap learning work so well?
• Tensor Network Renormalization
• Parameter Space Compression Underlies Emergent Theories and Predictive Models
• Derivative Expansion of the Exact Renormalization Group
• Fundamentals of the Exact Renormalization Group
• Variational Diffusion Models
• Identifying topological order through unsupervised machine learning
• An elementary introduction to information geometry
• Elements of the Continuous Renormalization Group
• Sloppiness, robustness, and evolvability in systems biology
• Holographic Geometry of Entanglement Renormalization in Quantum Field Theories
• Geometric Holography, the Renormalization Group and the c-Theorem
• Quantum data compression, quantum information generation, and the density-matrix renormalization group method
• Coarse--graining, fixed points, and scaling in a large population of neurons
• Markov property of the CFT vacuum and the a-theorem
• A c-theorem for the entanglement entropy
• Nonlinear Gravity from Entanglement in Conformal Field Theories
• Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise
• Canonical Energy is Quantum Fisher Information
• Exact Scheme Independence
• Modular Hamiltonian of Excited States in Conformal Field Theory
• Neural Networks and Quantum Field Theory
• Non-commutative calculus, optimal transport and functional inequalities in dissipative quantum systems
• Holographic Geometry of cMERA for Quantum Quenches and Finite Temperature
• Connecting Fisher information to bulk entanglement in holography
• Manifestly Gauge Invariant QCD
• Information geometric approach to the renormalisation group
• The Holographic Geometry of the Renormalization Group and Higher Spin Symmetries
• The Exact Renormalization Group and Higher-spin Holography
• Information geometry in quantum field theory: lessons from simple examples
• The renormalisation group via statistical inference
• Renormalization Group Flow as Optimal Transport
• Relevance in the Renormalization Group and in Information Theory
• The holographic map as a conditional expectation
• The Vacuum Energy Density and Gravitational Entropy
• Renormalization group and diffusion equation
• Statistical properties of large data sets with linear latent features
• Large $N$ von Neumann algebras and the renormalization of Newton's constant
• Exact Renormalization of Wave Functionals yields Continuous MERA

Cited by in corpus (11)

• Network Renormalization
• The Inverse of Exact Renormalization Group Flows as Statistical Inference
• On learning higher-order cumulants in diffusion models
• Bayesian RG Flow in Neural Network Field Theories
• Wilsonian Renormalization of Neural Network Gaussian Processes
• NCoder -- A Quantum Field Theory approach to encoding data
• A Lindbladian for exact renormalization of density operators in QFT
• Quantum error correction realized by the renormalization group in scalar field theories
• Learning 3-Manifold Triangulations
• Journey from the Wilson exact RG towards the Wegner-Morris Fokker-Planck RG and the Carosso field-coarsening via Langevin stochastic processes
• AInstein: Numerical Einstein Metrics via Machine Learning