Relevance in the Renormalization Group and in Information Theory

Abstract

The analysis of complex physical systems hinges on the ability to extract the relevant degrees of freedom from among the many others. Though much hope is placed in machine learning, it also brings challenges, chief of which is interpretability. It is often unclear what relation, if any, the architecture- and training-dependent learned "relevant" features bear to standard objects of physical theory. Here we report on theoretical results which may help to systematically address this issue: we establish equivalence between the information-theoretic notion of relevance defined in the Information Bottleneck (IB) formalism of compression theory, and the field-theoretic relevance of the Renormalization Group. We show analytically that for statistical physical systems described by a field theory the "relevant" degrees of freedom found using IB compression indeed correspond to operators with the lowest scaling dimensions. We confirm our field theoretic predictions numerically. We study dependence of the IB solutions on the physical symmetries of the data. Our findings provide a dictionary connecting two distinct theoretical toolboxes, and an example of constructively incorporating physical interpretability in applications of deep learning in physics.

References in corpus (16)

• Machine learning and the physical sciences
• The information bottleneck method
• Interpretable machine learning: definitions, methods, and applications
• "Deconfined" quantum critical points
• The Conformal Bootstrap: Theory, Numerical Techniques, and Applications
• Parameter Space Compression Underlies Emergent Theories and Predictive Models
• Mutual Information, Neural Networks and the Renormalization Group
• Machine learning meets quantum physics
• A c-theorem for the entanglement entropy
• Quantum information and physics: some future directions
• Field Theory Entropy, the $H$-theorem and the Renormalization Group
• Information geometric approach to the renormalisation group
• Machine-learning Iterative Calculation of Entropy for Physical Systems
• The renormalisation group via statistical inference
• Optimal Renormalization Group Transformation from Information Theory
• Coarse-grained distinguishability of field interactions

Cited by in corpus (11)

• Statistical physics through the lens of real-space mutual information
• Bayesian Renormalization
• Symmetries and phase diagrams with real-space mutual information neural estimation
• Network Renormalization
• Machine Learning the Operator Content of the Critical Self-Dual Ising-Higgs Gauge Model
• Compression theory for inhomogeneous systems
• Critical Slowing Down Near Topological Transitions in Rate-Distortion Problems
• Bayesian RG Flow in Neural Network Field Theories
• Intrinsic regularization effect in Bayesian nonlinear regression scaled by observed data
• Dreaming up scale invariance via inverse renormalization group
• Gaussian Information Bottleneck and the Non-Perturbative Renormalization Group