Mutual Information, Neural Networks and the Renormalization Group

Abstract

Physical systems differring in their microscopic details often display strikingly similar behaviour when probed at macroscopic scales. Those universal properties, largely determining their physical characteristics, are revealed by the powerful renormalization group (RG) procedure, which systematically retains "slow" degrees of freedom and integrates out the rest. However, the important degrees of freedom may be difficult to identify. Here we demonstrate a machine learning algorithm capable of identifying the relevant degrees of freedom and executing RG steps iteratively without any prior knowledge about the system. We introduce an artificial neural network based on a model-independent, information-theoretic characterization of a real-space RG procedure, performing this task. We apply the algorithm to classical statistical physics problems in one and two dimensions. We demonstrate RG flow and extract the Ising critical exponent. Our results demonstrate that machine learning techniques can extract abstract physical concepts and consequently become an integral part of theory- and model-building.

The accepted (substantially extended) version

References in corpus (3)

• Exact evolution equation for the effective potential
• Learning phase transitions by confusion
• Deep Learning the Quantum Phase Transitions in Random Electron Systems: Applications to Three Dimensions

Cited by in corpus (112)

• Machine learning and the physical sciences
• Social physics
• Discovering physical concepts with neural networks
• Quantum Machine Learning for Chemistry and Physics
• Neural Network Renormalization Group
• Machine Learning for Quantum Matter
• Identifying Quantum Phase Transitions with Adversarial Neural Networks
• Classifying Snapshots of the Doped Hubbard Model with Machine Learning
• Asymptotically unbiased estimation of physical observables with neural samplers
• Unsupervised phase discovery with deep anomaly detection
• Restricted Boltzmann Machines for Quantum States with Nonabelian or Anyonic Symmetries
• Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines
• Deep Learning and AdS/CFT
• Machine learning active-nematic hydrodynamics
• Machine Learning for Condensed Matter Physics
• AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity
• Scale-invariant Feature Extraction of Neural Network and Renormalization Group Flow
• Deep Learning the Ising Model Near Criticality
• Precise measurement of quantum observables with neural-network estimators
• Super-resolving the Ising model with convolutional neural networks
• Learning entropy production via neural networks
• Learning phase transitions from dynamics
• Smallest Neural Network to Learn the Ising Criticality
• Efficient Modelling of Trivializing Maps for Lattice ${\varphi }^4$ Theory Using Normalizing Flows: A First Look at Scalability
• Symbolic Pregression: Discovering Physical Laws from Distorted Video
• The Presence and Absence of Barren Plateaus in Tensor-network Based Machine Learning
• Machine Learning Holographic Mapping by Neural Network Renormalization Group
• Drawing Phase Diagrams of Random Quantum Systems by Deep Learning the Wave Functions
• Thermodynamics and Feature Extraction by Machine Learning
• Divergence of predictive model output as indication of phase transitions
• Learning the Ising Model with Generative Neural Networks
• Interpretable machine learning for inferring the phase boundaries in a nonequilibrium system
• Neural network setups for a precise detection of the many-body localization transition: finite-size scaling and limitations
• Soft Metamaterials: Adaptation and Intelligence
• Nonperturbative renormalization for the neural network-QFT correspondence
• Automatic Differentiation for Second Renormalization of Tensor Networks
• Is Deep Learning a Renormalization Group Flow?
• Deep learning black hole metrics from shear viscosity
• Revealing quantum chaos with machine learning
• QuCumber: wavefunction reconstruction with neural networks
• Renormalization Group Flow as Optimal Transport
• Extracting Critical Exponent by Finite-Size Scaling with Convolutional Neural Networks
• Singular measures and information capacity of turbulent cascades
• Self-organizing maps as a method for detecting phase transitions and phase identification
• NFDLM: A Lightweight Network Flow based Deep Learning Model for DDoS Attack Detection in IoT Domains
• Approximating power of machine-learning ansatz for quantum many-body states
• Optimal Renormalization Group Transformation from Information Theory
• Opening the black box of deep learning
• Emergent Quantum Mechanics in an Introspective Machine Learning Architecture
• Tensor networks for interpretable and efficient quantum-inspired machine learning
• Neural Information Squeezer for Causal Emergence
• Learning a Local Symmetry with Neural-Networks
• Statistical physics through the lens of real-space mutual information
• Relevance in the Renormalization Group and in Information Theory
• Symmetries and phase diagrams with real-space mutual information neural estimation
• Inverse Renormalization Group based on Image Super-Resolution using Deep Convolutional Networks
• Machine learning effective models for quantum systems
• How much entanglement can be created in a closed system?
• Neural Network flows of low q-state Potts and clock Models
• Neural Monte Carlo Renormalization Group
• Bridging the information and dynamics attributes of neural activities
• RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior
• Machine learning generated configurations in presence of a conserved quantity: a cautionary tale
• 'Place-cell' emergence and learning of invariant data with restricted Boltzmann machines: breaking and dynamical restoration of continuous symmetries in the weight space
• Differentiable Programming of Isometric Tensor Networks
• Adding machine learning within Hamiltonians: Renormalization group transformations, symmetry breaking and restoration
• Three Learning Stages and Accuracy-Efficiency Tradeoff of Restricted Boltzmann Machines
• Stochastic Replica Voting Machine Prediction of Stable Cubic and Double Perovskite Materials and Binary Alloys
• Network Renormalization
• Neural networks in quantum many-body physics: a hands-on tutorial
• Signal detection in nearly continuous spectra and symmetry breaking
• Learning entanglement breakdown as a phase transition by confusion
• Unsupervised mapping of phase diagrams of 2D systems from infinite projected entangled-pair states via deep anomaly detection
• Approach to Data Science with Multiscale Information Theory
• Detection of Berezinskii-Kosterlitz-Thouless transition via Generative Adversarial Networks
• Machine Learning the Operator Content of the Critical Self-Dual Ising-Higgs Gauge Model
• Machine learning algorithms based on generalized Gibbs ensembles
• Detecting composite orders in layered models via machine learning
• Probing Criticality in Quantum Spin Chains with Neural Networks
• Short sighted deep learning
• Renormalized Mutual Information for Artificial Scientific Discovery
• Superconducting fluctuations and charge-4$e$ plaquette state at strong coupling
• Exploring explicit coarse-grained structure in artificial neural networks
• Compression theory for inhomogeneous systems
• Tensor networks and efficient descriptions of classical data
• A sampling-guided unsupervised learning method to capture percolation in complex networks
• Localization renormalization and quantum Hall systems
• Categorical Representation Learning and RG flow operators for algorithmic classifiers
• Mimicking complex dislocation dynamics by interaction networks
• The Autodidactic Universe
• Renormalization-group-inspired neural networks for computing topological invariants
• Boundary between noise and information applied to filtering neural network weight matrices
• Bayesian RG Flow in Neural Network Field Theories
• Logical Reasoning for Revealing the Critical Temperature through Deep Learning of Configuration Ensemble of Statistical Systems
• Scanning-probe and information-concealing machine learning intermediate hexatic phase and critical scaling of solid-hexatic phase transition in deformable particles
• Boosting on the shoulders of giants in quantum device calibration
• Wilsonian Renormalization of Neural Network Gaussian Processes
• Functional renormalization group for signal detection and stochastic ergodicity breaking
• Surveying the space of descriptions of a composite system with machine learning
• Functional Renormalization Group Approach for Signal Detection
• Self-regularizing restricted Boltzmann machines
• A Two-Phase Perspective on Deep Learning Dynamics
• Predicting the Dynamics of Complex System via Multiscale Diffusion Autoencoder
• Dreaming up scale invariance via inverse renormalization group
• The many faces of deep learning
• Minimizing couplings in renormalization by preserving short-range mutual information
• Renormalization group for spectral collapse in random matrices with power-law variance profiles
• Application of deep neural networks for computing the renormalization group flow of the two-dimensional phi^4 field theory
• Learning disentangled representation for classical models
• Gaussian Information Bottleneck and the Non-Perturbative Renormalization Group
• Maximum Multiscale Entropy and Neural Network Regularization
• The Expressivity and Training of Deep Neural Networks: toward the Edge of Chaos?