Neural Network Renormalization Group

Abstract

We present a variational renormalization group (RG) approach using a deep generative model based on normalizing flows. The model performs hierarchical change-of-variables transformations from the physical space to a latent space with reduced mutual information. Conversely, the neural net directly maps independent Gaussian noises to physical configurations following the inverse RG flow. The model has an exact and tractable likelihood, which allows unbiased training and direct access to the renormalized energy function of the latent variables. To train the model, we employ probability density distillation for the bare energy function of the physical problem, in which the training loss provides a variational upper bound of the physical free energy. We demonstrate practical usage of the approach by identifying mutually independent collective variables of the Ising model and performing accelerated hybrid Monte Carlo sampling in the latent space. Lastly, we comment on the connection of the present approach to the wavelet formulation of RG and the modern pursuit of information preserving RG.

arXiv commentMain text: 4.5 pages, 4 figures. Supplement: 3 pages. Github link: https://github.com/li012589/NeuralRG

References in corpus (44)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Estimating Mutual Information
• WaveNet: A Generative Model for Raw Audio
• MCMC using Hamiltonian dynamics
• Efficient classical simulation of slightly entangled quantum computations
• Pixel Recurrent Neural Networks
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• Tensor-Entanglement-Filtering Renormalization Approach and Symmetry Protected Topological Order
• A class of quantum many-body states that can be efficiently simulated
• Entanglement Renormalization and Holography
• Area laws in quantum systems: mutual information and correlations
• Tensor renormalization group approach to 2D classical lattice models
• Classical simulation of quantum many-body systems with a tree tensor network
• Coarse-graining renormalization by higher-order singular value decomposition
• Why does deep and cheap learning work so well?
• Tensor Network Renormalization
• Parallel WaveNet: Fast High-Fidelity Speech Synthesis
• MADE: Masked Autoencoder for Distribution Estimation
• Tensor-entanglement renormalization group approach to 2D quantum systems
• Second Renormalization of Tensor-Network States
• Self-Learning Monte Carlo Method
• Accelerate Monte Carlo Simulations with Restricted Boltzmann Machines
• TensorFlow Distributions
• Mutual Information, Neural Networks and the Renormalization Group
• Renormalization of tensor-network states
• Loop optimization for tensor network renormalization
• Tensor network renormalization yields the multi-scale entanglement renormalization ansatz
• Real Space Renormalization in Statistical Mechanics
• Renormalization of tensor networks using graph independent local truncations
• Exact holographic mapping and emergent space-time geometry
• PCA meets RG
• Entanglement renormalization and wavelets
• Machine Learning Spatial Geometry from Entanglement Features
• Scale-invariant Feature Extraction of Neural Network and Renormalization Group Flow
• A-NICE-MC: Adversarial Training for MCMC
• Renormalization group flows of Hamiltonians using tensor networks
• Multiresolution analysis in statistical mechanics. II. The wavelet transform as a basis for Monte Carlo simulations on lattices
• Multiresolution analysis in statistical mechanics. I. Using wavelets to calculate thermodynamic properties
• Deep learning and the renormalization group
• Real-space renormalization yields finite correlations
• Representation and design of wavelets using unitary circuits
• Simulating the Ising Model with a Deep Convolutional Generative Adversarial Network

Cited by in corpus (95)

• Machine learning and the physical sciences
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Solving Statistical Mechanics Using Variational Autoregressive Networks
• Flow-based generative models for Markov chain Monte Carlo in lattice field theory
• Equivariant flow-based sampling for lattice gauge theory
• Machine Learning for Quantum Matter
• Sampling using $SU(N)$ gauge equivariant flows
• Asymptotically unbiased estimation of physical observables with neural samplers
• Restricted Boltzmann Machines for Quantum States with Nonabelian or Anyonic Symmetries
• Developments in the Tensor Network -- from Statistical Mechanics to Quantum Entanglement
• Machine Learning for Condensed Matter Physics
• Exploring QCD matter in extreme conditions with Machine Learning
• Targeted free energy estimation via learned mappings
• Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities
• Super-resolving the Ising model with convolutional neural networks
• Lattice gauge equivariant convolutional neural networks
• Learning entropy production via neural networks
• Efficient Modelling of Trivializing Maps for Lattice ${\varphi }^4$ Theory Using Normalizing Flows: A First Look at Scalability
• Flow-based sampling for fermionic lattice field theories
• Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalization
• Machine Learning Holographic Mapping by Neural Network Renormalization Group
• Gauge-equivariant flow models for sampling in lattice field theories with pseudofermions
• Drawing Phase Diagrams of Random Quantum Systems by Deep Learning the Wave Functions
• Riemannian geometry and automatic differentiation for optimization problems of quantum physics and quantum technologies
• Stochastic normalizing flows as non-equilibrium transformations
• Learning the Ising Model with Generative Neural Networks
• Unbiased Monte Carlo Cluster Updates with Autoregressive Neural Networks
• Interpretable machine learning for inferring the phase boundaries in a nonequilibrium system
• Neural network setups for a precise detection of the many-body localization transition: finite-size scaling and limitations
• Accelerating lattice quantum Monte Carlo simulation using artificial neural networks: an application to the Holstein model
• Nonperturbative renormalization for the neural network-QFT correspondence
• Automatic Differentiation for Second Renormalization of Tensor Networks
• Deep learning black hole metrics from shear viscosity
• Advances in machine-learning-based sampling motivated by lattice quantum chromodynamics
• Ab-initio study of interacting fermions at finite temperature with neural canonical transformation
• Compressing deep neural networks by matrix product operators
• Machine-learning-assisted Monte Carlo fails at sampling computationally hard problems
• Neural Canonical Transformation with Symplectic Flows
• Deep Learning the Functional Renormalization Group
• Optimal Renormalization Group Transformation from Information Theory
• Opening the black box of deep learning
• Deep Variational Free Energy Approach to Dense Hydrogen
• Neural Information Squeezer for Causal Emergence
• Inverse Renormalization Group in Quantum Field Theory
• Fourier-Flow model generating Feynman paths
• Inverse Renormalization Group based on Image Super-Resolution using Deep Convolutional Networks
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• Free energy calculation of crystalline solids using normalizing flow
• Machine learning effective models for quantum systems
• Stochastic Normalizing Flows
• Generation of ice states through deep reinforcement learning
• Differentiable Programming of Isometric Tensor Networks
• Machine learning generated configurations in presence of a conserved quantity: a cautionary tale
• RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior
• Adding machine learning within Hamiltonians: Renormalization group transformations, symmetry breaking and restoration
• Estimating the Euclidean quantum propagator with deep generative modeling of Feynman paths
• Network Renormalization
• Neural networks in quantum many-body physics: a hands-on tutorial
• Signal detection in nearly continuous spectra and symmetry breaking
• Wavelet Conditional Renormalization Group
• Machine Learning the Operator Content of the Critical Self-Dual Ising-Higgs Gauge Model
• Learning phase transitions from regression uncertainty: A new regression-based machine learning approach for automated detection of phases of matter
• Computing Absolute Free Energy with Deep Generative Models
• Bounding free energy difference with flow matching
• Short sighted deep learning
• Network-Initialized Monte Carlo Based on Generative Neural Networks
• Self-Supervised Learning of Generative Spin-Glasses with Normalizing Flows
• Temperature-steerable flows
• Variational Optimization for Quantum Problems using Deep Generative Networks
• Compressing Neural Networks Using Tensor Networks with Exponentially Fewer Variational Parameters
• Categorical Representation Learning and RG flow operators for algorithmic classifiers
• A Probability Density Theory for Spin-Glass Systems
• Exploring explicit coarse-grained structure in artificial neural networks
• Logical Reasoning for Revealing the Critical Temperature through Deep Learning of Configuration Ensemble of Statistical Systems
• The Autodidactic Universe
• Renormalization-group-inspired neural networks for computing topological invariants
• A Mathematical Framework for Learning Probability Distributions
• Inverse renormalization group of spin glasses
• Learning Non-linear Wavelet Transformation via Normalizing Flow
• Functional renormalization group for signal detection and stochastic ergodicity breaking
• Deep variational free energy prediction of dense hydrogen solid at 1200K
• Reinforcement Fine-Tuning for Materials Design
• An Enquiry on similarities between Renormalization Group and Auto-Encoders using Transfer Learning
• Deep generative modelling of canonical ensemble with differentiable thermal properties
• Functional Renormalization Group Approach for Signal Detection
• Energy based diffusion generator for efficient sampling of Boltzmann distributions
• Feature extraction of machine learning and phase transition point of Ising model
• Variationally Inferred Sampling Through a Refined Bound for Probabilistic Programs
• Weakly-supervised learning on Schrodinger equation
• Contributions to Large Scale Bayesian Inference and Adversarial Machine Learning
• Global sampling of Feynman's diagrams through Normalizing Flow
• Sampling metastable systems using collective variables and Jarzynski-Crooks paths
• Learning disentangled representation for classical models
• Applications of flow models to the generation of correlated lattice QCD ensembles
• Application of deep neural networks for computing the renormalization group flow of the two-dimensional phi^4 field theory