Short sighted deep learning

Abstract

A theory explaining how deep learning works is yet to be developed. Previous work suggests that deep learning performs a coarse graining, similar in spirit to the renormalization group (RG). This idea has been explored in the setting of a local (nearest neighbor interactions) Ising spin lattice. We extend the discussion to the setting of a long range spin lattice. Markov Chain Monte Carlo (MCMC) simulations determine both the critical temperature and scaling dimensions of the system. The model is used to train both a single RBM (restricted Boltzmann machine) network, as well as a stacked RBM network. Following earlier Ising model studies, the trained weights of a single layer RBM network define a flow of lattice models. In contrast to results for nearest neighbor Ising, the RBM flow for the long ranged model does not converge to the correct values for the spin and energy scaling dimension. Further, correlation functions between visible and hidden nodes exhibit key differences between the stacked RBM and RG flows. The stacked RBM flow appears to move towards low temperatures whereas the RG flow moves towards high temperature. This again differs from results obtained for nearest neighbor Ising.

References in corpus (20)

• Machine learning phases of matter
• The Conformal Bootstrap: Theory, Numerical Techniques, and Applications
• Discovering Phase Transitions with Unsupervised Learning
• Why does deep and cheap learning work so well?
• Machine learning quantum phases of matter beyond the fermion sign problem
• Restricted-Boltzmann-Machine Learning for Solving Strongly Correlated Quantum Systems
• Machine Learning Topological States
• Mutual Information, Neural Networks and the Renormalization Group
• An exact mapping between the Variational Renormalization Group and Deep Learning
• Neural Network Renormalization Group
• Real Space Renormalization in Statistical Mechanics
• Scale-invariant Feature Extraction of Neural Network and Renormalization Group Flow
• Deep Learning the Ising Model Near Criticality
• Smallest Neural Network to Learn the Ising Criticality
• Thermodynamics and Feature Extraction by Machine Learning
• Deep learning and the renormalization group
• Is Deep Learning a Renormalization Group Flow?
• Restricted Boltzmann Machines for the Long Range Ising Models
• Why does Deep Learning work? - A perspective from Group Theory
• Holographic Renormalization with Machine learning

Cited by in corpus (5)

• Nonperturbative renormalization for the neural network-QFT correspondence
• Why Unsupervised Deep Networks Generalize
• Functional renormalization group for signal detection and stochastic ergodicity breaking
• A Two-Phase Perspective on Deep Learning Dynamics
• Functional Renormalization Group Approach for Signal Detection