Generation of ice states through deep reinforcement learning

Abstract

We present a deep reinforcement learning framework where a machine agent is trained to search for a policy to generate a ground state for the square ice model by exploring the physical environment. After training, the agent is capable of proposing a sequence of local moves to achieve the goal. Analysis of the trained policy and the state value function indicates that the ice rule and loop-closing condition are learned without prior knowledge. We test the trained policy as a sampler in the Markov chain Monte Carlo and benchmark against the baseline loop algorithm. This framework can be generalized to other models with topological constraints where generation of constraint-preserving states is difficult.

arXiv comment10 pages, 13 figures, 2 tables. Published version

References in corpus (30)

• Generative Adversarial Networks
• Spin Ice State in Frustrated Magnetic Pyrochlore Materials
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• Density estimation using Real NVP
• StarCraft II: A New Challenge for Reinforcement Learning
• Emergence of Locomotion Behaviours in Rich Environments
• OpenAI Gym
• Active learning machine learns to create new quantum experiments
• Learning Thermodynamics with Boltzmann Machines
• Self-Learning Monte Carlo Method
• Breakdown of a topological phase: Quantum phase transition in a loop gas model with tension
• Accelerate Monte Carlo Simulations with Restricted Boltzmann Machines
• Reinforcement Learning with Neural Networks for Quantum Feedback
• Topological multicritical point in the Toric Code and 3D gauge Higgs Models
• Markov Chain Monte Carlo Method without Detailed Balance
• An exact mapping between the Variational Renormalization Group and Deep Learning
• Neural Network Renormalization Group
• Monte Carlo simulation of ice models
• Regressive and generative neural networks for scalar field theory
• Neural Belief-Propagation Decoders for Quantum Error-Correcting Codes
• Self-Learning Monte Carlo Method in Fermion Systems
• Self-Learning Determinantal Quantum Monte Carlo Method
• Self-Learning Monte Carlo Method: Continuous-Time Algorithm
• Quantum Spin Liquid with Even Ising Gauge Field Structure on Kagome Lattice
• Order-by-disorder and quantum Coulomb phase in quantum square ice
• Magnetic three states of matter: A quantum Monte Carlo study of spin liquids
• Policy Guided Monte Carlo: Reinforcement Learning Markov Chain Dynamics
• Simulating the Ising Model with a Deep Convolutional Generative Adversarial Network
• Tunneling-induced restoration of classical degeneracy in quantum kagome ice

Cited by in corpus (10)

• Machine Learning for Quantum Matter
• When does reinforcement learning stand out in quantum control? A comparative study on state preparation
• Unsupervised identification of topological order using predictive models
• Drawing Phase Diagrams of Random Quantum Systems by Deep Learning the Wave Functions
• Extending machine learning classification capabilities with histogram reweighting
• Generative models for sampling and phase transition indication in spin systems
• Neural networks in quantum many-body physics: a hands-on tutorial
• Policy-guided Monte Carlo on general state spaces: Application to glass-forming mixtures
• Unsupervised Machine Learning of Quenched Gauge Symmetries: A Proof-of-Concept Demonstration
• Human-machine collaboration: ordering mechanism of rank-2 spin liquid on breathing pyrochlore lattice