Policy Guided Monte Carlo: Reinforcement Learning Markov Chain Dynamics

Abstract

We introduce \textit{Policy Guided Monte Carlo} (PGMC), a computational framework using reinforcement learning to improve Markov chain Monte Carlo (MCMC) sampling. The methodology is generally applicable, unbiased and opens up a new path to automated discovery of efficient MCMC samplers. After developing a general theory, we demonstrate some of PGMC's prospects on an Ising model on the kagome lattice, including when the model is in its computationally challenging kagome spin ice regime. Here, we show that PGMC is able to automatically machine learn efficient MCMC updates without a priori knowledge of the physics at hand.

arXiv comment18 pages, 13 figures. Submitted to Physical Review E

References in corpus (23)

• Magnetic Monopoles in Spin Ice
• Quantum Monte Carlo with Directed Loops
• A Short History of Markov Chain Monte Carlo: Subjective Recollections from Incomplete Data
• Self-Learning Monte Carlo Method
• Accelerate Monte Carlo Simulations with Restricted Boltzmann Machines
• Rejection-free Monte-Carlo sampling for general potentials
• Self-learning Kinetic Monte-Carlo method: application to Cu(111)
• Monte Carlo simulation of ice models
• Self-Learning Monte Carlo Method in Fermion Systems
• Self-Learning Determinantal Quantum Monte Carlo Method
• Self-Learning Monte Carlo Method: Continuous-Time Algorithm
• Symmetry Enforced Self-Learning Monte Carlo Method Applied to the Holstein Model
• Adaptive Gibbs samplers and related MCMC methods
• Itinerant quantum critical point with frustration and non-Fermi-liquid
• Directed geometrical worm algorithm applied to the quantum rotor model
• Can Boltzmann Machines Discover Cluster Updates ?
• Recommender Engine for Continuous Time Quantum Monte Carlo Methods
• Classical Topological Order in Kagome Ice
• Cluster algorithms for frustrated two dimensional Ising antiferromagnets via dual worm constructions
• Loop algorithm for classical Heisenberg models with spin-ice type degeneracy
• Fast Langevin based algorithm for MCMC in high dimensions
• A cluster algorithm for Monte Carlo simulations of spin ice
• A generalized Monte Carlo loop algorithm for frustrated Ising models

Cited by in corpus (12)

• Restricted Boltzmann Machines for Quantum States with Nonabelian or Anyonic Symmetries
• Machine Learning for Condensed Matter Physics
• A reinforcement learning approach to rare trajectory sampling
• Revealing Fermionic Quantum Criticality from New Monte Carlo Techniques
• Roadmap on machine learning glassy dynamics
• Evolutionary reinforcement learning of dynamical large deviations
• Generation of ice states through deep reinforcement learning
• Deep recurrent networks predicting the gap evolution in adiabatic quantum computing
• Policy-guided Monte Carlo on general state spaces: Application to glass-forming mixtures
• Variational Autoencoder Analysis of Ising Model Statistical Distributions and Phase Transitions
• Self-Tuning Hamiltonian Monte Carlo for Accelerated Sampling
• Computational Methods toward Ultrastable Glasses