Policy-guided Monte Carlo on general state spaces: Application to glass-forming mixtures

Abstract

Policy-guided Monte Carlo is an adaptive method to simulate classical interacting systems. It adjusts the proposal distribution of the Metropolis-Hastings algorithm to maximize the sampling efficiency, using a formalism inspired by reinforcement learning. In this work, we first extend the policy-guided method to deal with a general state space, comprising, for instance, both discrete and continuous degrees of freedom, and then apply it to a few paradigmatic models of glass-forming mixtures. We assess the efficiency of a set of physically inspired moves whose proposal distributions are optimized through on-policy learning. Compared to conventional Monte Carlo methods, the optimized proposals are two orders of magnitude faster for an additive soft sphere mixture but yield a much more limited speed-up for the well-studied Kob-Andersen model. We discuss the current limitations of the method and suggest possible ways to improve it.

arXiv comment14 pages, 11 figures. Data relevant to this work are available at https://doi.org/10.5281/zenodo.11396665

References in corpus (26)

• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Theoretical perspective on the glass transition and amorphous materials
• Testing Mode-Coupling Theory for a Supercooled Binary Lennard-Jones Mixture I: The van Hove Correlation Function
• Models and algorithms for the next generation of glass transition studies
• On the ergodicity properties of some adaptive MCMC algorithms
• Corresponding States of Structural Glass Formers
• Solving Statistical Mechanics Using Variational Autoregressive Networks
• Fast Monte Carlo algorithm for supercooled soft spheres
• The Monte-Carlo dynamics of a binary Lennard-Jones glass-forming mixture
• Rejection-free Geometric Cluster Algorithm for Complex Fluids
• Nonequilibrium candidate Monte Carlo: A new tool for efficient equilibrium simulation
• Modern computational studies of the glass transition
• Zero-temperature glass transition in two dimensions
• Transformer variational wave functions for frustrated quantum spin systems
• Ultrastable metallic glasses in silico
• Adaptive Gibbs samplers and related MCMC methods
• Dynamic and thermodynamic crossover scenarios in the Kob-Andersen mixture: Insights from multi-CPU and multi-GPU simulations
• Machine-learning-assisted Monte Carlo fails at sampling computationally hard problems
• Policy Guided Monte Carlo: Reinforcement Learning Markov Chain Dynamics
• Irreversible Monte Carlo algorithms for hard disk glasses: from event-chain to collective swaps
• Generation of ice states through deep reinforcement learning
• Efficient Rare Event Sampling with Unsupervised Normalising Flows
• Normalizing flows as an enhanced sampling method for atomistic supercooled liquids
• A New Monte Carlo Method and Its Implications for Generalized Cluster Algorithms
• Wavelet Conditional Renormalization Group
• Self-Tuning Hamiltonian Monte Carlo for Accelerated Sampling

Cited by in corpus (6)

• Roadmap on machine learning glassy dynamics
• Performance of machine-learning-assisted Monte Carlo in sampling from simple statistical physics models
• Irreversible swap algorithms for soft sphere glasses
• Computational Methods toward Ultrastable Glasses
• Nearest-Neighbours Neural Network architecture for efficient sampling of statistical physics models
• Energy based diffusion generator for efficient sampling of Boltzmann distributions