Policy-guided Monte Carlo on general state spaces: Application to glass-forming mixtures

Abstract

Policy-guided Monte Carlo is an adaptive method to simulate classical interacting systems. It adjusts the proposal distribution of the Metropolis-Hastings algorithm to maximize the sampling efficiency, using a formalism inspired by reinforcement learning. In this work, we first extend the policy-guided method to deal with a general state space, comprising, for instance, both discrete and continuous degrees of freedom, and then apply it to a few paradigmatic models of glass-forming mixtures. We assess the efficiency of a set of physically inspired moves whose proposal distributions are optimized through on-policy learning. Compared to conventional Monte Carlo methods, the optimized proposals are two orders of magnitude faster for an additive soft sphere mixture but yield a much more limited speed-up for the well-studied Kob-Andersen model. We discuss the current limitations of the method and suggest possible ways to improve it.

14 pages, 11 figures. Data relevant to this work are available at https://doi.org/10.5281/zenodo.11396665

References in corpus (11)

• Theoretical perspective on the glass transition and amorphous materials
• Modern computational studies of the glass transition
• Transformer variational wave functions for frustrated quantum spin systems
• Machine-learning-assisted Monte Carlo fails at sampling computationally hard problems
• Irreversible Monte Carlo algorithms for hard disk glasses: from event-chain to collective swaps
• Generation of ice states through deep reinforcement learning
• Efficient Rare Event Sampling with Unsupervised Normalising Flows
• Normalizing flows as an enhanced sampling method for atomistic supercooled liquids
• A New Monte Carlo Method and Its Implications for Generalized Cluster Algorithms
• Wavelet Conditional Renormalization Group
• Self-Tuning Hamiltonian Monte Carlo for Accelerated Sampling

Cited by in corpus (6)

• Roadmap on machine learning glassy dynamics
• Performance of machine-learning-assisted Monte Carlo in sampling from simple statistical physics models
• Irreversible swap algorithms for soft sphere glasses
• Computational Methods toward Ultrastable Glasses
• Nearest-Neighbours Neural Network architecture for efficient sampling of statistical physics models
• Energy based diffusion generator for efficient sampling of Boltzmann distributions