Evolutionary reinforcement learning of dynamical large deviations

Abstract

We show how to calculate the likelihood of dynamical large deviations using evolutionary reinforcement learning. An agent, a stochastic model, propagates a continuous-time Monte Carlo trajectory and receives a reward conditioned upon the values of certain path-extensive quantities. Evolution produces progressively fitter agents, eventually allowing the calculation of a piece of a large-deviation rate function for a particular model and path-extensive quantity. For models with small state spaces the evolutionary process acts directly on rates, and for models with large state spaces the process acts on the weights of a neural network that parameterizes the model's rates. This approach shows how path-extensive physics problems can be considered within a framework widely used in machine learning.

References in corpus (38)

• SchNet - a deep learning architecture for molecules and materials
• The large deviation approach to statistical mechanics
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Machine learning phases of matter
• Entropy production along a stochastic trajectory and an integral fluctuation theorem
• The Arcade Learning Environment: An Evaluation Platform for General Agents
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• Dissipation bounds all steady-state current fluctuations
• OpenAI Gym
• Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
• DeepMind Control Suite
• Geometrical explanation and scaling of dynamical heterogeneities in glass forming systems
• First-order dynamical phase transition in models of glasses: an approach based on ensembles of histories
• Direct evaluation of large-deviation functions
• Universal bounds on current fluctuations
• Identifying Quantum Phase Transitions using Artificial Neural Networks on Experimental Data
• Canonical structure of dynamical fluctuations in mesoscopic nonequilibrium steady states
• Deep learning and the Schrödinger equation
• Simple bounds on fluctuations and uncertainty relations for first-passage times of counting observables
• Constructing first-principles phase diagrams of amorphous LixSi using machine-learning-assisted sampling with an evolutionary algorithm
• Variational and optimal control representations of conditioned and driven processes
• Activity driven fluctuations in living cells
• Deep Learning Approximation for Stochastic Control Problems
• Adaptive importance sampling for control and inference
• Finite-Size Scaling of a First-Order Dynamical Phase Transition: Adaptive Population Dynamics and an Effective Model
• Using matrix product states to study the dynamical large deviations of kinetically constrained models
• Exact fluctuations of nonequilibrium steady states from approximate auxiliary dynamics
• Learning to grow: control of material self-assembly using evolutionary reinforcement learning
• Adaptive sampling of large deviations
• Large deviation function for the entropy production: Optimal trajectory and role of fluctuations
• Finite size scaling of the dynamical free-energy in a kinetically constrained model
• On the Relationship Between the OpenAI Evolution Strategy and Stochastic Gradient Descent
• Current fluctuations in systems with diffusive dynamics, in and out of equilibrium
• Direct evaluation of dynamical large-deviation rate functions using a variational ansatz
• Policy Guided Monte Carlo: Reinforcement Learning Markov Chain Dynamics
• Sampling algorithms for validation of supervised learning models for Ising-like systems
• A Deep Learning Functional Estimator of Optimal Dynamics for Sampling Large Deviations
• Crawling in Rogue's dungeons with (partitioned) A3C

Cited by in corpus (18)

• Phase Diagram of Active Brownian Spheres: Crystallization and the Metastability of Motility-Induced Phase Separation
• A reinforcement learning approach to rare trajectory sampling
• Robust Prediction of Force Chains in Jammed Solids using Graph Neural Networks
• Learning nonequilibrium control forces to characterize dynamical phase transitions
• Reinforcement learning of rare diffusive dynamics
• Finite time large deviations via matrix product states
• Dynamical large deviations of two-dimensional kinetically constrained models using a neural-network state ansatz
• Phase Coexistence Implications of Violating Newton's Third Law
• Landau theory for finite-time dynamical phase transitions
• A large deviation theory perspective on nanoscale transport phenomena
• Biased ensembles of pulsating active matter
• Varied phenomenology of models displaying dynamical large-deviation singularities
• Thermodynamic uncertainty relation for Langevin dynamics by scaling time
• Entropy Regularized Reinforcement Learning Using Large Deviation Theory
• Physics-informed graph neural networks enhance scalability of variational nonequilibrium optimal control
• Inverse Design of Non-Equilibrium Steady-States: A Large Deviation Approach
• Accelerating GMRES with Deep Learning in Real-Time
• Towards neural reinforcement learning for large deviations in nonequilibrium systems with memory