Towards neural reinforcement learning for large deviations in nonequilibrium systems with memory
arXiv:2501.12333 · doi:10.1088/1742-5468/adea65
Abstract
We introduce a reinforcement learning method for a class of non-Markov systems; our approach extends the actor-critic framework given by Rose et al. [New J. Phys. 23 013013 (2021)] for obtaining scaled cumulant generating functions characterizing the fluctuations. The actor-critic is implemented using neural networks; a particular innovation in our method is the use of an additional neural policy for processing memory variables. We demonstrate results for current fluctuations in various memory-dependent models with special focus on semi-Markov systems where the dynamics is controlled by nonexponential interevent waiting time distributions.
48 pages, 16 figures. Accepted in JSTAT
References in corpus (52)
- Adam: A Method for Stochastic Optimization
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- WaveNet: A Generative Model for Raw Audio
- Brownian motors: noisy transport far from equilibrium
- The large deviation approach to statistical mechanics
- Soft Actor-Critic Algorithms and Applications
- Machine learning phases of matter
- A Gallavotti-Cohen Type Symmetry in the Large Deviation Functional for Stochastic Dynamics
- Variational Inference with Normalizing Flows
- Non equilibrium steady states: fluctuations and large deviations of the density and of the current
- Non-equilibrium statistical mechanics: From a paradigmatic model to biological transport
- First-order dynamical phase transition in models of glasses: an approach based on ensembles of histories
- Direct evaluation of large-deviation functions
- Elephants can always remember: Exact long-range memory effects in a non-Markovian random walk
- Running and tumbling with E. coli in polymeric solutions
- Learning Thermodynamics with Boltzmann Machines
- Intra-cellular transport of single-headed molecular motors KIF1A
- Probability currents as principal characteristics in the statistical mechanics of non-equilibrium steady states
- A numerical approach to large deviations in continuous-time
- Variational and optimal control representations of conditioned and driven processes
- The Exclusion Process: A paradigm for non-equilibrium behaviour
- Current fluctuations in the zero-range process with open boundaries
- The Physicist's Companion to Current Fluctuations: One-Dimensional Bulk-Driven Lattice Gases
- Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving
- Amnestically induced persistence in random walks
- A possible classification of nonequilibrium steady states
- Using matrix product states to study the dynamical large deviations of kinetically constrained models
- DMRG-study of current and activity fluctuations near non-equilibrium phase transitions
- Continuous time random walk for open systems: Fluctuation theorems and counting statistics
- Rejuvenation and Memory Effects in a Structural Glass
- Reinforcement learning approach to non-equilibrium quantum thermodynamics
- Current fluctuations in stochastic systems with long-range memory
- Dynamical fluctuations for semi-Markov processes
- Phase transitions in large deviations of reset processes
- Current reversals in a rocking ratchet: dynamical vs symmetry-breaking mechanisms
- A reinforcement learning approach to rare trajectory sampling
- Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks
- A basic introduction to large deviations: Theory, applications, simulations
- Stochastic theory of protein synthesis and polysome: ribosome profile on a single mRNA transcript
- Learning nonequilibrium control forces to characterize dynamical phase transitions
- Spatiotemporally complete condensation in a non-Poissonian exclusion process
- Run-and-tumble motion in a linear ratchet potential: Analytic solution, power extraction, and first-passage properties
- Giant leaps and long excursions: fluctuation mechanisms in systems with long-range memory
- Direct evaluation of dynamical large-deviation rate functions using a variational ansatz
- Evolutionary reinforcement learning of dynamical large deviations
- Fluctuations in interacting particle systems with memory
- A framework for the direct evaluation of large deviations in non-Markovian processes
- Analytic Formulation, Exact Solutions, and Generalizations of the Elephant and the Alzheimer Random Walks
- Large deviations of Markov chains with multiple time-scales
- Efficient Rare Event Sampling with Unsupervised Normalising Flows
- Confined run and tumble particles with non-Markovian tumbling statistics
- Context Aware Machine Learning