Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning

Abstract

Using a model heat engine, we show that neural network-based reinforcement learning can identify thermodynamic trajectories of maximal efficiency. We consider both gradient and gradient-free reinforcement learning. We use an evolutionary learning algorithm to evolve a population of neural networks, subject to a directive to maximize the efficiency of a trajectory composed of a set of elementary thermodynamic processes; the resulting networks learn to carry out the maximally-efficient Carnot, Stirling, or Otto cycles. When given an additional irreversible process, this evolutionary scheme learns a previously unknown thermodynamic cycle. Gradient-based reinforcement learning is able to learn the Stirling cycle, whereas an evolutionary approach achieves the optimal Carnot cycle. Our results show how the reinforcement learning strategies developed for game playing can be applied to solve physical problems conditioned upon path-extensive order parameters.

11 pages, 5 figures

References in corpus (19)

• Stochastic thermodynamics, fluctuation theorems, and molecular machines
• Entropy production along a stochastic trajectory and an integral fluctuation theorem
• Deep Reinforcement Learning for De-Novo Drug Design
• The Arcade Learning Environment: An Evaluation Platform for General Agents
• Optimization of Molecules via Deep Reinforcement Learning
• First-order dynamical phase transition in models of glasses: an approach based on ensembles of histories
• Dynamic Pathways for Viral Capsid Assembly
• Thermodynamic formalism for systems with Markov dynamics
• The Statistical Mechanics of Dynamic Pathways to Self-assembly
• Reversible self-assembly of patchy particles into monodisperse icosahedral clusters
• ViZDoom Competitions: Playing Doom from Pixels
• Quantum error correction for the toric code using deep reinforcement learning
• Biomimetic Ultra-Broadband Perfect Absorbers Optimised with Reinforcement Learning
• Biochemical machines for the interconversion of mutual information and work
• Allocating dissipation across a molecular machine cycle to maximize flux
• Large deviation function for the entropy production: Optimal trajectory and role of fluctuations
• Current fluctuations in systems with diffusive dynamics, in and out of equilibrium
• Controlled Online Optimization Learning (COOL): Finding the ground state of spin Hamiltonians with reinforcement learning
• Fluctuations in interacting particle systems with memory

Cited by in corpus (2)

• Optimal control of quantum thermal machines using machine learning
• Estimating Reaction Barriers with Deep Reinforcement Learning