Control of chaotic systems by Deep Reinforcement Learning

Abstract

Deep Reinforcement Learning (DRL) is applied to control a nonlinear, chaotic system governed by the one-dimensional Kuramoto-Sivashinsky (KS) equation. DRL uses reinforcement learning principles for the determination of optimal control solutions and deep Neural Networks for approximating the value function and the control policy. Recent applications have shown that DRL may achieve superhuman performance in complex cognitive tasks. In this work, we show that using restricted, localized actuations, partial knowledge of the state based on limited sensor measurements, and model-free DRL controllers, it is possible to stabilize the dynamics of the KS system around its unstable fixed solutions, here considered as target states. The robustness of the controllers is tested by considering several trajectories in the phase-space emanating from different initial conditions; we show that the DRL is always capable of driving and stabilizing the dynamics around the target states. The complexity of the KS system, the possibility of defining the DRL control policies by solely relying on the local measurements of the system, and their efficiency in controlling its nonlinear dynamics pave the way for the application of RL methods in control of complex fluid systems such as turbulent boundary layers, turbulent mixers or multiphase flows.

References in corpus (7)

• Artificial Neural Networks trained through Deep Reinforcement Learning discover control strategies for active flow control
• An Introduction to Adjoint Problems
• Closed-loop separation control using machine learning
• Reduced order models for control of fluids using the Eigensystem Realization Algorithm
• On the state space geometry of the Kuramoto-Sivashinsky flow in a periodic domain
• A statistical learning strategy for closed-loop control of fluid flows
• Edge states as mediators of bypass transition in boundary-layer flows

Cited by in corpus (19)

• Accelerating Deep Reinforcement Learning strategies of Flow Control through a multi-environment approach
• Applying deep reinforcement learning to active flow control in turbulent conditions
• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• DRLinFluids -- An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM
• Comparative analysis of machine learning methods for active flow control
• Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need
• Deep reinforcement learning for large-eddy simulation modeling in wall-bounded turbulence
• Neural Ordinary Differential Equation Control of Dynamics on Graphs
• Reinforcement-learning-based actuator selection method for active flow control
• Multi-fidelity reinforcement learning framework for shape optimization
• Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning
• Machine learning and control engineering: The model-free case
• Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order models
• Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models
• Deep reinforcement learning for the management of the wall regeneration cycle in wall-bounded turbulent flows
• Suppressing Modulation Instability with Reinforcement Learning
• Data-assimilated model-informed reinforcement learning
• Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control
• Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamics