A Survey on Physics Informed Reinforcement Learning: Review and Open Problems

Abstract

The inclusion of physical information in machine learning frameworks has revolutionized many application areas. This involves enhancing the learning process by incorporating physical constraints and adhering to physical laws. In this work we explore their utility for reinforcement learning applications. We present a thorough review of the literature on incorporating physics information, as known as physics priors, in reinforcement learning approaches, commonly referred to as physics-informed reinforcement learning (PIRL). We introduce a novel taxonomy with the reinforcement learning pipeline as the backbone to classify existing works, compare and contrast them, and derive crucial insights. Existing works are analyzed with regard to the representation/ form of the governing physics modeled for integration, their specific contribution to the typical reinforcement learning architecture, and their connection to the underlying reinforcement learning pipeline stages. We also identify core learning architectures and physics incorporation biases (i.e., observational, inductive and learning) of existing PIRL approaches and use them to further categorize the works for better understanding and adaptation. By providing a comprehensive perspective on the implementation of the physics-informed capability, the taxonomy presents a cohesive approach to PIRL. It identifies the areas where this approach has been applied, as well as the gaps and opportunities that exist. Additionally, the taxonomy sheds light on unresolved issues and challenges, which can guide future research. This nascent field holds great potential for enhancing reinforcement learning algorithms by increasing their physical plausibility, precision, data efficiency, and applicability in real-world scenarios.

References in corpus (17)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• End-to-End Training of Deep Visuomotor Policies
• Experimental Comparison of Open Source Visual-Inertial-Based State Estimation Algorithms in the Underwater Domain
• Hamiltonian Neural Networks
• The GOOSE Dataset for Perception in Unstructured Environments
• A Point-Cloud Deep Learning Framework for Prediction of Fluid Flow Fields on Irregular Geometries
• GelTip: A Finger-shaped Optical Tactile Sensor for Robotic Manipulation
• Enforcing Statistical Constraints in Generative Adversarial Networks for Modeling Chaotic Dynamical Systems
• Federated Multi-Agent Deep Reinforcement Learning Approach via Physics-Informed Reward for Multi-Microgrid Energy Management
• Safe Exploration in Model-based Reinforcement Learning using Control Barrier Functions
• Safe Exploration of State and Action Spaces in Reinforcement Learning
• Barrier-Certified Adaptive Reinforcement Learning with Applications to Brushbot Navigation
• Learning Functional Priors and Posteriors from Data and Physics
• On Passivity, Reinforcement Learning and Higher-Order Learning in Multi-Agent Finite Games
• End-To-End Latent Variational Diffusion Models for Inverse Problems in High Energy Physics
• A physics-informed reinforcement learning approach for the interfacial area transport in two-phase flow
• Physics-Informed Computer Vision: A Review and Perspectives

Cited by in corpus (1)

• Control of dynamical systems with neural networks