Neurosymbolic Reinforcement Learning and Planning: A Survey

Abstract

The area of Neurosymbolic Artificial Intelligence (Neurosymbolic AI) is rapidly developing and has become a popular research topic, encompassing sub-fields such as Neurosymbolic Deep Learning (Neurosymbolic DL) and Neurosymbolic Reinforcement Learning (Neurosymbolic RL). Compared to traditional learning methods, Neurosymbolic AI offers significant advantages by simplifying complexity and providing transparency and explainability. Reinforcement Learning(RL), a long-standing Artificial Intelligence(AI) concept that mimics human behavior using rewards and punishment, is a fundamental component of Neurosymbolic RL, a recent integration of the two fields that has yielded promising results. The aim of this paper is to contribute to the emerging field of Neurosymbolic RL by conducting a literature survey. Our evaluation focuses on the three components that constitute Neurosymbolic RL: neural, symbolic, and RL. We categorize works based on the role played by the neural and symbolic parts in RL, into three taxonomies:Learning for Reasoning, Reasoning for Learning and Learning-Reasoning. These categories are further divided into sub-categories based on their applications. Furthermore, we analyze the RL components of each research work, including the state space, action space, policy module, and RL algorithm. Additionally, we identify research opportunities and challenges in various applications within this dynamic field.

arXiv comment16 pages, 9 figures, IEEE Transactions on Artificial Intelligence

References in corpus (33)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• A Brief Survey of Deep Reinforcement Learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Trust Region Policy Optimization
• A Survey on Knowledge Graphs: Representation, Acquisition and Applications
• The Symbol Grounding Problem
• Prioritized Experience Replay
• A Deep Reinforced Model for Abstractive Summarization
• The Arcade Learning Environment: An Evaluation Platform for General Agents
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
• Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
• OpenAI Gym
• Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning
• Neural-Symbolic Learning and Reasoning: A Survey and Interpretation
• A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments
• Reinforcement Learning for IoT Security: A Comprehensive Survey
• Verifiable Reinforcement Learning via Policy Extraction
• Survey on reinforcement learning for language processing
• Towards Deep Symbolic Reinforcement Learning
• Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective
• Reinforcement Learning for Ridesharing: An Extended Survey
• Causal Discovery with Reinforcement Learning
• A survey on intrinsic motivation in reinforcement learning
• A Survey of Explainable Reinforcement Learning
• Learning Neuro-Symbolic Skills for Bilevel Planning
• Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computing
• Learning to Synthesize Programs as Interpretable and Generalizable Policies
• Learning Symbolic Rules for Interpretable Deep Reinforcement Learning
• Survey on Applications of Neurosymbolic Artificial Intelligence
• A Survey on Deep Reinforcement Learning-based Approaches for Adaptation and Generalization
• An Introduction of mini-AlphaStar
• A Survey on Neural-symbolic Learning Systems

Cited by in corpus (6)

• A Survey on Symbolic Knowledge Distillation of Large Language Models
• A Survey on Verification and Validation, Testing and Evaluations of Neurosymbolic Artificial Intelligence
• A Review of Symbolic, Subsymbolic and Hybrid Methods for Sequential Decision Making
• NeuroStrata: Harnessing Neurosymbolic Paradigms for Improved Design, Testability, and Verifiability of Autonomous CPS
• Neuro-Symbolic Imitation Learning: Discovering Symbolic Abstractions for Skill Learning
• Neurosymbolic AI for Travel Demand Prediction: Integrating Decision Tree Rules into Neural Networks