From semantics to execution: Integrating action planning with reinforcement learning for robotic causal problem-solving
arXiv:1905.09683
Abstract
Reinforcement learning is an appropriate and successful method to robustly perform low-level robot control under noisy conditions. Symbolic action planning is useful to resolve causal dependencies and to break a causally complex problem down into a sequence of simpler high-level actions. A problem with the integration of both approaches is that action planning is based on discrete high-level action- and state spaces, whereas reinforcement learning is usually driven by a continuous reward function. However, recent advances in reinforcement learning, specifically, universal value function approximators and hindsight experience replay, have focused on goal-independent methods based on sparse rewards. In this article, we build on these novel methods to facilitate the integration of action planning with reinforcement learning by exploiting the reward-sparsity as a bridge between the high-level and low-level state- and control spaces. As a result, we demonstrate that the integrated neuro-symbolic method is able to solve object manipulation problems that involve tool use and non-trivial causal dependencies under noisy conditions, exploiting both data and knowledge.
References in corpus (12)
- Continuous control with deep reinforcement learning
- Trust Region Policy Optimization
- Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
- State Representation Learning for Control: An Overview
- Learning Dexterous In-Hand Manipulation
- FeUdal Networks for Hierarchical Reinforcement Learning
- Data-Efficient Hierarchical Reinforcement Learning
- Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning
- Observe and Look Further: Achieving Consistent Performance on Atari
- Learning Multi-Level Hierarchies with Hindsight
- Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfer
- Hierarchical Reinforcement Learning with Abductive Planning