Deep Reinforcement Learning in Parameterized Action Space

Abstract

Recent work has shown that deep neural networks are capable of approximating both value functions and policies in reinforcement learning domains featuring continuous state and action spaces. However, to the best of our knowledge no previous work has succeeded at using deep neural networks in structured (parameterized) continuous action spaces. To fill this gap, this paper focuses on learning within the domain of simulated RoboCup soccer, which features a small set of discrete action types, each of which is parameterized with continuous variables. The best learned agent can score goals more reliably than the 2012 RoboCup champion agent. As such, this paper represents a successful extension of deep reinforcement learning to the class of parameterized action space MDPs.

References in corpus (4)

• Adam: A Method for Stochastic Optimization
• ADADELTA: An Adaptive Learning Rate Method
• End-to-End Training of Deep Visuomotor Policies
• Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models

Cited by in corpus (17)

• A Closer Look at Invalid Action Masking in Policy Gradient Algorithms
• Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning
• Guided Deep Reinforcement Learning for Swarm Systems
• Combining Deep Reinforcement Learning and Safety Based Control for Autonomous Driving
• Worst Cases Policy Gradients
• Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics
• Learning Event-triggered Control from Data through Joint Optimization
• Deep Reinforcement Learning for Event-Triggered Control
• Striving for Simplicity and Performance in Off-Policy DRL: Output Normalization and Non-Uniform Sampling
• An Artificial Agent for Robust Image Registration
• Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks
• Hierarchical Reinforcement Learning Method for Autonomous Vehicle Behavior Planning
• Hierarchical Approaches for Reinforcement Learning in Parameterized Action Space
• Macro Action Reinforcement Learning with Sequence Disentanglement using Variational Autoencoder
• Situational Awareness by Risk-Conscious Skills
• Situationally Aware Options
• Parameterized MDPs and Reinforcement Learning Problems -- A Maximum Entropy Principle Based Framework