Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space

Abstract

Most existing deep reinforcement learning (DRL) frameworks consider either discrete action space or continuous action space solely. Motivated by applications in computer games, we consider the scenario with discrete-continuous hybrid action space. To handle hybrid action space, previous works either approximate the hybrid space by discretization, or relax it into a continuous set. In this paper, we propose a parametrized deep Q-network (P- DQN) framework for the hybrid action space without approximation or relaxation. Our algorithm combines the spirits of both DQN (dealing with discrete action space) and DDPG (dealing with continuous action space) by seamlessly integrating them. Empirical results on a simulation example, scoring a goal in simulated RoboCup soccer and the solo mode in game King of Glory (KOG) validate the efficiency and effectiveness of our method.

References in corpus (2)

• Continuous control with deep reinforcement learning
• Guided Policy Search as Approximate Mirror Descent

Cited by in corpus (14)

• Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
• D3PG: Dirichlet DDPG for Task Partitioning and Offloading with Constrained Hybrid Action Space in Mobile Edge Computing
• Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spaces
• Discrete and Continuous Action Representation for Practical RL in Video Games
• Parameterized Reinforcement Learning for Optical System Optimization
• Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics
• Optimal Scheduling in IoT-Driven Smart Isolated Microgrids Based on Deep Reinforcement Learning
• Reinforcement Learning for Molecular Design Guided by Quantum Mechanics
• Deep Imitation Learning for Bimanual Robotic Manipulation
• Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks
• TAAC: Temporally Abstract Actor-Critic for Continuous Control
• Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives
• HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation
• Parameterized MDPs and Reinforcement Learning Problems -- A Maximum Entropy Principle Based Framework