Soft Actor-Critic for Discrete Action Settings

Abstract

Soft Actor-Critic is a state-of-the-art reinforcement learning algorithm for continuous action settings that is not applicable to discrete action settings. Many important settings involve discrete actions, however, and so here we derive an alternative version of the Soft Actor-Critic algorithm that is applicable to discrete action settings. We then show that, even without any hyperparameter tuning, it is competitive with the tuned model-free state-of-the-art on a selection of games from the Atari suite.

References in corpus (7)

• Auto-Encoding Variational Bayes
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Addressing Function Approximation Error in Actor-Critic Methods
• Soft Actor-Critic Algorithms and Applications
• Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
• Model-Based Reinforcement Learning for Atari
• Dopamine: A Research Framework for Deep Reinforcement Learning

Cited by in corpus (22)

• Identifying optimal cycles in quantum thermal machines with reinforcement-learning
• d3rlpy: An Offline Deep Reinforcement Learning Library
• Differentiated Federated Reinforcement Learning Based Traffic Offloading on Space-Air-Ground Integrated Networks
• Discrete and Continuous Action Representation for Practical RL in Video Games
• A Deep Reinforcement Learning-Based Resource Scheduler for Massive MIMO Networks
• Hit and Lead Discovery with Explorative RL and Fragment-based Molecule Generation
• Strategies for Using Proximal Policy Optimization in Mobile Puzzle Games
• Pipeline PSRO: A Scalable Approach for Finding Approximate Nash Equilibria in Large Games
• SOAC: The Soft Option Actor-Critic Architecture
• Learning to Robustly Negotiate Bi-Directional Lane Usage in High-Conflict Driving Scenarios
• Counterfactual experience augmented off-policy reinforcement learning
• Deep Inductive Logic Programming meets Reinforcement Learning
• Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning
• Certifiable Robustness to Adversarial State Uncertainty in Deep Reinforcement Learning
• Ground States of Quantum Many Body Lattice Models via Reinforcement Learning
• Investigation of Factorized Optical Flows as Mid-Level Representations
• Soft Actor-Critic With Integer Actions
• Visual Radial Basis Q-Network
• RTDK-BO: High Dimensional Bayesian Optimization with Reinforced Transformer Deep kernels
• Dual Behavior Regularized Reinforcement Learning
• Simultaneous Navigation and Construction Benchmarking Environments
• Count-Based Temperature Scheduling for Maximum Entropy Reinforcement Learning