DSAC: Distributional Soft Actor-Critic for Risk-Sensitive Reinforcement Learning

Abstract

We present Distributional Soft Actor-Critic (DSAC), a distributional reinforcement learning (RL) algorithm that combines the strengths of distributional information of accumulated rewards and entropy-driven exploration from Soft Actor-Critic (SAC) algorithm. DSAC models the randomness in both action and rewards, surpassing baseline performances on various continuous control tasks. Unlike standard approaches that solely maximize expected rewards, we propose a unified framework for risk-sensitive learning, one that optimizes the risk-related objective while balancing entropy to encourage exploration. Extensive experiments demonstrate DSAC's effectiveness in enhancing agent performances for both risk-neutral and risk-sensitive control tasks.

Accecpted by Journal of Artificial Intelligence Research

Cited by in corpus (4)

• Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Control
• A Risk-Sensitive Approach to Policy Optimization
• Mean-Semivariance Policy Optimization via Risk-Averse Reinforcement Learning
• Reinforcement Learning for Vision-based Object Manipulation with Non-parametric Policy and Action Primitives