Distributional Reinforcement Learning with Ensembles

Abstract

It is well known that ensemble methods often provide enhanced performance in reinforcement learning. In this paper, we explore this concept further by using group-aided training within the distributional reinforcement learning paradigm. Specifically, we propose an extension to categorical reinforcement learning, where distributional learning targets are implicitly based on the total information gathered by an ensemble. We empirically show that this may lead to much more robust initial learning, a stronger individual performance level, and good efficiency on a per-sample basis.

arXiv comment15 pages, 2 figures

References in corpus (5)

• The Arcade Learning Environment: An Evaluation Platform for General Agents
• A Distributional Perspective on Reinforcement Learning
• An Analysis of Categorical Distributional Reinforcement Learning
• Parametric Return Density Estimation for Reinforcement Learning
• Distributional reinforcement learning with linear function approximation

Cited by in corpus (1)

• A Hormetic Approach to the Value-Loading Problem: Preventing the Paperclip Apocalypse?