Dealing with Sparse Rewards in Reinforcement Learning

Abstract

Successfully navigating a complex environment to obtain a desired outcome is a difficult task, that up to recently was believed to be capable only by humans. This perception has been broken down over time, especially with the introduction of deep reinforcement learning, which has greatly increased the difficulty of tasks that can be automated. However, for traditional reinforcement learning agents this requires an environment to be able to provide frequent extrinsic rewards, which are not known or accessible for many real-world environments. This project aims to explore and contrast existing reinforcement learning solutions that circumnavigate the difficulties of an environment that provide sparse rewards. Different reinforcement solutions will be implemented over a several video game environments with varying difficulty and varying frequency of rewards, as to properly investigate the applicability of these solutions. This project introduces a novel reinforcement learning solution by combining aspects of two existing state of the art sparse reward solutions, curiosity driven exploration and unsupervised auxiliary tasks.

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Playing Atari with Deep Reinforcement Learning
• Trust Region Policy Optimization
• Dueling Network Architectures for Deep Reinforcement Learning
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• StarCraft II: A New Challenge for Reinforcement Learning
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
• Large-Scale Study of Curiosity-Driven Learning
• Hindsight Experience Replay
• Reinforcement Learning with Unsupervised Auxiliary Tasks
• Exploration by Random Network Distillation

Cited by in corpus (6)

• ScheduleNet: Learn to solve multi-agent scheduling problems with reinforcement learning
• Systematic Generalisation through Task Temporal Logic and Deep Reinforcement Learning
• Locally Persistent Exploration in Continuous Control Tasks with Sparse Rewards
• Goal-constrained Sparse Reinforcement Learning for End-to-End Driving
• Learning in Sparse Rewards settings through Quality-Diversity algorithms
• HMRL: Hyper-Meta Learning for Sparse Reward Reinforcement Learning Problem