Learning What to Memorize: Using Intrinsic Motivation to Form Useful Memory in Partially Observable Reinforcement Learning

Abstract

Reinforcement Learning faces an important challenge in partial observable environments that has long-term dependencies. In order to learn in an ambiguous environment, an agent has to keep previous perceptions in a memory. Earlier memory based approaches use a fixed method to determine what to keep in the memory, which limits them to certain problems. In this study, we follow the idea of giving the control of the memory to the agent by allowing it to have memory-changing actions. This learning mechanism is supported by an intrinsic motivation to memorize rare observations that can help the agent to disambiguate its state in the environment. Our approach is experimented and analyzed on several partial observable tasks with long-term dependencies and compared with other memory based methods.

arXiv commentAcknowledgements are added. Appl Intell (2023)

References in corpus (9)

• Count-Based Exploration with Neural Density Models
• Sample Efficient Actor-Critic with Experience Replay
• Learning Finite-State Controllers for Partially Observable Environments
• Solving POMDPs by Searching the Space of Finite Policies
• Learning Policies with External Memory
• A survey on intrinsic motivation in reinforcement learning
• On Improving Deep Reinforcement Learning for POMDPs
• Deep Hierarchical Reinforcement Learning Algorithm in Partially Observable Markov Decision Processes
• Scheduled Intrinsic Drive: A Hierarchical Take on Intrinsically Motivated Exploration