Information Theoretically Aided Reinforcement Learning for Embodied Agents

Abstract

Reinforcement learning for embodied agents is a challenging problem. The accumulated reward to be optimized is often a very rugged function, and gradient methods are impaired by many local optimizers. We demonstrate, in an experimental setting, that incorporating an intrinsic reward can smoothen the optimization landscape while preserving the global optimizers of interest. We show that policy gradient optimization for locomotion in a complex morphology is significantly improved when supplementing the extrinsic reward by an intrinsic reward defined in terms of the mutual information of time consecutive sensor readings.

arXiv comment10 pages, 4 figures, 8 pages appendix

References in corpus (1)

• Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning

Cited by in corpus (6)

• VIME: Variational Information Maximizing Exploration
• A unified strategy for implementing curiosity and empowerment driven reinforcement learning
• A Unified Bellman Optimality Principle Combining Reward Maximization and Empowerment
• Curiosity-driven reinforcement learning with homeostatic regulation
• Adaptive Reward-Free Exploration
• Mega-Reward: Achieving Human-Level Play without Extrinsic Rewards