Behavioural Repertoire via Generative Adversarial Policy Networks

Abstract

Learning algorithms are enabling robots to solve increasingly challenging real-world tasks. These approaches often rely on demonstrations and reproduce the behavior shown. Unexpected changes in the environment may require using different behaviors to achieve the same effect, for instance to reach and grasp an object in changing clutter. An emerging paradigm addressing this robustness issue is to learn a diverse set of successful behaviors for a given task, from which a robot can select the most suitable policy when faced with a new environment. In this paper, we explore a novel realization of this vision by learning a generative model over policies. Rather than learning a single policy, or a small fixed repertoire, our generative model for policies compactly encodes an unbounded number of policies and allows novel controller variants to be sampled. Leveraging our generative policy network, a robot can sample novel behaviors until it finds one that works for a new environment. We demonstrate this idea with an application of robust ball-throwing in the presence of obstacles. We show that this approach achieves a greater diversity of behaviors than an existing evolutionary approach, while maintaining good efficacy of sampled behaviors, allowing a Baxter robot to hit targets more often when ball throwing in the presence of obstacles.

arXiv commentIn Proceedings of 2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), pages 320 - 326

References in corpus (5)

• Continuous control with deep reinforcement learning
• Conditional Generative Adversarial Nets
• Generative Adversarial Text to Image Synthesis
• Robots that can adapt like animals
• Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning

Cited by in corpus (4)

• A Review on Neural Network Models of Schizophrenia and Autism Spectrum Disorder
• From exploration to control: learning object manipulation skills through novelty search and local adaptation
• Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?
• Disentangling causal effects for hierarchical reinforcement learning