Autonomous skill discovery with Quality-Diversity and Unsupervised Descriptors

Abstract

Quality-Diversity optimization is a new family of optimization algorithms that, instead of searching for a single optimal solution to solving a task, searches for a large collection of solutions that all solve the task in a different way. This approach is particularly promising for learning behavioral repertoires in robotics, as such a diversity of behaviors enables robots to be more versatile and resilient. However, these algorithms require the user to manually define behavioral descriptors, which is used to determine whether two solutions are different or similar. The choice of a behavioral descriptor is crucial, as it completely changes the solution types that the algorithm derives. In this paper, we introduce a new method to automatically define this descriptor by combining Quality-Diversity algorithms with unsupervised dimensionality reduction algorithms. This approach enables robots to autonomously discover the range of their capabilities while interacting with their environment. The results from two experimental scenarios demonstrate that robot can autonomously discover a large range of possible behaviors, without any prior knowledge about their morphology and environment. Furthermore, these behaviors are deemed to be similar to handcrafted solutions that uses domain knowledge and significantly more diverse than when using existing unsupervised methods.

arXiv commentThe Genetic and Evolutionary Computation Conference 2019

References in corpus (1)

• Hierarchical Behavioral Repertoires with Unsupervised Descriptors

Cited by in corpus (21)

• Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space
• Evolutionary Reinforcement Learning: A Survey
• Multi-Emitter MAP-Elites: Improving quality, diversity and convergence speed with heterogeneous sets of emitters
• Seeking Quality Diversity in Evolutionary Co-design of Morphology and Control of Soft Tensegrity Modular Robots
• Open-Ended Evolution for Minecraft Building Generation
• Expressivity of Parameterized and Data-driven Representations in Quality Diversity Search
• Hierarchical Quality-Diversity for Online Damage Recovery
• Monte Carlo Elites: Quality-Diversity Selection as a Multi-Armed Bandit Problem
• Ensemble Feature Extraction for Multi-Container Quality-Diversity Algorithms
• Quality-Diversity Meta-Evolution: customising behaviour spaces to a meta-objective
• Learning to Walk Autonomously via Reset-Free Quality-Diversity
• Goals as Reward-Producing Programs
• An Analysis of Phenotypic Diversity in Multi-Solution Optimization
• Relevance-guided Unsupervised Discovery of Abilities with Quality-Diversity Algorithms
• BR-NS: an Archive-less Approach to Novelty Search
• Gradient-Informed Quality Diversity for the Illumination of Discrete Spaces
• Open-ended search for environments and adapted agents using MAP-Elites
• Curiosity creates Diversity in Policy Search
• Online Behavior Modification for Expressive User Control of RL-Trained Robots
• On the use of feature-maps and parameter control for improved quality-diversity meta-evolution
• Evolution of linkages for prototyping of linkage based robots