Relevance-guided Unsupervised Discovery of Abilities with Quality-Diversity Algorithms
arXiv:2204.09828 · doi:10.1145/3512290.3528837
Abstract
Quality-Diversity algorithms provide efficient mechanisms to generate large collections of diverse and high-performing solutions, which have shown to be instrumental for solving downstream tasks. However, most of those algorithms rely on a behavioural descriptor to characterise the diversity that is hand-coded, hence requiring prior knowledge about the considered tasks. In this work, we introduce Relevance-guided Unsupervised Discovery of Abilities; a Quality-Diversity algorithm that autonomously finds a behavioural characterisation tailored to the task at hand. In particular, our method introduces a custom diversity metric that leads to higher densities of solutions near the areas of interest in the learnt behavioural descriptor space. We evaluate our approach on a simulated robotic environment, where the robot has to autonomously discover its abilities based on its full sensory data. We evaluated the algorithms on three tasks: navigation to random targets, moving forward with a high velocity, and performing half-rolls. The experimental results show that our method manages to discover collections of solutions that are not only diverse, but also well-adapted to the considered downstream task.
arXiv commentAccepted at GECCO 2022
References in corpus (20)
- Adam: A Method for Stochastic Optimization
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Learning agile and dynamic motor skills for legged robots
- Robots that can adapt like animals
- Solving Rubik's Cube with a Robot Hand
- Illuminating search spaces by mapping elites
- Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space
- Reset-free Trial-and-Error Learning for Robot Damage Recovery
- Autonomous skill discovery with Quality-Diversity and Unsupervised Descriptors
- Empowering Quality Diversity in Dungeon Design with Interactive Constrained MAP-Elites
- Quality Diversity for Multi-task Optimization
- Transforming Exploratory Creativity with DeLeNoX
- Interactive Constrained MAP-Elites: Analysis and Evaluation of the Expressiveness of the Feature Dimensions
- Adaptive Prior Selection for Repertoire-based Online Adaptation in Robotics
- From exploration to control: learning object manipulation skills through novelty search and local adaptation
- Differentiable Quality Diversity
- Ensemble Feature Extraction for Multi-Container Quality-Diversity Algorithms
- Sparse Reward Exploration via Novelty Search and Emitters
- Unsupervised Behaviour Discovery with Quality-Diversity Optimisation
- BR-NS: an Archive-less Approach to Novelty Search