Hierarchical Quality-Diversity for Online Damage Recovery
arXiv:2204.05726 · doi:10.1145/3512290.3528751
Abstract
Adaptation capabilities, like damage recovery, are crucial for the deployment of robots in complex environments. Several works have demonstrated that using repertoires of pre-trained skills can enable robots to adapt to unforeseen mechanical damages in a few minutes. These adaptation capabilities are directly linked to the behavioural diversity in the repertoire. The more alternatives the robot has to execute a skill, the better are the chances that it can adapt to a new situation. However, solving complex tasks, like maze navigation, usually requires multiple different skills. Finding a large behavioural diversity for these multiple skills often leads to an intractable exponential growth of the number of required solutions. In this paper, we introduce the Hierarchical Trial and Error algorithm, which uses a hierarchical behavioural repertoire to learn diverse skills and leverages them to make the robot more adaptive to different situations. We show that the hierarchical decomposition of skills enables the robot to learn more complex behaviours while keeping the learning of the repertoire tractable. The experiments with a hexapod robot show that our method solves maze navigation tasks with 20% less actions in the most challenging scenarios than the best baseline while having 57% less complete failures.
arXiv commentAccepted at GECCO 2022
References in corpus (18)
- Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design
- Robots that can adapt like animals
- Learning robust perceptive locomotion for quadrupedal robots in the wild
- Solving Rubik's Cube with a Robot Hand
- Illuminating search spaces by mapping elites
- First return, then explore
- Reset-free Trial-and-Error Learning for Robot Damage Recovery
- Adaptive CLF-MPC With Application To Quadrupedal Robots
- Autonomous skill discovery with Quality-Diversity and Unsupervised Descriptors
- Curiosity Driven Exploration of Learned Disentangled Goal Spaces
- Quality Diversity for Multi-task Optimization
- Hierarchical Behavioral Repertoires with Unsupervised Descriptors
- Adaptive Prior Selection for Repertoire-based Online Adaptation in Robotics
- From exploration to control: learning object manipulation skills through novelty search and local adaptation
- From Pixels to Legs: Hierarchical Learning of Quadruped Locomotion
- Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems
- Unsupervised Behaviour Discovery with Quality-Diversity Optimisation
- BR-NS: an Archive-less Approach to Novelty Search