Adaptive Prior Selection for Repertoire-based Online Adaptation in Robotics
arXiv:1907.07029 · doi:10.3389/frobt.2019.00151
Abstract
Repertoire-based learning is a data-efficient adaptation approach based on a two-step process in which (1) a large and diverse set of policies is learned in simulation, and (2) a planning or learning algorithm chooses the most appropriate policies according to the current situation (e.g., a damaged robot, a new object, etc.). In this paper, we relax the assumption of previous works that a single repertoire is enough for adaptation. Instead, we generate repertoires for many different situations (e.g., with a missing leg, on different floors, etc.) and let our algorithm selects the most useful prior. Our main contribution is an algorithm, APROL (Adaptive Prior selection for Repertoire-based Online Learning) to plan the next action by incorporating these priors when the robot has no information about the current situation. We evaluate APROL on two simulated tasks: (1) pushing unknown objects of various shapes and sizes with a robotic arm and (2) a goal reaching task with a damaged hexapod robot. We compare with "Reset-free Trial and Error" (RTE) and various single repertoire-based baselines. The results show that APROL solves both the tasks in less interaction time than the baselines. Additionally, we demonstrate APROL on a real, damaged hexapod that quickly learns to pick compensatory policies to reach a goal by avoiding obstacles in the path.
Frontiers in Robotics and AI. Vol. 6, p. 151, 2020. Video : http://tiny.cc/aprol_video
References in corpus (6)
- Robots that can adapt like animals
- Emergence of Locomotion Behaviours in Rich Environments
- Illuminating search spaces by mapping elites
- Dynamics-Aware Unsupervised Discovery of Skills
- Adaptive Prior Selection for Repertoire-based Online Adaptation in Robotics
- Safety-Aware Robot Damage Recovery Using Constrained Bayesian Optimization and Simulated Priors
Cited by in corpus (10)
- Quality Diversity for Multi-task Optimization
- Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning
- Adaptive Prior Selection for Repertoire-based Online Adaptation in Robotics
- REX: Designing User-centered Repair and Explanations to Address Robot Failures
- Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoires
- Hierarchical Quality-Diversity for Online Damage Recovery
- Evolving the Behavior of Machines: From Micro to Macroevolution
- Relevance-guided Unsupervised Discovery of Abilities with Quality-Diversity Algorithms
- Competitiveness of MAP-Elites against Proximal Policy Optimization on locomotion tasks in deterministic simulations
- A Simple Approach to Continual Learning by Transferring Skill Parameters