Robots that can adapt like animals
arXiv:1407.3501 · doi:10.1038/nature14422
Abstract
As robots leave the controlled environments of factories to autonomously function in more complex, natural environments, they will have to respond to the inevitable fact that they will become damaged. However, while animals can quickly adapt to a wide variety of injuries, current robots cannot "think outside the box" to find a compensatory behavior when damaged: they are limited to their pre-specified self-sensing abilities, can diagnose only anticipated failure modes, and require a pre-programmed contingency plan for every type of potential damage, an impracticality for complex robots. Here we introduce an intelligent trial and error algorithm that allows robots to adapt to damage in less than two minutes, without requiring self-diagnosis or pre-specified contingency plans. Before deployment, a robot exploits a novel algorithm to create a detailed map of the space of high-performing behaviors: This map represents the robot's intuitions about what behaviors it can perform and their value. If the robot is damaged, it uses these intuitions to guide a trial-and-error learning algorithm that conducts intelligent experiments to rapidly discover a compensatory behavior that works in spite of the damage. Experiments reveal successful adaptations for a legged robot injured in five different ways, including damaged, broken, and missing legs, and for a robotic arm with joints broken in 14 different ways. This new technique will enable more robust, effective, autonomous robots, and suggests principles that animals may use to adapt to injury.
References in corpus (2)
Cited by in corpus (147)
- Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
- Solving Rubik's Cube with a Robot Hand
- Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
- First return, then explore
- Go-Explore: a New Approach for Hard-Exploration Problems
- A soft robot that adapts to environments through shape change
- Multi-expert learning of adaptive legged locomotion
- Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning
- Evolving embodied intelligence from materials to machines
- Evolution of Collective Behaviors for a Real Swarm of Aquatic Surface Robots
- Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions
- The evolutionary origins of hierarchy
- Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space
- Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents
- Meta Reinforcement Learning with Latent Variable Gaussian Processes
- Reset-free Trial-and-Error Learning for Robot Damage Recovery
- Embodied Evolution in Collective Robotics: A Review
- Learning to Walk in the Real World with Minimal Human Effort
- Evolutionary Reinforcement Learning: A Survey
- Discovering the Elite Hypervolume by Leveraging Interspecies Correlation
- Data-efficient Domain Randomization with Bayesian Optimization
- Curiosity Driven Exploration of Learned Disentangled Goal Spaces
- Quality Diversity for Multi-task Optimization
- Automated shapeshifting for function recovery in damaged robots
- Turing learning: a metric-free approach to inferring behavior and its application to swarms
- Data-Efficient Exploration, Optimization, and Modeling of Diverse Designs through Surrogate-Assisted Illumination
- Effective Diversity in Population Based Reinforcement Learning
- Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions
- Diversity Policy Gradient for Sample Efficient Quality-Diversity Optimization
- Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning
- Bayesian Optimization with Adaptive Kernels for Robot Control
- Fast and stable MAP-Elites in noisy domains using deep grids
- Hierarchical Behavioral Repertoires with Unsupervised Descriptors
- Discovering Representations for Black-box Optimization
- Adaptive Prior Selection for Repertoire-based Online Adaptation in Robotics
- Multi-Objective Quality Diversity Optimization
- From exploration to control: learning object manipulation skills through novelty search and local adaptation
- Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion
- Differentiable Quality Diversity
- Multi-Emitter MAP-Elites: Improving quality, diversity and convergence speed with heterogeneous sets of emitters
- Policy Manifold Search: Exploring the Manifold Hypothesis for Diversity-based Neuroevolution
- Towards the Targeted Environment-Specific Evolution of Robot Components
- Robust Optimization through Neuroevolution
- Behavioural Repertoire via Generative Adversarial Policy Networks
- Intrinsic Motivation and Mental Replay enable Efficient Online Adaptation in Stochastic Recurrent Networks
- The Emergence of Canalization and Evolvability in an Open-Ended, Interactive Evolutionary System
- Deep Surrogate Assisted MAP-Elites for Automated Hearthstone Deckbuilding
- Preparation of ordered states in ultra-cold gases using Bayesian optimization
- Adaptive Robotic Information Gathering via Non-Stationary Gaussian Processes
- Empirical analysis of PGA-MAP-Elites for Neuroevolution in Uncertain Domains
- Multi-Robot Coordination and Layout Design for Automated Warehousing
- Evolution of a Functionally Diverse Swarm via a Novel Decentralised Quality-Diversity Algorithm
- Efficient Pipeline Planning for Expedited Distributed DNN Training
- Interoceptive robustness through environment-mediated morphological development
- Resilient robot teams: a review integrating decentralised control, change-detection, and learning
- Noisy-Input Entropy Search for Efficient Robust Bayesian Optimization
- How to Blend a Robot within a Group of Zebrafish: Achieving Social Acceptance through Real-time Calibration of a Multi-level Behavioural Model
- Designing Air Flow with Surrogate-assisted Phenotypic Niching
- Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skills
- Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation
- Qualities, challenges and future of genetic algorithms: a literature review
- Learning Directed Locomotion in Modular Robots with Evolvable Morphologies
- Evolving and Merging Hebbian Learning Rules: Increasing Generalization by Decreasing the Number of Rules
- Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoires
- Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomization
- Expressivity of Parameterized and Data-driven Representations in Quality Diversity Search
- Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems
- Evolving the Behavior of Machines: From Micro to Macroevolution
- Hierarchical Quality-Diversity for Online Damage Recovery
- Bayesian Optimization Meets Riemannian Manifolds in Robot Learning
- The Role of Morphological Variation in Evolutionary Robotics: Maximizing Performance and Robustness
- Bayesian Optimization in Variational Latent Spaces with Dynamic Compression
- Few-shot Quality-Diversity Optimization
- Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning
- Workspace Optimization Techniques to Improve Prediction of Human Motion During Human-Robot Collaboration
- A Survey of Behavior Learning Applications in Robotics -- State of the Art and Perspectives
- Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems
- Map-based Multi-Policy Reinforcement Learning: Enhancing Adaptability of Robots by Deep Reinforcement Learning
- First do not fall: learning to exploit a wall with a damaged humanoid robot
- Rapidly adapting robot swarms with Swarm Map-based Bayesian Optimisation
- Fast Adaptation via Policy-Dynamics Value Functions
- Automatic Calibration of Artificial Neural Networks for Zebrafish Collective Behaviours using a Quality Diversity Algorithm
- Quality-Diversity Meta-Evolution: customising behaviour spaces to a meta-objective
- Efficient reinforcement learning control for continuum robots based on Inexplicit Prior Knowledge
- Modeling User Selection in Quality Diversity
- Sampling diverse near-optimal solutions via algorithmic quantum annealing
- Regenerating Soft Robots through Neural Cellular Automata
- From Intelligent Agents to Trustworthy Human-Centred Multiagent Systems
- An Analysis of Phenotypic Diversity in Multi-Solution Optimization
- Human irrationality: both bad and good for reward inference
- DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics
- Overcoming Deceptiveness in Fitness Optimization with Unsupervised Quality-Diversity
- Parametric-Task MAP-Elites
- BOP-Elites, a Bayesian Optimisation algorithm for Quality-Diversity search
- Relevance-guided Unsupervised Discovery of Abilities with Quality-Diversity Algorithms
- Fiber: A Platform for Efficient Development and Distributed Training for Reinforcement Learning and Population-Based Methods
- Towards Multi-Morphology Controllers with Diversity and Knowledge Distillation
- Deep Kernels for Optimizing Locomotion Controllers
- Generating and Adapting to Diverse Ad-Hoc Cooperation Agents in Hanabi
- An Information Theoretic Measure for Robot Expressivity
- Expressivity in Natural and Artificial Systems
- On the Importance of Environments in Human-Robot Coordination
- Advanced turning maneuver of a multi-legged robot using pitchfork bifurcation
- Learning Generalizable Locomotion Skills with Hierarchical Reinforcement Learning
- Coevo: a collaborative design platform with artificial agents
- Planning in Learned Latent Action Spaces for Generalizable Legged Locomotion
- Scalable sim-to-real transfer of soft robot designs
- An Energy-Saving Snake Locomotion Gait Policy Obtained Using Deep Reinforcement Learning
- Past Visions of Artificial Futures: One Hundred and Fifty Years under the Spectre of Evolving Machines
- Multi-objective Analysis of MAP-Elites Performance
- Harnessing Distribution Ratio Estimators for Learning Agents with Quality and Diversity
- Evolutionary Gait Transfer of Multi-Legged Robots in Complex Terrains
- Open-ended search for environments and adapted agents using MAP-Elites
- Neuroevolution in Games: State of the Art and Open Challenges
- Curiosity creates Diversity in Policy Search
- A Simple Approach to Continual Learning by Transferring Skill Parameters
- Comparing reliability of grid-based Quality-Diversity algorithms using artificial landscapes
- Fault-Aware Robust Control via Adversarial Reinforcement Learning
- Competitiveness of MAP-Elites against Proximal Policy Optimization on locomotion tasks in deterministic simulations
- Evolving Flying Machines in Minecraft Using Quality Diversity
- GeMuCo: Generalized Multisensory Correlational Model for Body Schema Learning
- Understanding trade-offs in classifier bias with quality-diversity optimization: an application to talent management
- A Quality Diversity Approach to Evolving Model Rockets
- Growable and Interpretable Neural Control with Online Continual Learning for Autonomous Lifelong Locomotion Learning Machines
- Survivable Hyper-Redundant Robotic Arm with Bayesian Policy Morphing
- Robot Design With Neural Networks, MILP Solvers and Active Learning
- Self-Referential Quality Diversity Through Differential Map-Elites
- High-Dimensional Bayesian Optimization via Nested Riemannian Manifolds
- Protective Policy Transfer
- Diversity-based Design Assist for Large Legged Robots
- Bayesian Optimization for Developmental Robotics with Meta-Learning by Parameters Bounds Reduction
- Divergent Search for Few-Shot Image Classification
- Policy Manifold Search for Improving Diversity-based Neuroevolution
- EPANer Team Description Paper for World Robot Challenge 2020
- Motion Generation Considering Situation with Conditional Generative Adversarial Networks for Throwing Robots
- Selecting for Selection: Learning To Balance Adaptive and Diversifying Pressures in Evolutionary Search
- Robust Policy Search for Robot Navigation
- QED: using Quality-Environment-Diversity to evolve resilient robot swarms
- Natural Language Interaction to Facilitate Mental Models of Remote Robots
- Discovering Evolutionary Stepping Stones through Behavior Domination
- Learning in Sparse Rewards settings through Quality-Diversity algorithms
- Deep Bayesian Optimization on Attributed Graphs
- Learning Transferable Concepts in Deep Reinforcement Learning
- On the use of feature-maps and parameter control for improved quality-diversity meta-evolution
- Lifetime policy reuse and the importance of task capacity
- Hierarchical Surrogate Modeling for Illumination Algorithms
- Generating Diverse Challenging Terrains for Legged Robots Using Quality-Diversity Algorithm