Neural Graph Evolution: Towards Efficient Automatic Robot Design

Abstract

Despite the recent successes in robotic locomotion control, the design of robot relies heavily on human engineering. Automatic robot design has been a long studied subject, but the recent progress has been slowed due to the large combinatorial search space and the difficulty in evaluating the found candidates. To address the two challenges, we formulate automatic robot design as a graph search problem and perform evolution search in graph space. We propose Neural Graph Evolution (NGE), which performs selection on current candidates and evolves new ones iteratively. Different from previous approaches, NGE uses graph neural networks to parameterize the control policies, which reduces evaluation cost on new candidates with the help of skill transfer from previously evaluated designs. In addition, NGE applies Graph Mutation with Uncertainty (GM-UC) by incorporating model uncertainty, which reduces the search space by balancing exploration and exploitation. We show that NGE significantly outperforms previous methods by an order of magnitude. As shown in experiments, NGE is the first algorithm that can automatically discover kinematically preferred robotic graph structures, such as a fish with two symmetrical flat side-fins and a tail, or a cheetah with athletic front and back legs. Instead of using thousands of cores for weeks, NGE efficiently solves searching problem within a day on a single 64 CPU-core Amazon EC2 machine.

ICLR 2019

References in corpus (13)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Semi-Supervised Classification with Graph Convolutional Networks
• Practical Bayesian Optimization of Machine Learning Algorithms
• Playing Atari with Deep Reinforcement Learning
• Neural Architecture Search with Reinforcement Learning
• Deep Convolutional Networks on Graph-Structured Data
• Emergence of Locomotion Behaviours in Rich Environments
• Efficient Neural Architecture Search via Parameter Sharing
• DeepMind Control Suite
• Gated Graph Sequence Neural Networks
• Hierarchical Representations for Efficient Architecture Search
• Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
• Emergent Complexity via Multi-Agent Competition

Cited by in corpus (12)

• DiffAqua: A Differentiable Computational Design Pipeline for Soft Underwater Swimmers with Shape Interpolation
• One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control
• Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation
• Learning Robot Structure and Motion Embeddings using Graph Neural Networks
• Self-supervised Graph Representation Learning via Bootstrapping
• Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design
• Task-Agnostic Morphology Evolution
• Multi-Objective Graph Heuristic Search for Terrestrial Robot Design
• Task2Morph: Differentiable Task-inspired Framework for Contact-Aware Robot Design
• Emergent Hand Morphology and Control from Optimizing Robust Grasps of Diverse Objects
• A Graph Deep Learning Framework for High-Level Synthesis Design Space Exploration
• Neural fidelity warping for efficient robot morphology design