Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity

Abstract

Contemporary sensorimotor learning approaches typically start with an existing complex agent (e.g., a robotic arm), which they learn to control. In contrast, this paper investigates a modular co-evolution strategy: a collection of primitive agents learns to dynamically self-assemble into composite bodies while also learning to coordinate their behavior to control these bodies. Each primitive agent consists of a limb with a motor attached at one end. Limbs may choose to link up to form collectives. When a limb initiates a link-up action, and there is another limb nearby, the latter is magnetically connected to the 'parent' limb's motor. This forms a new single agent, which may further link with other agents. In this way, complex morphologies can emerge, controlled by a policy whose architecture is in explicit correspondence with the morphology. We evaluate the performance of these dynamic and modular agents in simulated environments. We demonstrate better generalization to test-time changes both in the environment, as well as in the structure of the agent, compared to static and monolithic baselines. Project video and code are available at https://pathak22.github.io/modular-assemblies/

arXiv commentNeurIPS 2019 (Spotlight). Videos at https://pathak22.github.io/modular-assemblies/

Cited by in corpus (17)

• Unity: A General Platform for Intelligent Agents
• DiffAqua: A Differentiable Computational Design Pipeline for Soft Underwater Swimmers with Shape Interpolation
• One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control
• Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems
• Evolving Structures in Complex Systems
• Meta Learning Backpropagation And Improving It
• Decentralized Reinforcement Learning: Global Decision-Making via Local Economic Transactions
• Task-Agnostic Morphology Evolution
• Self-Organizing Intelligent Matter: A blueprint for an AI generating algorithm
• Hierarchically Decoupled Imitation for Morphological Transfer
• Scale invariant robot behavior with fractals
• Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezing
• Learning to Represent Action Values as a Hypergraph on the Action Vertices
• Emergent Hand Morphology and Control from Optimizing Robust Grasps of Diverse Objects
• Neural fidelity warping for efficient robot morphology design
• What Robot do I Need? Fast Co-Adaptation of Morphology and Control using Graph Neural Networks
• Deep learning for bioimage analysis