Learning a Decentralized Multi-arm Motion Planner

Abstract

We present a closed-loop multi-arm motion planner that is scalable and flexible with team size. Traditional multi-arm robot systems have relied on centralized motion planners, whose runtimes often scale exponentially with team size, and thus, fail to handle dynamic environments with open-loop control. In this paper, we tackle this problem with multi-agent reinforcement learning, where a decentralized policy is trained to control one robot arm in the multi-arm system to reach its target end-effector pose given observations of its workspace state and target end-effector pose. The policy is trained using Soft Actor-Critic with expert demonstrations from a sampling-based motion planning algorithm (i.e., BiRRT). By leveraging classical planning algorithms, we can improve the learning efficiency of the reinforcement learning algorithm while retaining the fast inference time of neural networks. The resulting policy scales sub-linearly and can be deployed on multi-arm systems with variable team sizes. Thanks to the closed-loop and decentralized formulation, our approach generalizes to 5-10 multi-arm systems and dynamic moving targets (>90% success rate for a 10-arm system), despite being trained on only 1-4 arm planning tasks with static targets. Code and data links can be found at https://multiarm.cs.columbia.edu.

CoRL 2020

References in corpus (8)

• Continuous control with deep reinforcement learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols
• From Perception to Decision: A Data-driven Approach to End-to-end Motion Planning for Autonomous Ground Robots
• Emergent Tool Use From Multi-Agent Autocurricula
• Intelligent bidirectional rapidly-exploring random trees for optimal motion planning in complex cluttered environments
• Potentially Guided Bidirectionalized RRT* for Fast Optimal Path Planning in Cluttered Environments
• Fast, High-Quality Dual-Arm Rearrangement in Synchronous, Monotone Tabletop Setups

Cited by in corpus (5)

• Long-Horizon Multi-Robot Rearrangement Planning for Construction Assembly
• Nothing But Geometric Constraints: A Model-Free Method for Articulated Object Pose Estimation
• A Multi-Agent Approach for Adaptive Finger Cooperation in Learning-based In-Hand Manipulation
• DAIR: Disentangled Attention Intrinsic Regularization for Safe and Efficient Bimanual Manipulation
• Distilling Motion Planner Augmented Policies into Visual Control Policies for Robot Manipulation