Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learning

Abstract

We demonstrate the possibility of learning drone swarm controllers that are zero-shot transferable to real quadrotors via large-scale multi-agent end-to-end reinforcement learning. We train policies parameterized by neural networks that are capable of controlling individual drones in a swarm in a fully decentralized manner. Our policies, trained in simulated environments with realistic quadrotor physics, demonstrate advanced flocking behaviors, perform aggressive maneuvers in tight formations while avoiding collisions with each other, break and re-establish formations to avoid collisions with moving obstacles, and efficiently coordinate in pursuit-evasion tasks. We analyze, in simulation, how different model architectures and parameters of the training regime influence the final performance of neural swarms. We demonstrate the successful deployment of the model learned in simulation to highly resource-constrained physical quadrotors performing station keeping and goal swapping behaviors. Code and video demonstrations are available on the project website at https://sites.google.com/view/swarm-rl.

arXiv comment14 pages, 11 figures

References in corpus (8)

• Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
• Multi-Robot Collision Avoidance under Uncertainty with Probabilistic Safety Barrier Certificates
• Graph Policy Gradients for Large Scale Robot Control
• Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learning
• Autonomous Drone Racing with Deep Reinforcement Learning
• Downwash-Aware Trajectory Planning for Large Quadrotor Teams
• Neural-Swarm: Decentralized Close-Proximity Multirotor Control Using Learned Interactions
• Neural-Swarm2: Planning and Control of Heterogeneous Multirotor Swarms using Learned Interactions