Deep Drone Racing: Learning Agile Flight in Dynamic Environments

Abstract

Autonomous agile flight brings up fundamental challenges in robotics, such as coping with unreliable state estimation, reacting optimally to dynamically changing environments, and coupling perception and action in real time under severe resource constraints. In this paper, we consider these challenges in the context of autonomous, vision-based drone racing in dynamic environments. Our approach combines a convolutional neural network (CNN) with a state-of-the-art path-planning and control system. The CNN directly maps raw images into a robust representation in the form of a waypoint and desired speed. This information is then used by the planner to generate a short, minimum-jerk trajectory segment and corresponding motor commands to reach the desired goal. We demonstrate our method in autonomous agile flight scenarios, in which a vision-based quadrotor traverses drone-racing tracks with possibly moving gates. Our method does not require any explicit map of the environment and runs fully onboard. We extensively test the precision and robustness of the approach in simulation and in the physical world. We also evaluate our method against state-of-the-art navigation approaches and professional human drone pilots.

arXiv commentAccepted for publication in the Conference on Robotic Learning (CoRL) 2018, Zurich. 10 pages (+3 supplementary)

References in corpus (3)

• Aggressive Quadrotor Flight through Narrow Gaps with Onboard Sensing and Computing using Active Vision
• Deep Neural Network for Real-Time Autonomous Indoor Navigation
• Teaching UAVs to Race: End-to-End Regression of Agile Controls in Simulation

Cited by in corpus (11)

• Visual Model-predictive Localization for Computationally Efficient Autonomous Racing of a 72-gram Drone
• Autonomous Navigation of MAVs in Unknown Cluttered Environments
• Back to Newton's Laws: Learning Vision-based Agile Flight via Differentiable Physics
• Automatic Curriculum Learning through Value Disagreement
• Generalized Hindsight for Reinforcement Learning
• SADA: Semantic Adversarial Diagnostic Attacks for Autonomous Applications
• PIVEN: A Deep Neural Network for Prediction Intervals with Specific Value Prediction
• Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flight
• From explanation to synthesis: Compositional program induction for learning from demonstration
• Soft Multicopter Control using Neural Dynamics Identification
• Learning How to Trade-Off Safety with Agility Using Deep Covariance Estimation for Perception Driven UAV Motion Planning