Learning High-Speed Flight in the Wild

Abstract

Quadrotors are agile. Unlike most other machines, they can traverse extremely complex environments at high speeds. To date, only expert human pilots have been able to fully exploit their capabilities. Autonomous operation with on-board sensing and computation has been limited to low speeds. State-of-the-art methods generally separate the navigation problem into subtasks: sensing, mapping, and planning. While this approach has proven successful at low speeds, the separation it builds upon can be problematic for high-speed navigation in cluttered environments. Indeed, the subtasks are executed sequentially, leading to increased processing latency and a compounding of errors through the pipeline. Here we propose an end-to-end approach that can autonomously fly quadrotors through complex natural and man-made environments at high speeds, with purely onboard sensing and computation. The key principle is to directly map noisy sensory observations to collision-free trajectories in a receding-horizon fashion. This direct mapping drastically reduces processing latency and increases robustness to noisy and incomplete perception. The sensorimotor mapping is performed by a convolutional network that is trained exclusively in simulation via privileged learning: imitating an expert with access to privileged information. By simulating realistic sensor noise, our approach achieves zero-shot transfer from simulation to challenging real-world environments that were never experienced during training: dense forests, snow-covered terrain, derailed trains, and collapsed buildings. Our work demonstrates that end-to-end policies trained in simulation enable high-speed autonomous flight through challenging environments, outperforming traditional obstacle avoidance pipelines.

arXiv comment16 pages (+7 supplementary)

References in corpus (5)

• Voxblox: Incremental 3D Euclidean Signed Distance Fields for On-Board MAV Planning
• Deep Drone Racing: From Simulation to Reality with Domain Randomization
• Fast, Autonomous Flight in GPS-Denied and Cluttered Environments
• High-speed Flight in an Ergodic Forest
• Does computer vision matter for action?

Cited by in corpus (35)

• Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds
• Agilicious: Open-Source and Open-Hardware Agile Quadrotor for Vision-Based Flight
• Autonomous Drone Racing: A Survey
• Event-aided Direct Sparse Odometry
• Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots
• Learning Minimum-Time Flight in Cluttered Environments
• Collaborative Target Search with a Visual Drone Swarm: An Adaptive Curriculum Embedded Multistage Reinforcement Learning Approach
• Back to Newton's Laws: Learning Vision-based Agile Flight via Differentiable Physics
• Vision-based Learning for Drones: A Survey
• Visual Attention Prediction Improves Performance of Autonomous Drone Racing Agents
• Learning Multi-Pursuit Evasion for Safe Targeted Navigation of Drones
• Fully Onboard SLAM for Distributed Mapping with a Swarm of Nano-Drones
• A roadmap for AI in robotics
• Flying in Highly Dynamic Environments with End-to-end Learning Approach
• Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy
• Vision-State Fusion: Improving Deep Neural Networks for Autonomous Robotics
• Race Against the Machine: a Fully-annotated, Open-design Dataset of Autonomous and Piloted High-speed Flight
• Using Simulation Optimization to Improve Zero-shot Policy Transfer of Quadrotors
• Vision-based Navigation of Unmanned Aerial Vehicles in Orchards: An Imitation Learning Approach
• Towards autonomous photogrammetric forest inventory using a lightweight under-canopy robotic drone
• Microgravity induces overconfidence in perceptual decision-making
• LiCS: Navigation using Learned-imitation on Cluttered Space
• Neuromorphic computing for attitude estimation onboard quadrotors
• Synergistic Reinforcement and Imitation Learning for Vision-driven Autonomous Flight of UAV Along River
• Aerial Field Robotics
• Neuromorphic force-control in an industrial task: validating energy and latency benefits
• Decentralized and Asymmetric Multi-Agent Learning in Construction Sites
• RESC: A Reinforcement Learning Based Search-to-Control Framework for Quadrotor Local Planning in Dense Environments
• iKap: Kinematics-aware Planning with Imperative Learning
• AutoTune: Controller Tuning for High-Speed Flight
• Scalable Outdoors Autonomous Drone Flight with Visual-Inertial SLAM and Dense Submaps Built without LiDAR
• FlightBench: Benchmarking Learning-based Methods for Ego-vision-based Quadrotors Navigation
• WAND: Learning Robust Navigation under Complex Wind Disturbances and Dense Obstacles for Quadrotors
• Vision-Guided Outdoor Flight and Obstacle Evasion via Reinforcement Learning
• iWalker: Imperative Visual Planning for Walking Humanoid Robot