A Sim-to-Real Deep Learning-based Framework for Autonomous Nano-drone Racing
arXiv:2312.08991 · doi:10.1109/LRA.2024.3349814
Abstract
Autonomous drone racing competitions are a proxy to improve unmanned aerial vehicles' perception, planning, and control skills. The recent emergence of autonomous nano-sized drone racing imposes new challenges, as their ~10cm form factor heavily restricts the resources available onboard, including memory, computation, and sensors. This paper describes the methodology and technical implementation of the system winning the first autonomous nano-drone racing international competition: the IMAV 2022 Nanocopter AI Challenge. We developed a fully onboard deep learning approach for visual navigation trained only on simulation images to achieve this goal. Our approach includes a convolutional neural network for obstacle avoidance, a sim-to-real dataset collection procedure, and a navigation policy that we selected, characterized, and adapted through simulation and actual in-field experiments. Our system ranked 1st among seven competing teams at the competition. In our best attempt, we scored 115m of traveled distance in the allotted 5-minute flight, never crashing while dodging static and dynamic obstacles. Sharing our knowledge with the research community, we aim to provide a solid groundwork to foster future development in this field.
8 pages, 10 Figures, 3 Tables, This paper has been accepted for publication in the IEEE Robotics and Automation Letters (RAL). Copyright 2023 IEEE
References in corpus (6)
- Time-Optimal Planning for Quadrotor Waypoint Flight
- Efficient Optical flow and Stereo Vision for Velocity Estimation and Obstacle Avoidance on an Autonomous Pocket Drone
- Autonomous Drone Racing: A Survey
- Improving Autonomous Nano-drones Performance via Automated End-to-End Optimization and Deployment of DNNs
- The Artificial Intelligence behind the winning entry to the 2019 AI Robotic Racing Competition
- Efficient 2D Graph SLAM for Sparse Sensing