An Open Source and Open Hardware Deep Learning-powered Visual Navigation Engine for Autonomous Nano-UAVs

Abstract

Nano-size unmanned aerial vehicles (UAVs), with few centimeters of diameter and sub-10 Watts of total power budget, have so far been considered incapable of running sophisticated visual-based autonomous navigation software without external aid from base-stations, ad-hoc local positioning infrastructure, and powerful external computation servers. In this work, we present what is, to the best of our knowledge, the first 27g nano-UAV system able to run aboard an end-to-end, closed-loop visual pipeline for autonomous navigation based on a state-of-the-art deep-learning algorithm, built upon the open-source CrazyFlie 2.0 nano-quadrotor. Our visual navigation engine is enabled by the combination of an ultra-low power computing device (the GAP8 system-on-chip) with a novel methodology for the deployment of deep convolutional neural networks (CNNs). We enable onboard real-time execution of a state-of-the-art deep CNN at up to 18Hz. Field experiments demonstrate that the system's high responsiveness prevents collisions with unexpected dynamic obstacles up to a flight speed of 1.5m/s. In addition, we also demonstrate the capability of our visual navigation engine of fully autonomous indoor navigation on a 113m previously unseen path. To share our key findings with the embedded and robotics communities and foster further developments in autonomous nano-UAVs, we publicly release all our code, datasets, and trained networks.

arXiv commentAccepted for publication in Proceeding of International Conference on Distributed Computing in Sensor Systems (DCOSS 2019). arXiv admin note: text overlap with arXiv:1805.01831

References in corpus (2)

• Efficient Optical flow and Stereo Vision for Velocity Estimation and Obstacle Avoidance on an Autonomous Pocket Drone
• Local Histogram Matching for Efficient Optical Flow Computation Applied to Velocity Estimation on Pocket Drones

Cited by in corpus (8)

• Machine Learning for Microcontroller-Class Hardware: A Review
• Agilicious: Open-Source and Open-Hardware Agile Quadrotor for Vision-Based Flight
• Improving Autonomous Nano-drones Performance via Automated End-to-End Optimization and Deployment of DNNs
• A Sim-to-Real Deep Learning-based Framework for Autonomous Nano-drone Racing
• Vision-State Fusion: Improving Deep Neural Networks for Autonomous Robotics
• Deep Neural Network Architecture Search for Accurate Visual Pose Estimation aboard Nano-UAVs
• An Efficient Ground-aerial Transportation System for Pest Control Enabled by AI-based Autonomous Nano-UAVs
• Latency-aware and -predictable Communication with Open Protocol Stacks for Remote Drone Control