Deep Learning for UAV-based Object Detection and Tracking: A Survey

Abstract

Owing to effective and flexible data acquisition, unmanned aerial vehicle (UAV) has recently become a hotspot across the fields of computer vision (CV) and remote sensing (RS). Inspired by recent success of deep learning (DL), many advanced object detection and tracking approaches have been widely applied to various UAV-related tasks, such as environmental monitoring, precision agriculture, traffic management. This paper provides a comprehensive survey on the research progress and prospects of DL-based UAV object detection and tracking methods. More specifically, we first outline the challenges, statistics of existing methods, and provide solutions from the perspectives of DL-based models in three research topics: object detection from the image, object detection from the video, and object tracking from the video. Open datasets related to UAV-dominated object detection and tracking are exhausted, and four benchmark datasets are employed for performance evaluation using some state-of-the-art methods. Finally, prospects and considerations for the future work are discussed and summarized. It is expected that this survey can facilitate those researchers who come from remote sensing field with an overview of DL-based UAV object detection and tracking methods, along with some thoughts on their further developments.

References in corpus (61)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• YOLOv3: An Incremental Improvement
• Simple Online and Realtime Tracking
• Unmanned Aerial Vehicles: A Survey on Civil Applications and Key Research Challenges
• Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark
• Graph Convolutional Networks for Hyperspectral Image Classification
• More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification
• Focal Loss for Dense Object Detection
• Arbitrary-Oriented Scene Text Detection via Rotation Proposals
• An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing
• TextBoxes++: A Single-Shot Oriented Scene Text Detector
• Tracking without bells and whistles
• T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos
• Feature Extraction for Hyperspectral Imagery: The Evolution from Shallow to Deep (Overview and Toolbox)
• Cascaded Recurrent Neural Networks for Hyperspectral Image Classification
• A Review on Deep Learning in UAV Remote Sensing
• SINet: A Scale-insensitive Convolutional Neural Network for Fast Vehicle Detection
• Vision Meets Drones: A Challenge
• Light-Head R-CNN: In Defense of Two-Stage Object Detector
• Object Detection from Video Tubelets with Convolutional Neural Networks
• SlimYOLOv3: Narrower, Faster and Better for Real-Time UAV Applications
• Learnable Manifold Alignment (LeMA) : A Semi-supervised Cross-modality Learning Framework for Land Cover and Land Use Classification
• DetNet: A Backbone network for Object Detection
• CoSpace: Common Subspace Learning from Hyperspectral-Multispectral Correspondences
• ORSIm Detector: A Novel Object Detection Framework in Optical Remote Sensing Imagery Using Spatial-Frequency Channel Features
• Mini-Unmanned Aerial Vehicle-Based Remote Sensing: Techniques, Applications, and Prospects
• Seq-NMS for Video Object Detection
• CornerNet: Detecting Objects as Paired Keypoints
• Simple Online and Realtime Tracking with a Deep Association Metric
• Flow-Guided Feature Aggregation for Video Object Detection
• Pelee: A Real-Time Object Detection System on Mobile Devices
• An Experimental Survey on Correlation Filter-based Tracking
• Aerial Images Processing for Car Detection using Convolutional Neural Networks: Comparison between Faster R-CNN and YoloV3
• Multi-Object Tracking with Multiple Cues and Switcher-Aware Classification
• Learning rotation invariant convolutional filters for texture classification
• Perceptual Generative Adversarial Networks for Small Object Detection
• Efficient ConvNet-based Object Detection for Unmanned Aerial Vehicles by Selective Tile Processing
• Learning regression and verification networks for long-term visual tracking
• Eye in the Sky: Drone-Based Object Tracking and 3D Localization
• RON: Reverse Connection with Objectness Prior Networks for Object Detection
• Vehicle Detection of Multi-source Remote Sensing Data Using Active Fine-tuning Network
• Parallel Tracking and Verifying: A Framework for Real-Time and High Accuracy Visual Tracking
• Fast and accurate object detection in high resolution 4K and 8K video using GPUs
• CFENet: An Accurate and Efficient Single-Shot Object Detector for Autonomous Driving
• Towards High Performance Video Object Detection for Mobiles
• Oriented Response Networks
• Equivariant Transformer Networks
• Looking Fast and Slow: Memory-Guided Mobile Video Object Detection
• Polar Transformer Networks
• Impression Network for Video Object Detection
• Multi-Object Tracking with Siamese Track-RCNN
• HRDNet: High-resolution Detection Network for Small Objects
• PENet: Object Detection using Points Estimation in Aerial Images
• Clustered Object Detection in Aerial Images
• Okutama-Action: An Aerial View Video Dataset for Concurrent Human Action Detection
• Guided Attention Network for Object Detection and Counting on Drones
• Object detection on aerial imagery using CenterNet
• COMET: Context-Aware IoU-Guided Network for Small Object Tracking
• Multiple Object Tracking with Motion and Appearance Cues
• SCNN: A General Distribution based Statistical Convolutional Neural Network with Application to Video Object Detection

Cited by in corpus (5)

• UIU-Net: U-Net in U-Net for Infrared Small Object Detection
• Universal Adversarial Examples in Remote Sensing: Methodology and Benchmark
• Oriented object detection in optical remote sensing images using deep learning: a survey
• Archangel: A Hybrid UAV-based Human Detection Benchmark with Position and Pose Metadata
• BioDrone: A Bionic Drone-based Single Object Tracking Benchmark for Robust Vision