Road Damage Detection Using Deep Neural Networks with Images Captured Through a Smartphone

Abstract

Research on damage detection of road surfaces using image processing techniques has been actively conducted, achieving considerably high detection accuracies. Many studies only focus on the detection of the presence or absence of damage. However, in a real-world scenario, when the road managers from a governing body need to repair such damage, they need to clearly understand the type of damage in order to take effective action. In addition, in many of these previous studies, the researchers acquire their own data using different methods. Hence, there is no uniform road damage dataset available openly, leading to the absence of a benchmark for road damage detection. This study makes three contributions to address these issues. First, to the best of our knowledge, for the first time, a large-scale road damage dataset is prepared. This dataset is composed of 9,053 road damage images captured with a smartphone installed on a car, with 15,435 instances of road surface damage included in these road images. In order to generate this dataset, we cooperated with 7 municipalities in Japan and acquired road images for more than 40 hours. These images were captured in a wide variety of weather and illuminance conditions. In each image, we annotated the bounding box representing the location and type of damage. Next, we used a state-of-the-art object detection method using convolutional neural networks to train the damage detection model with our dataset, and compared the accuracy and runtime speed on both, using a GPU server and a smartphone. Finally, we demonstrate that the type of damage can be classified into eight types with high accuracy by applying the proposed object detection method. The road damage dataset, our experimental results, and the developed smartphone application used in this study are publicly available (https://github.com/sekilab/RoadDamageDetector/).

arXiv comment14 pages, 7 figures

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• SSD: Single Shot MultiBox Detector
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• An Empirical Evaluation of Deep Learning on Highway Driving
• Automatic Pavement Crack Detection Based on Structured Prediction with the Convolutional Neural Network

Cited by in corpus (21)

• Global Road Damage Detection: State-of-the-art Solutions
• Automatic Crack Detection on Road Pavements Using Encoder Decoder Architecture
• Balanced Semi-Supervised Generative Adversarial Network for Damage Assessment from Low-Data Imbalanced-Class Regime
• Transfer Learning-based Road Damage Detection for Multiple Countries
• Damage detection using in-domain and cross-domain transfer learning
• Road surface detection and differentiation considering surface damages
• YOLO9tr: A Lightweight Model for Pavement Damage Detection Utilizing a Generalized Efficient Layer Aggregation Network and Attention Mechanism
• Deep Transfer Learning for Identifications of Slope Surface Cracks
• Bridge Damage Detection using a Single-Stage Detector and Field Inspection Images
• Integrating GAN and Texture Synthesis for Enhanced Road Damage Detection
• A deep learning based solution for construction equipment detection: from development to deployment
• Investigation of wind pressures on tall building under interference effects using machine learning techniques
• Vehicle Telematics Via Exteroceptive Sensors: A Survey
• Characterization of Semantic Segmentation Models on Mobile Platforms for Self-Navigation in Disaster-Struck Zones
• Pavement Distress Detection and Segmentation using YOLOv4 and DeepLabv3 on Pavements in the Philippines
• CNN Model & Tuning for Global Road Damage Detection
• Classifying Bicycle Infrastructure Using On-Bike Street-Level Images
• FasterRCNN Monitoring of Road Damages: Competition and Deployment
• AI Driven Road Maintenance Inspection
• Model Uncertainty Quantification for Reliable Deep Vision Structural Health Monitoring
• Improving Post-Earthquake Crack Detection using Semi-Synthetic Generated Images