Road Pothole Detection Using Unmanned Aerial Vehicle Imagery and Deep Learning Technique

Authors

  • Dana Mohammed Ali Department of Geomatics (Surveying), College of Engineering, Salahaddin University- Erbil, Kurdistan Region, Iraq
  • Haval A.Sadeq Department of Geomatics (Surveying), College of Engineering, Salahaddin University- Erbil, Kurdistan Region, Iraq

DOI:

https://doi.org/10.21271/ZJPAS.34.6.12

Keywords:

UAV, Deep Learning, YOLO, Pothole Detection

Abstract

     Potholes are considered the main factor for road defects, which leads to road status deterioration, which, consequently will lead to increased road accidents. The first step in road maintenance is to inspect the road surface and then accurately detect potholes. However, manually identifying them is costly and time-consuming. In this study, unmanned aerial vehicle (UAV) imagery was used to create orthophotos of the roads and, using deep learning methods, potholes were detected. The used deep learning method in this study is the "you only look once" (YOLO) algorithm. YOLO is one of the "deep learning-based approaches" to detecting objects and is a single-stage network which requires only one forward propagation across the neural network and focuses on the entire image. The fourth version of YOLO is YOLOV4, which has two different architectures (YOLOv4 and YOLOv4-tiny). Two roads were chosen as the study areas, and to generate the orthophotos of the roads, UAV was used to acquire images. To train both methods in the process of detecting potholes using deep learning, 5300 images were used, 90% used for training and 10% applied for testing. The two used architectures were trained for 6000 iterations. Both methods were evaluated based on the average loss, mean average precision (mAP), and training and testing time. The results showed that the (mAP) values for YOLOv4 and YOLOv4-tiny were 91. 2% and 85.7%, respectively. At the end of the 6000 iterations, the average loss for YOLOv4 is 0.30% and for YOLOv4-tiny is 0.34%. In the training process, YOLOv4 needs 29 seconds for each iteration, while YOLOv4-tiny requires only 8 seconds. In the test process, YOLOv4-tiny is faster at detecting potholes than YOLOv4. The approaches were tested on orthophotos created by processing UAV photos. When comparing the detection of both architectures with visual detection, the results showed that YOLOv4 was able to detect most of the potholes on roads, but YOLOv4-tiny detected a lower number of potholes.

References

ATTOH-OKINE, N. & ADARKWA, O. J. D. C. F. T., UNIVERSITY OF DELAWARE: NEWARK, DE, USA 2013. Pavement condition surveys–overview of current practices.

BAR HILLEL, A., LERNER, R., LEVI, D., RAZ, G. J. M. V. & APPLICATIONS 2014. Recent progress in road and lane detection: a survey. 25, 727-745.

BOCHKOVSKIY, A., WANG, C.-Y. & LIAO, H.-Y. M. J. A. P. A. 2020. Yolov4: Optimal speed and accuracy of object detection.

CHEN, A. Y., HUANG, Y.-N., HAN, J.-Y. & KANG, S.-C. J. J. S. S. S. 2014. A review of rotorcraft unmanned aerial vehicle (UAV) developments and applications in civil engineering. 13, 1065-1094.

CHITALE, P. A., KEKRE, K. Y., SHENAI, H. R., KARANI, R. & GALA, J. P. Pothole detection and dimension estimation system using deep learning (yolo) and image processing. 2020 35th International Conference on Image and Vision Computing New Zealand (IVCNZ), 2020. IEEE, 1-6.

CHOI, J., ZHU, L., KUROSU, H. J. I. A. O. T. P., REMOTE SENSING & SCIENCES, S. I. 2016. Detection of cracks in paved road surface using laser scan image data. 41.

DU, Y., PAN, N., XU, Z., DENG, F., SHEN, Y. & KANG, H. J. I. J. O. P. E. 2021. Pavement distress detection and classification based on YOLO network. 22, 1659-1672.

HUANG, Z., WANG, J., FU, X., YU, T., GUO, Y. & WANG, R. J. I. S. 2020. DC-SPP-YOLO: Dense connection and spatial pyramid pooling based YOLO for object detection. 522, 241-258.

KIM, T., RYU, S.-K. J. J. O. E. T. I. C. & SCIENCES, I. 2014. Review and analysis of pothole detection methods. 5, 603-608.

KOCH, C., GEORGIEVA, K., KASIREDDY, V., AKINCI, B. & FIEGUTH, P. J. A. E. I. 2015. A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. 29, 196-210.

LAURENT, J., HÉBERT, J. F., LEFEBVRE, D. & SAVARD, Y. Using 3D laser profiling sensors for the automated measurement of road surface conditions. 7th RILEM international conference on cracking in pavements, 2012. Springer, 157-167.

PAN, Y., ZHANG, X., CERVONE, G., YANG, L. J. I. J. O. S. T. I. A. E. O. & SENSING, R. 2018. Detection of asphalt pavement potholes and cracks based on the unmanned aerial vehicle multispectral imagery. 11, 3701-3712.

PATRA, S., MIDDYA, A. I., ROY, S. J. M. T. & APPLICATIONS 2021. PotSpot: Participatory sensing based monitoring system for pothole detection using deep learning. 80, 25171-25195.

REDMON, J., DIVVALA, S., GIRSHICK, R. & FARHADI, A. J. A. P. A. 2015. You look only once: unified real-time object detection.

SALINI, R., XU, B. & PAPLAUSKAS, P. J. S. O.-T. C. E. J. 2017. Pavement distress detection with picucha methodology for area-scan cameras and dark images. 34-45.

SILVA, L. A., SANCHEZ SAN BLAS, H., PERAL GARCÍA, D., SALES MENDES, A. & VILLARUBIA GONZÁLEZ, G. J. S. 2020. An architectural multi-agent system for a pavement monitoring system with pothole recognition in UAV images. 20, 6205.

TAN, Y. & LI, Y. J. I. I. J. O. G.-I. 2019. UAV photogrammetry-based 3D road distress detection. 8, 409.

TEDESCHI, A. & BENEDETTO, F. J. A. E. I. 2017. A real-time automatic pavement crack and pothole recognition system for mobile Android-based devices. 32, 11-25.

WU, C. Towards linear-time incremental structure from motion. 2013 International Conference on 3D Vision-3DV 2013, 2013. IEEE, 127-134.

ZEYBEK, M., BIÇICI, S. J. T. J. O. R. S. & GIS 2020. Road distress measurements using UAV. 1, 13-23.

ZHAO, L. & LI, S. J. E. 2020. Object detection algorithm based on improved YOLOv3. 9, 537.

https://dl.djicdn.com/downloads/Mavic_2/Mavic+ 2+Pro+Zoom+User+Manual+V1.4.pdf

Published

2022-12-20

Issue

Section

Agricultural and Environmental Researches