decentralised pavement distress detection based on deep

PowerPoint-PräsentationProf. Dr.-Ing. Markus König
GTC Europe 2018 | October 9-11, 2018 | Munich, Germany | Kristina Doycheva
Decentralised Pavement Distress Detection
Based on Deep Learning
Ruhr-University Bochum, Germany
Chair of Computing in Engineering
GTC Europe 2018 | October 9-11, 2018 | Munich, Germany | Kristina Doycheva 2
Ruhr-University Bochum
[radiobochum.de]
Ruhr-University Bochum: Location
of the most
Google
Ruhr-University Bochum: Location
car currently takes about 6 hours and
30 minutes (22nd August 2018)
Google
Roadworks
Google
Pavement distress
State of practice
State of research
Majority of publications related to cracks
Methods that find cracks in the images and methods that are capable of
determining the type of the cracks
Most of the crack classification approaches are based on rules, support
vector machines (SVM) or neural networks
Pothole detection methods proposed only in few publications
Based on image histograms and segmentation
Consider only a certain type of roads
Video data usually stored before actual processing
Tested under ideal conditions (weather, no artifacts such as leaves or oil
marks)
Large amount of stored data
Problem statement and objectives
and cost-effectively detected without affecting
traffic?
processing be reduced? How can we execute
distress detection methods fast enough, so that
real-time processing of the images is possible?
How can distress be detected on various types of pavement surfaces
under diverse lighting conditions?
Is it possible to distinguish between different distress types and, if so, how
can that be achieved?
hardware?
Concept
Concept
Wavelet-based approach
Pre-processing
= erosion; ⊕ = dilation
Wavelet transform
Applied by Zhou et al. [2006] for pavement distress detection
Wavelet transform
Wavelet modulus:
M(p, q) = [HL2(p, q) + LH2(p, q) + HH2(p, q)] 1 2
Binarized modulus:
D p, q = 1 if M p, q ≥ Cth 0 if M p, q < Cth
High-amplitude wavelet coefficient percentage
Wavelet transform
Wavelet-based approach
Wavelet-based approach on GPU
Implementation 256x256 512x512 1024x1024 2048x2048
Sequential 217.407 823.436 3213.158 12509.4564
GPU 0.02226623 0.08221098 0.33134483 1.3884667
Speed-up 9763.97715 10016.1317 9697.32345 9009.54728
Textural features
Repeating patterns of local variation in image intensity [Jain, 1995]
Haralick textural features [1973]
Haralick features
Haralick features
=1
Inverse difference moment (IDM):

Entropy:

neighbours
Haralick features on GPU
Haralick features on GPU
Haralick features
Performance: GLCM
OpenCL 0.101 0.168 0.662 2.623
T im
e i
n m
il li
Haralick features
Sequential C++ 6.85200
Distress detection with hand-crafted features
Case study
Number of
images 19,511 9,876 8,364 231
Distress detection with hand-crafted features
Classified category
Intact pavement 17,992 1519
Concept
Deep Learning: implementation
up or down
Deep Learning: implementation
Deep learning results: detection
Training accuracy in % 94.00 98.00 97.00
Validation accuracy in % 95.16 94.65 96.73
Final test accuracy in % 94.85 93.92 96.08
Training cross entropy 0.21 0.13 0.09
Validation cross entropy 0.13 0.15 0.10
Deep learning results: detection
Intact pavement 3320 174
Deep learning results: classification
Training accuracy in % 97.00 95.00 94.00
Validation accuracy in % 91.18 91.13 92.80
Final test accuracy in % 90.70 90.40 93.30
Training cross entropy 0.13 0.14 0.20
Validation cross entropy 0.27 0.25 0.21
A c c u
C ro
Crack Patch Pothole Intact
Classified category
Crack 1708 86 6 160
Patch 66 1608 2 40
Pothole 0 0 34 0
Intact 84 44 22 3756
Average prediction confidence
ac = ∑pc / nc
ai = ∑pi/ni
pc = the confidence for a specific class when the top prediction matches this
class
nc = the number of correctly classified images of this class
pi = the confidence for a specific class when the top prediction does not match
this class (i.e., the confidence for false positives for the specific class)
ni = the number of incorrectly classified images of this class
Average prediction confidence
Average confidence
crack 45% crack 34%
patch 45% patch 37%
pothole 46% pothole 34%
intact 45% intact 38%
State of research and conclusion
Do not distinguish between different types of distress
Consider only a certain type of roads
Video data usually stored before actual processing
Tested under ideal conditions (weather, no artifacts such as leaves or oil
marks)
Large amount of stored data
Outlook
types of distress at the same location
Localizing the distress position within the image -> distress assessment
Integrating pavement distress data into Building Information Models (BIM) for
roads
References
Zhou, J., Huang, P. S. & Chiang, F.-P., 2006. Wavelet-based pavement
distress detection and evaluation. Opt. Eng., 45(2).
Jain, R., Kasturi, R. and Schunck, B. [1995], Machine vision, McGraw-Hill.
Haralick, R. M., Shanmugam, K. & Dinstein, I., 1973. Textural features for
image classification. IEEE Transactions on systems, man and cybernetics,
SMC-3(6), pp. 610-621.
Prof. Dr.-Ing. Markus König
GTC Europe 2018 | October 9-11, 2018 | Munich, Germany | Kristina Doycheva
Decentralised Pavement Distress Detection
Based on Deep Learning
Ruhr-University Bochum, Germany

decentralised pavement distress detection based on deep

Documents