v_reportautomatic rail track inspection and assessment

Automatic Rail track inspection and assessment 2015

T ABLE O F C ONTENTS

Acknowledgement iii

Chapter 1: Introduction

Background and motivation…………………………………………………………………………………….…1

Chapter 2: Current Scenario 2

Chapter 3: Techniques used in rail track detection 3

3.1: Long Range Ultrasonic Testing (LRUT)……………………………………4

3.2: LED-LDR Assembly…………………………………………………………4

3.3 : Railway Machine-Vision Inspection Systems……………………………5

3.4 : Train-Mounted GPR……………………………………………………………………………………..6

Chapter 4: DATA COLLECTION………………………………………………………..8

4.1: Image and Video Acquisition…………………………………………………8

4.2: Virtual Track Model for Initial Algorithm Development …………………….9

Chapter 5: ALGORITHM DEVELOPMENT AND DATA ANLYSIS SYSTEM...........12

Chapter 6: Conclusion……………………………………………………………………….……19

Chapter 7: REFERENCES……………………………………………………………………..21

Dept of Electronics and communication, BMSIT


1.Introduction

1.1 Background and motivation

North American Railways and the United States Department of Transportation (US DOT)

Federal Railroad Administration (FRA) require periodic inspection of railway

infrastructure to ensure the safety of railway operation. This inspection is a critical, but

labor-intensive task resulting in large annual operating expenditures and it has limitations

in speed, quality, objectivity, and scope. A machine vision approach is being developed

to automate inspection of specific components in the track structure. The machine vision

system consists of a video acquisition system for recording digital images of track and

custom designed algorithms to identify defects and symptomatic conditions from these

images, providing a robust solution to facilitate more efficient and effective track

inspection. The main focus of the system is the detection of irregularities and defects in

wood-tie fasteners, rail anchors, and turnout components. An experimental on-track

image acquisition system has been developed and used to acquire video in the field of

different track classes. The machine-vision algorithms use a global-to-local component

recognition approach, in which edge and texture-based detection techniques are used to

narrow the search area where components are likely to be detected. The system will be

designed to evaluate the railway infrastructure in accordance with FRA track safety

regulations, but will be adaptable to railroad-specific track standards. Some of the track

inspections such as measuring the track’s curvature and alignment, as well as the cross

level of the two rails, have already been automated using a track geometry car; however,

other inspections, such as monitoring the spiking and anchor patterns and detecting raised

or missing spikes and anchors, are still manually and visually conducted by rail road track

inspectors. It is, thus, of great interest to railroad companies to enhance the current

manual inspection process using machine vision technology for more efficient, effective,

and objective inspections. It also helps them lower maintenance costs and increase track

capacity



2.Current Scenario

The prompt detection of the conditions in rails that may lead to a crack or rather a break

now plays a critical role in the maintenance of rails worldwide. The understanding of

these mechanisms is constantly improving and the evolution of a range of complementary

(Non Destructive Testing)NDT techniques has resulted in a number of tools for us to

choose from. Among the inspection methods used to ensure rail integrity, the common

ones are visual inspection, ultrasonic inspection and eddy current inspection. It is a

relatively well understood technique and was thought to be the best solution to crack

detection. However, Ultrasonics can only inspect the core of materials; that is, the method

cannot check for surface and near-surface cracking where many of the faults are located.

Eddy currents are used to tide over this limitation associated with ultrasonics. They are

effectively used to check for cracks located at the surface of metals such as rails.

MPI is also used in the rail industry but there are a number of problems inherent with this

technique, some of which are mentioned below:

• Surface of the rail or component must first be cleaned of

all coatings, rust and so on.

• To get a sensitive reading, contrast paint must first be

applied to the rail, followed by the magnetic particle coating.

• The same inspection must then be carried out in two

different directions at a very slow overall speed.

However, in the Indian scenario, we find that the visual form of inspection is widely used,

though it produces the poorest results of all the methods. It is now becoming widely

accepted that even surface cracking often cannot be seen by the naked eye.



3. Techniques For Inspecting the cracks in Rail Track

3.1 Long Range Ultrasonic Testing (LRUT)Long Range Ultrasonic Testing (LRUT) technique is proposed as a complimentary

inspection technique to examine the foot of rails, especially in track regions where

corrosion and associated fatigue cracking is likely, such as at level crossings. LRUT

technique is found to be suitable for examining inaccessible areas of railway tracks such

as areas where corrosion occurs and susceptible areas of fatigue cracking. In different

parts of the rail section (such as head, web and foot) properties of guided waves are used

and are examined for their capability to detect defects in each part.

A suitable array of transducers is developed that is able to generate selected guided wave

modes in rails which allow a reliable long range inspection of the rail. The characteristics

of ultrasonic guided waves in the rail complex geometrical profile have been identified.

3.2 LED-LDR Assembly

An algorithm for crack detection in rail tracks is uses [9] Light Emitting Diode and

Light Emitting Resistor (LED-LDR) assembly which tracks the exact location of faulty

track. The design proposed by the authors includes LED which are attached to one side

of the rails and the LDR to the opposite side. When there are no cracks i.e. during

normal operation, the LED light does not fall on the LDR and hence the LDR resistance is

high. Subsequently, when the LED light falls on the LDR, the resistance of the LDR gets

reduced and the amount of reduction will be approximately proportional to the intensity

of the incident light. Consequently the light from the LED deviates from its path due to

the presence of a crack or a break and there is a sudden decrease in the resistance value

of the LDR. This change in resistance indicates presence of a crack or some other similar

structural defect in the rails. In order to detect the current location of the device in case

of detection of a crack, a GPS receiver whose function is to receive the current latitude



and longitude data is used. To communicate the received information, a GSM modem

has been utilized. The function of the GSM module being used is to send the current

latitude and longitude data to the relevant authority as an SMS. The robot is driven by

four DC motors. If this system is employed only latitudes and longitudes of the broken

track will only be received so that the exact location cannot be known. GPRS module is

used to get exact location of the broken rail track. ARM7 controller is also used owning

to is low cost and less power consumption it also decreases the time used in detecting

cracks.

3.3 Railway Machine-Vision Inspection Systems

Railway applications of machine-vision technology that were previously developed or are

under development at UIUC have three main elements (Figure1). The first element is the

image acquisition system, in which digital cameras are used to obtain images or video in

the visible or infrared spectrum. The next component is the image analysis system, where

the images or videos are processed using machine-vision algorithms that identify specific

items of interest and assess the condition of the detected items. The final component is

the data analysis system, which compares and verifies whether or not the condition of

track features or mechanical components comply with parameters specified by the

individual railroad or the FRA. This component will also record and compare data needed

for trend analysis.

Figure 1. Primary Components of a Machine Vision SystemThe advantages of machine vision include greater objectivity and consistency as

compared to manual (i.e. visual) inspection, and the ability to record and organize large

quantities of visual data in a quantitative format. Gathering and organizing quantitative



data facilitates analysis of the health of track or vehicle components over both time and

space. These features, combined with data archiving and recall capabilities, provide

powerful trending capabilities in addition to the enhanced inspection capability itself.

Some disadvantages of machine vision include difficulties in coping with unusual or

unforeseen circumstances (e.g. unique track components) and the need to control or

augment variable outdoor lighting conditions typical of the railroad environment.

3.4 Train-Mounted GPR

A technique based on Ground-penetrating radar (GPR) is used for obtaining quantitative

information about the depth and degree of deterioration of the track. This paper aims at

automating the processing and interpretation of data to the extent whereby on-site

interpretations may be achieved with minimal intervention of the expert. This is done

through the development of new image and signal processing tools specifically for GPR

data and the range of anomalies found on the track bed. For monitoring track conditions

and other infrastructure assets the most efficient way is by means of a train, which can

collect data for many parameters simultaneously, where possible at normal line speed. A

multichannel ground- penetrating radar system is presented in the paper which is capable

of operating at speeds of up to 200 kmph. A road-rail variant of the system is also

presented which can collect up to 6 simultaneous continuous channels across the track,

and can deliver on-site interpretation of ballast thickness and quality, irregularities, weak

spots and utilities. Novel multivariate signal and image processing techniques are used

that can automatically detect, quantify and map variations in ballast depth and condition.

To enable automatic characterization and classification of regions of interest within the

radargrams, multi-resolution texture analysis techniques are applied



TECHNOLOGY INVOLVED:

Fig 2. Block diagram of the proposed rail inspection system.It has three main elements the first element is the image acquisition system, in which

digital cameras are used to obtain images or video in the visible or infrared spectrum.

The next component is the image analysis system, where the images or videos are

processed using machine-vision algorithms that identify specific items of interest and

assess the condition of the detected items. The final component is the data analysis

system, which compares and verifies whether or not the condition of track features or

mechanical components comply with parameters specified by the individual railroad or

the FRA. This component will also record and compare data needed for trend analysis.

Next, we integrate the evidence from multiple information sources, including cameras,

Global Positioning System (GPS), and distance measurement instrument (DMI), and

apply a global optimization approach to further improve the component detection

accuracy. Both the cross-object spatial constraint, as enforced by the sequential structure

of rail tracks, and the cross frame and cross-view constraints in camera streams are

applied during this optimization process. Finally, anchor conditions are assessed.



4. DATA COLLECTION

4.1 Image and Video Acquisition : Collecting images and video of components to be inspected is a critical part in the

development of the this system. There are important trade-offs between where the

components to be inspected are located in the view, how many components can be seen in

a single view, and also what views are required to perform the desired inspections. Views

of the components must not only show the entire component in its functional situation,

but also be conducive to obtaining measurements during the inspection of these

components. In addition, the cameras must be placed to provide views that permit the

algorithms to consistently and reliably detect the track components of interest.

Once viewing angles are determined, another challenge is to collect images of

components that are deformed or defective. However, due to the scarcity of defects, the

number of violations that can be found locally are far fewer than the examples needed to

properly develop a machine vision system. Therefore, methods for finding or creating

these conditions must be addressed.

Fig. 1 shows the block diagram of the overall data process. Specifically, given four video

streams captured by cameras focusing on four different views of the rails, namely, left

field view, left gauge view, right gauge view, and right field view we first detect all

necessary components from each of them by applying various image and video analytics.

Note that, as the heads of anchors only appear in gauge views, anchors are only detected

from the gauge view streams; however, both tie and tie plate will be detected from all

four video streams. Next, we integrate the evidence from multiple information sources,

including cameras, Global Positioning System (GPS), and distance measurement

instrument (DMI), and apply a global optimization approach to further improve the

component detection accuracy. Both the cross-object spatial constraint, as enforced by the



sequential structure of rail tracks, and the cross frame and cross-view constraints in

camera streams are applied during this optimization process.

4.2 Virtual Track Model for Initial Algorithm Development An important consideration in the development of the image acquisition system is the

placement of cameras to acquire suitable images of desired components in their

functional settings. Securing time to test the image acquisition system on active track

during the developmental phases proved difficult, so a virtual track model (VTM) was

created. The VTM used American Railway Engineering and Maintenance-of-Way

Association (AREMA) recommended practices for the design of track components to

model FRA Class 4 and 5 track and included sections of both tangent and curved track.

(AAR clearance plates were incorporated into the VTM to ensure camera placements

were in feasible locations .

The angles of the virtual cameras were then adjusted until they enabled viewing of the

relevant track components and allowed assessment of the conditions of interest that were

conducive to algorithm development. The VTM camera view experimentation resulted in

the selection of two initial camera views: the lateral view (Figure 4A) and the over-the-

rail view (Figure 4B) (1, 2). The lateral view provides a good view of tie plates, spikes

and anchors. The over-the-rail view provides perpendicular views of the spike and

anchors to combine with the lateral view for increasing the accuracy of the

measurements. In addition, it also provides a view of the ties for future inspection tasks.

A: Lateral view showing view of simulated B: Over-the-rail view showing both sides



track and tie plate of the simulated tie plate and crib ballest Figure 3. Virtual Camera Views

4.3 Track Cart for Field Video Acquisition

Beyond the virtual images, a method to capture video that would be representative of

future cameras attached to a track inspection vehicle, was needed for further development of

the machine-vision inspection algorithms. For this reason, and the need to minimize the use

of high-rail vehicles and mainline track capacity, an experimental data acquisition system

referred to as the Video Track Cart (VTC) was designed for collecting continuous video of

track sections of interest on low-density track.

A: Over-the-rail View B: Lateral View Fig. 4 Initial Camera Views



5. ALGORITHM DEVELOPMENT AND DATA ANLYSIS SYSTEM

5.1 Track Inspection Algorithms Early algorithm development focused on spike and anchor detection and defect

recognition. These algorithms can be summarized as a coarse-to-fine approach for

detecting objects. We first locate the track components with little variability in

appearance and predictable locations (e.g. the rail), and then locate objects that are

subject to high appearance variability (e.g. spike heads and anchors) in subsequent stages.

This increases the robustness of component detection by restricting the search space for

the smaller components, whose appearances can vary.To further increase robustness to

changing environmental conditions and changes in object appearance (e.g. differing

material types or corrosion), we have selected features that do not rely on a specific

spatial description, but rather a configuration of simple, local features that are known to

be valuable in classification. The simple, local features that we use include edges and

Gabor features. Edges are frequently used to detect objects in machine vision since object

boundaries often generate sharp changes in brightness (21). Image gradients (edges)

should be consistent among differing ties and rails, but unanticipated track obstacles

could create unanticipated edges, causing difficulty for the algorithms. For this reason,

texture information from the ballast, tie, and steel was incorporated into the edge-based

algorithm to improve its robustness. This approach relied on texture classification using

Gabor filters, which produced low-level texture features. Gabor filtering is used to

summarize two-dimensional spatial frequencies, and this can be used in texture

discrimination.

5.2 Image Decomposition



Since we operate using a coarse-to-fine approach, we decompose the image beginning

with the rail, which is the largest, most consistently detectable object. Then, we

differentiate ballast texture from non-ballast texture using Gabor filtering. Labeled

examples of ballast, tie, and steel textures were created using previously stored images

(Figure 5). When presented with a previously unseen image, texture patches are extracted

and classified as either “ballast” or “non-ballast”. This classification incurs some errors

due to foreign objects and other image noise, and the patches do not necessarily occur on

object boundaries. Though the boundaries are inexact and the classification imperfect, in

all test images, the tie, rail, and ballast areas were reliably isolated for subsequent

processing.

Figure 5. Template Images of Specific Ballast, Rail, and Tie Textures Used for Image Processing

After isolating the foreground portion of the tie, an accurate boundary for both the tie

plate and tie must be obtained to determine if an anchor has moved from its proper

position. the dimensions of the tie plate can be compared to the image to calibrate its

scale for defect measurement estimations. In order to capture lateral views of the gauge

side and the field side of both rails, we use four cameras in total. All four cameras are

connected on the same FireWire bus, which controls the time-synchronization between

cam-eras with high accuracy. The field of view of each camera are set to 24 inches to

guarantee 50% overlap of images when traveling at 10 mph. For this camera setting, at

each time point, each side of each tie plate is seen by only one camera, a tie will be seen

by all four cameras, and the anchors will be seen only by the two gauge view cameras.

5.3 Tie and Anchor Inspection



5.3.1 Tie Plate Detection:To detect tie plates, we applied the same approach presented in [6]. Specifically:

1. Use Hough transform to detect two dominant horizon-tal lines in the image, which correspond to two horizontal edges of the tie plate.

2. Find the two vertical edges of the tie plate as follows.

For the image region between the two detected horizontal lines, compute its edge map

using the Sobel operator, then sum up the edge magnitude for each column.

For each column, sum up all magnitudes within a window that is centered on it. The

window size approximately equals the width of a tie plate (assumed to be fixed).

Find the two minimums in the above plot, corresponding to the tie plates left and right

vertical edges

.

5.3.2 Tie Detection After tie plates are detected, we implement a simple and robust approach for tie

detection. The upper horizontal edge of the tie is aligned with the upper edge of the tie

plate. The lower horizontal edge of the tie usually is aligned with the bottom boundary of

the image. The remaining task is to identified two near-vertical edges of the tie:

1.Use Hough transform to detect near-vertical lines in close proximity of the vertical

edges of detected tie plates.

2.For each detected vertical line, compute the mean in-tensity difference between its left

image region and its right image region. The intuition is that the tie surface is uniformly

texture, and so is the ballast surface on the two sides of the tie, however the texture of the

tie and the ballast are very different.

3.Select the two lines with max distance computed in step 2, corresponding to the left and

right edges of the tie . Note that if tie plate is not detected in a frame, the search area for

the vertical lines becomes the whole image, in which case tie detection may suffer from

higher false positive rate. We then represent each detected tie as a rectangular bounding

box. Although the polygon formed by the four detected lines are not always rectangular,



it can be closely approximated by a rectangle.

5.3.3 Anchor Detection

Detecting anchors is a crucial step towards detecting anchor defects (shift, spread) and anchor pattern compliance exceptions, which are potential causes of derailment. We implement a learning-based anchor detector based on the Adaboost discriminative classifier. We observe that a long track segment may include multiple subsegments, each with a different type of anchor. Training only one single cascade classifier for all subclasses of anchors would de-crease its discriminative power, due to high in-class variability of anchors. As opposed to the standard Adaboost algorithm that used a single cascade classifier, we em-ploy multiple cascade classifiers, somewhat similar to that introduced in as depicted in Figure. Specifically, we train multiple binary classifiers, each corresponds to a subclass of anchors. For detection, we employ a model-switching mechanism as follows. We keep all classifiers running simultaneously, but at any time point we only return the detection results from one selected classifier - the one with the highest number of detections in the last 50 frames. For each frame, we apply a sliding window detection ap-proach within a ROI, which is defined to be the horizontal image stripe covering the region around the lower edge of the rail, where anchors should be installed. The width of the stripe equals the image width.

5.3.4 Anchor Condition Assessment After anchors are detected and located, the next step is to assess their conditions. An

anchor is considered shifted if it is more than 1 inch away from its associated tie horizontally (Figure 6 (a)). A spread occurs when the horizontal distance between two anchors of the same tie is 4 inches more than the tie width (Figure 6 (b)), i.e.: D1 + D2 = D W 4 inches. Therefore a spread of an anchor pair is automatically obtained by computing the shift values of each anchor in the pair. Both shifts and spreads are considered track defects, since they are strong evidence that the rail at that location is running (unstable).



Figure 6. Definition of anchor shift (a) and spread (b).

Pixel-Inch CalibrationSince shifts and spreads are defined in inches, while the anchor-tie distance is

computed in pixels, we have to be able to reliably convert distances from pixels to

inches in order to accurately detect shifts and spreads. Due to wide angle fisheye

distortion, the pixel to inch mapping is not uniform for all columns in the image. We

take advantage of the fact that the width of the tie plates are fixed at 7:5 inches. We an-

notate the bounding boxes for roughly 3; 000 tie plates for each gauge-view cameras.

Since we’re mainly interested in the horizontal distance, we plot the tie plates’ width

val-ues in pixels with respect to the X coordinate of their loca-tions. We quantize the X

coordinates to 20 different bins. The conversion curve is fitted by performing bin

averaging for each of the 20 bins and followed by linear interpola-tion. The resulted

curve allows us to map an arbitrary im-age pixel to inches, given its column index. The

pixel-inch mapping function roughly approxi-mates a quadratic function. typically be

visible in two frames when traveling at speed, and while the vehicle is still accelerating

components can be detected in many consecutive frames.

Cross-view Matching:At any time point a tie can be seen by as many as four camera views. When the



expected position of a tie plate moves out of the camera’s field of view, we attempt to

match the tie plate with tie plates from other views that have also finished passing the

field of view. Pairs of tie plates are matched by comparing their positions in frames

where both tie plates were present. Tie plates are matched iteratively in an attempt to

build up a set of detections in all four views representing one complete tie. If a

complete set of tie plate detections cannot be found, and some tie plates’ expected

position moves further outside the field of view we assume tie plate detection failed for

the missing view(s) and report the largest set of tie plates found as a complete tie with

missing data. If anchors were found at all four positions on the combined tie, we say

the tie has boxed anchors and can maintain a count of ties with boxed anchors to

evaluate compliance with railroad safety rules.

For any rail track geographical location, we need to obtain the geo-reference data which contain the required anchor pattern for that specific geo-location, which is indexed by milepost and footage, or GPS latitude and longitude. Based on such data, for any given 100-foot track that is captured in the video, we will know exactly what is its target anchor pattern by matching the GPS data. To detect compliance exception, we first count the total number of boxed ties for every 100-foot rail track (denoted by C). A boxed tie is a tie with all 4 anchors in normal (not shifted) condition. We then compare the tie count with the required number (denoted by R). If the count is smaller than 85% of the requirement, i.e. (R C)=R >= 15%, then a compliance-level exception is declared.

5.3.5 Anchor Pattern Compliance Exception Detection:Detecting compliance exceptions for railroad tracks is expected to achieve a high

detection rate and low false pos-itive rate. A compliance exception negatively affects the

rail safety at the sequence level, thus a failure to detect any single one of those can



potentially leads to grave conse-quences. On the other hand, verifying a compliance

excep-tion requires a lot of time and resource for a railroad com-pany, since it involves

visually scanning a 100-foot track segment. With a high false positive rate, it would be

very challenging for human inspectors to scan through all the re-ported exceptions to find

true ones. From our own investigation with railroad companies, the desired false positive

rate for compliance exception detection is 1 false positive per 1 hour of inspection at 95%

detection rate.

Since there are no true exception in the 3-mile track we used in earlier tests, we

performed this test on an 1-hour video captured from a different rail track at a different

time. For this one-hour video, there are 3 genuine compliance exceptions. Our system

detected all of them, achieving 100 % detection rate, while generating 3 false positives

per hour. This is a very promising result compared to the desired performance from

railroad companies.

Experimental Results:

Component Precision Recall Tie plate 99:3% 100% Tie 88.2% 82.3% Anchor 96:5% 96.7%



Conclusion

This paper has described our recent engagement with a railroad company to develop a

real-time automatic vision based rail inspection system. Specifically, the system is able to

robustly detect important rail components with high accuracy and efficiency based on

visual, location, DMI, and contextual information. We have further discussed anchor

exception detection at both tie and compliance levels. Quantitative analysis performed on

a large video data set captured with different track and lighting conditions has

demonstrated very encouraging performance. The main challenge for us in the near future

is to handle scenarios in which heavy shadows and light overexposure exist in the videos.

In addition, we believe that our current tie detection approach needs to be further

improved, and the global component optimization approach needs to be evaluated on

other rail objects other than the tie plate. Third, we will conduct more extensive testing

covering longer railroad tracks with varying defect conditions. Inspection for other rail

objects such as spikes, spike holes, and joint bars needs to be developed as well.

Finally, we need to enhance our algorithms with a potentially

modified imaging system to accommodate a faster and more desirable inspection speed

(e.g., 40 km/h). Controlled illumination of the rail infrastructure will be also explored to

avoid ambient lighting artifacts. Finally, we would like to note the following: 1) the

object detection and optimization approaches that we proposed here can be either applied

readily or with minor tunings to other rail fastening systems and 2) while the vision

algorithms that we developed are finely tuned toward rail track structure and components,

the encouraging results we achieved have demonstrated the applicability of machine

vision technology to real applications in the general transportation domain (such as the

advanced driver assistance system).



REFERENCES

[1] Thomas J. Watson Res. Center, Yorktown Heights, NY, USA “Rail Component

Detection, Optimization, and Assessment for Automatic Rail Track Inspection” In

IEEE International Conference on Computational Intelligence for Homeland Security and

Personal Safety,2014.

[2] H. Trinh, N. Haas, Y. Li, C. Otto, and S. Pankanti, “Enhanced rail component

detection and consolidation for rail track inspection,” in Proc.WACV, 2012, pp. 289–295.

[3] Y. Li, C. Otto, N. Haas, Y. Fujiki, and S. Pankanti, “Component-based track

inspection using machine-vision technology,” in Proc. ACM Int. Conf. Multimedia

Retrieval, 2011.

[4] M. Group, Track Surface Inspection System—TSIS, Nov. 2010.

[5] Y. Li, C. Otto, N. Haas, Y. Fujiki, and S. Pankanti. Component-based track inspection

using machine-vision technology. In ICMR, 2011.

[6] H. Hsieh, N. Chen, and C. Liao. Visual recognition system of elastic rail clips for

mass rapid transit systems. In Proceed-ings of the ASME/IEEE Joint Rail Conference and

Internal Combustion Engine Spring Technical Conference, 2007.

,”


v_reportautomatic rail track inspection and assessment

Documents