image processing to automate condition assessment of overhead line components

IEEE INTERNATIONAL CONFERENCE ON APPLIED ROBOTICS FOR THE POWER INDUSTRY (CARPI 2010), MONTREAL, CANADA, OCTOBER, 2010 1

Image Processing to Automate ConditionAssessment of Overhead Line Components

Wai Ho Li, Arman Tajbakhsh, Carl Rathbone, and Yogendra Vashishtha

Abstract—Condition monitoring of overhead electricity trans-mission line assets is essential to network operation. Traditionally,the condition of overhead lines are assessed visually. Visualinspection is difficult to apply to phase conductors due to theirheight above ground. As such, aerial imaging surveys seem tobe an ideal solution to this problem. However, the large numberof high resolution images generated by aerial surveys are costlyto inspect in terms of time and labour. This paper presents animage processing system that automates conductor localizationand spacer detection in order to reduce the work required invisual inspection. The implemented system was tested on overfour thousand video images from actual aerial surveys of quad-conductor transmission line assets. Experimental results showhighly accurate conductor localization and a robust hit rate forspacer detection. These results suggest that image processing canbe used to help automate labour intensive tasks in the conditionassessment of overhead line components.

Index Terms—Image processing, computer vision, conditionmonitoring, condition assessment, conductor localization, spacerdetection, automation, aerial survey, aerial imaging, line detection

I. INTRODUCTION

SP AusNet monitors the condition of its overhead electricity

transmission line assets to ensure the safe, cost effective

and reliable operation of its network. Prioritisation of mainte-

nance works requires decision making systems which rely on

knowledge of the asset base condition that is current, complete

and of sufficient quality.

Bare overhead ACSR conductors and conductor fittings

such as spacers, dampers and joints are generally expected

to remain in service for a period of 50-75 years but this

can vary substantially. Lines traversing benign environments

are expected to last longer, whereas lines in more aggressive

environments or defective lines can suffer from early or

accelerated deterioration.

The condition of overhead line assets has traditionally

been assessed primarily using visual techniques. Difficulties

arise when trying to inspect phase conductor visually due to

its height above ground, electrical hazards and inaccessible

underlying terrain. SP AusNet proposes to develop an image

capture system which uses helicopter-mounted high resolution

Dr. Wai Ho Li works at the Department of Electrical and ComputerSystems Engineering, Monash University, Clayton, Melbourne, Australia;Email: [email protected]

A. Tajbakhsh works in the Asset Innovation and Research group at SPAusNet, Australia; Email: [email protected]

C. Rathbone works in the Lines Asset Management group in SP AusNet,Australia; Email: [email protected]

Dr. Y. Vashishtha is Asset Innovation and Research manager at SP AusNet,Australia; Email: [email protected]

video cameras to capture a continuous stream of digital images

of overhead conductors and overhead line components.

An obstacle to the widespread deployment of such an image

capture system is the impracticability of manually inspecting

the images returned by photographic surveys. The proposed

system is capable of capturing multi-megapixel images at

around 5 frames per second. Photos collected over the course

of a single day’s survey can number in the hundreds of

thousands and requires a team of engineers many days to

inspect, significantly increasing the cost. Manual inspection

of digital images over long periods of time carries the risk

of operator fatigue and lacks repeatability when conducted by

different inspectors.

Computerized image processing can vastly reduce the man-

ual labour required to perform image inspections. From license

plate recognition to fingerprint matching, image processing has

been successfully used to transfer labour intensive tasks to

the tireless computer. Therefore it is proposed that in parallel

to the development of the image capture system an image

processing capability be developed to automatically detect

defects and assess asset condition.

Through its ability to provide comprehensive assessment of

aerial conductors and conductor fittings, the use of automatic

image processing of aerial photography promises to revolu-

tionize electricity network asset inspections in the near future.

It is anticipated that our automated image processing capability

will extend to images captured with other platforms, such as

line inspection robots, unmanned aerial vehicles (UAV) and

ground vehicles. Apart from improving the efficiency of offline

transmission line inspection, the research presented here also

opens up the possibilities of real time image processing for

condition monitoring, repairs and vegetation management.

II. AUTOMATIC IMAGE PROCESSING

A. System overview

The proposed image processing system is outlined in Fig-

ure 1. This paper details the Conductor Localization and

Spacer Detection modules, including experimental results on

real world aerial survey images. Note that the conductor

localization module also performs preprocessing in converting

high resolution survey video into lower resolution grayscale

images. Both modules are written in C/C++ using OpenCV [1].

Dashed lines indicate planned future work.

The user specifies the number of conductors he or she

expects to see in the video images. Images where an unex-

pected number of conductors is found, such as detecting only

3 conductors for a quad-conductor line, will be returned by


Digital Imagesfrom AerialSurvey Video

ConductorLocalization Spacer Detection

Other ComponentDetection Methods

Fig. 1. Image processing system overview

our system for manual inspection. Only images where the

expected number of conductors are localized will be passed

onto the spacer detection module. The spacer detection results

are intended for manual inspection. The goal is to drastically

reduce the number of images that an engineer must inspect

when he or she wishes to check a particular type of overhead

line component. For example, in the experiments presented in

Section III our test data contains over four thousand images but

spacers are only seen in 211 video frames. Note that the image

processing steps presented below are automatic and requires

no user intervention apart from a priori specification of the

number of expected conductors.

B. Conductor localization

We define conductor localization as the automatic process

of locating the conductor(s) in a transmission line aerial

survey image. This process is the lynch pin of subsequent

processing as many line components are physically attached to

the conductor. Given that the camera field of view used during

image capture is quite narrow, conductor localization can be

treated as the problem of detecting straight lines in a digital

image. Image processing literature contains a plethora of line

detection techniques, from Hough transform [2] to multi-scale

line detectors using image pyramids [3]. After comparing the

performance of several widely used techniques, we decided to

apply a template matching approach for conductor localization.

Let us first examine the conductor localization results in

Figure 2, which will provide some insight into why we chose

a template matching method.

Fig. 2. Conductor localization by template matching (Left: Input image;Right: Conductor localization results in red)

The left image in Figure 2 illustrates some of the real world

challenges faced by our conductor localization algorithm.

Detecting the correct conductors is a non-trivial problem given

the large number of line features in the input image. Notice

that some of the visual noise is actually supplied by other con-

ductors in the background. There are also several line features

oriented in parallel to the conductors. Despite the visual noise,

our template-based approach is able to automatically localize

the correct quad-bundle of transmission conductors.

The results in Figure 2 help illustrate why we decided to use

a template-based localization approach. Conductor detection

approaches such as edge detection or Gabor filtering followed

by Hough transform [4] runs the risk of detecting too many

lines, which undermines the goal of automation as a way to

reduce manual labour during inspection. The computational

costs of Hough transform can also be prohibitive. Similarly,

multi-scale approaches will return many false positives, unless

we restrict their scale such that only lines of a similar width

to the conductors are found. This is why we decided to use

a template matching approach, where some tolerance in line

width is provided through the use of a blurred template.

Conductor localization is performed by sliding a line tem-

plate vertically down the input image and matching it against

the image pixel values. Matching is performed in grayscale

using a template with the same width as the input image. To

account for conductor orientation, the input image is rotated

over multiple orientations with template matching performed

for each orientation. The orientation that provides the strongest

matching response tells us the overall conductor orientation.

For each orientation, the template matching provides a ver-

tical vector of Normalized Cross Correlation (NCC) values,

the maxima of which represents the vertical coordinates of

possible conductors. Non-maxima suppression is applied to the

vector of NCC values to detect and localize the conductors.

The maximum number of conductors that should be localized

is specified by the user.

Note that in most aerial survey videos, not all images

contain the expected number of conductors. One or more

conductors can be out of the camera’s field of view. For

example, some survey images were captured when the camera

is aimed at transmission towers resulting in no conductors

being seen. A threshold on the NCC result is used to identify

images were no conductors are present so that they can

be returned for manual inspection. In addition, a dynamic

threshold is applied on a frame-by-frame basis to find images

that do not contain the expected number of conductors. The

threshold is calculated as a ratio of the maximum NCC peak

for each image. These images are also returned for manual

inspection.

C. Spacer detection

This section describes our spacer detection algorithm, which

detects the presence of a quad-conductor spacer in an image

and localizes the spacer by returning a bounding rectangle. The

detection process begins by cropping and rotating the original

image using the conductor localization results so that it only

contains the conductors with some vertical padding. Then a

Gabor filter [5] is applied to the cropped image to find features

spanning the set of conductors. From here on in, we shall refer

to the results of Gabor filtering designed to find spacers as


the spacer Gabor image. Spacers are detected by looking for

large clusters of pixels that respond strongly to Gabor filtering.

An example illustrating the spacer detection steps is shown in

Figure 3.

One can imagine that spacer-like features in the background

may cause false positives. To reduce false positives caused by

background objects that lie across the conductors, we use the

fact that a spacer disrupts the usually smooth contours of a

conductor at the location where it attaches to the conductor.

Figure 4 shows conductor-spacer joints detected automatically

by our image processing system.

Figure 4 also shows that one or more conductor-spacer

joints can be missed due to occlusion and image noise. As

such, our algorithm only requires the presence of one or more

disruptions as sufficient proof that a spacer is present. Figure 5

shows an example where despite a strong spacer Gabor re-

sponse, which would normally result in an incorrectly detected

spacer, our algorithm automatically rejected this potential false

positive due to the lack of conductor-spacer joints.

III. EXPERIMENTAL RESULTS

Our image processing system was tested on 6 video se-

quences of quad-conductor transmission lines containing a

total of 4437 four-megapixel images. The same parameters

are used across all six video sequences and all processing was

carried out automatically.

A. Conductor localization

Conductor localization results were analyzed manually by

going through the results frame-by-frame. Successful localiza-

tion is defined as having the detected conductor lying on top

of the actual conductor in a manner that is visually correct.

As survey images are usually inspected manually, the authors

feel this method of validation is apt.

There were 4349 test images where all four quad-conductors

are within the camera’s field of view. Our system was able to

automatically localize all conductors in these images. How-

ever, 13 false positives were also detected along a short span of

video due to a combination of insulators and background line

features. Some false positive examples are shown in Figure 6.

The remaining 75 video frames where one or more conductors

were absent from the camera image were correctly returned for

manual inspection. As only the false positives are incorrectly

labelled, it can be said that our conductor localization method

achieved an accuracy of 99.71%.

B. Spacer detection

The spacer detection module operated on 4362 images as 75

images were returned for manual inspection by the conductor

localization step. Note that false positives from conductor

localization were carried over to the spacer detection step.

Again, we manually inspected the video frames to find ground

truth. In this case, we found 136 actual physical spacers, some

of which are visible across multiple consecutive frames. The

spacers can be seen in 211 images, including cases where a

spacer is only partially visible.

Fig. 3. Spacer detection steps. From top to bottom: Input image, input afterautomatic cropping and rotation, spacer Gabor image, detection results


Fig. 4. Detecting disruptions caused by conductor-spacer joints in order toreject false positives. For the sake of continuity, the same input image fromFigure 3 is used to illustrate the process. Top: Cropped and rotated inputimage. Bottom: Conductor-spacer joint detection results as red squares

Fig. 5. False positive spacer detection successfully prevented by checkingfor conductor-spacer joints. Top: Input image. Bottom: Spacer gabor

Fig. 6. Examples of false positive conductor localizations

Our system was able to correctly detect spacers in 204

images, resulting in a hit rate of 96.68%. However, the system

also returned 47 additional false positives. This equates to a

false alarm rate of 1.13%. The majority of false positives can

be attributed to three main causes. Firstly, other objects on

the conductor such as dampers can appear like spacers in the

Gabor filtered image. Secondly, frames where the conductors

terminate or change appearance mid-frame can result in false

spacers being detected. These frames tend to occur where the

conductor arrives at a tower. These two cases of false positives

are not of major concern as they are frames that will require

manual inspection for separate reasons. Thirdly, unforunate

configurations of background features can also result in a

false positive. Background objects caused 10 of the 47 false

positives and will be a focus of future research. False positive

examples are shown in Figure 8.

As mentioned earlier, there were 7 missed spacer images.

Four missed images occurred because the spacer was only

partially visible at the border of the image. These images can

be seen in the first row of Figure 9. Note that in all instances,

the same physical spacer is found automatically by our system

in the subsequent video frame.

The remaining 3 missed spacers occurred because the

conductors and the spacers are too far away from the camera.

This meant our system cannot identify the spacers as it was

not able to detect the conductor-spacer joints due to their small

size. These images are presented in Figure 7.

Fig. 7. Missed spacers caused by failure of conductor-joint detection due tolarger than expected camera-conductor distance

C. Timing analysis

Timing analysis was performed across all test images.

Timing was conducted on a desktop PC with an Intel Xeon

CPU (2.27 GHz) and 2GB of RAM. The C/C++ code was

compiled using Microsoft Visual C++ 2008 compiler with

optimization disabled (/Od) and running under Windows XP


TABLE IAVERAGE PROCESSING TIMES (MS)

Preprocessing 52.6

Conductor localization 204

Spacer detection 16

Total 272.6

Professional (Service Pack 2). The average processing times

across the test image set are detailed in Table I.

Preprocessing includes loading images from hard drive

into memory as well as decompressing and resizing images

for subsequent processing. Conductor localization and spacer

detection includes all tasks described in Sections II-B and

II-C respectively. The total processing time across all test

data (4437 images) was roughly 20 minutes. Processing time

can be reduced by enabling compiler optimizations and par-

allelizing computations given that images can be processed

independently.

IV. CONCLUSIONS AND FUTURE WORKS

The conductor localization results clearly show that tem-

plate matching is a robust means to identify line features of

roughly-known width in aerial imagery across varying lighting

conditions and backgrounds. However, ground truth locations

of conductors is needed for more quantitative analyses of the

localization results. Detailed comparisons against competing

line detection methods is planned as future work.

Despite the presence of false positives in spacer detection,

the number of images returned by the system for manual

inspection is an order of magnitude less than the quantity

of test images. This promises large efficiency gains for the

manual inspection of overhead components. The number of

false positives can be reduced by the rejection of dampers

and tuning of parameters such as the number of expected

conductor-spacer joints. A simple aspect ratio constraint on

the width and height of detected spacers can also reject many

false positives found in the test set.

Computationally, the system is running near real time. En-

abling compiler optimizations will further increase processing

speed. The majority of computations is spent on template

matching, which can be sped up through the use of FFT-

based NCC as well as parallelization using multiple CPU cores

or GPU architecture. Overall, the preliminary results suggest

that image processing can help automate tedious and labour

intensive tasks that are a regular part of condition assessment

of overhead line components. Similar image processing tech-

niques may also hold promise in automating surveys carried

out via non-aerial means.

ACKNOWLEDGMENTS

The authors thank Geoff Fairweather and David McLennan

of Sp AusNet for making time to share their knowledge of the

transmission network.

REFERENCES

[1] “Opencv,” http://opencv.willowgarage.com/wiki/.[2] R. O. Duda and P. E. Hart, “Use of the hough transformation to detect

lines and curves in pictures,” Communications of the ACM, vol. 15, no. 1,pp. 11–15, 1972.

[3] T. Lindeberg, “Edge detection and ridge detection with automatic scaleselection,” in IEEE Computer Society Conference on Computer Visionand Pattern Recognition, San Francisco, USA, June 1996.

[4] C. Mu, J. Yu, Y. Feng, and J. Cai, “Power lines extraction from aerialimages based on gabor filter,” in International Symposium on SpatialAnalysis, Spatial-Temporal Data Modeling, and Data Mining, Y. Liuand X. Tang, Eds., vol. 7492, no. 1. SPIE, 2009, p. 74923P. [Online].Available: http://link.aip.org/link/?PSI/7492/74923P/1

[5] J. Daugman, “Two-dimensional spectral analysis of cortical receptive fieldprofiles,” Vision Research, vol. 20, no. 10, p. 84756, 1980.


Fig. 8. Examples of spacer detection false positives. Columns from left to right: False positives caused by dampers, mid-frame conductor termination andbackground features. Rows from top to bottom: Input image showing conductor-spacer joints, spacer Gabor image, detection results

Fig. 9. Missed partially visible spacers (first row) and the same spacer automatically detected in the subsequent video frame (second row)

image processing to automate condition assessment of overhead line components

Documents