robotized inspection of power lines with infrared visionfetter/carpi2010_0699.pdf · 1 robotized...

1

Robotized Inspection of Power Lines with Infrared

VisionJonathan Henrique Efigenio de Oliveira, and Walter Fetter Lages, Member, IEEE

Abstract—This paper presents a method to automaticallydetect faults in transmission lines by using thermographic images.It involves the acquisition of infrared images over a networkand its processing. Since the method is intented to be usedembedded in a robot, low demands on processing power arerequired, in order to enable inspection on large extensions oftransmission lines. The automatic inspection of power lines,allows for inspections over a longer period of time and witha decreased risk for the professionals involved. Experimentalresults show that the proposed method is able to detect cableswith problems.

Index Terms—Filed robotics, Automation, Sensor, Vision, Im-plementation, Inspection, Maintenance.

I. INTRODUCTION

THE methods for inspection of power transmission lines

are expensive, dangerous and inaccurate. This article

presents a method for making preventive inspections of trans-

mission power lines. The preventive inspections are important

as they will be able to detect faults before they occur.

The proposed method is based on thermographic analysis

of the line by using a infrared camera mounted on a robot

that moves on the line. This way, there is a decrease in the

need for maintenance personnel to stay close to energized lines

and therefore an increase in safety, besides enabling a more

accurate process since operator fatigue is avoided.

Typically, problems on cables, connections and even insu-

lators have a direct consequence on their electrical resistance

and therefore on the electrical current through the device,

increasing the temperature around the point with problem.

Note that although the usual behavior of conductors (cables

and connections) and insulators are very different, under

failure both present an increase in temperature. In conductors,

the increase in the temperature is due to an increase in their

electrical resistance, while in insulators, the increase in the

temperature is due to an increase in the leak current through

them. Thus an abnormal increasing in temperature can be

associated to a failing device.

Thermographic inspection has been growing in various

industry segments, since it is a nondestructive method [1],

is fast and does not require contact with the body under

inspection. For electrical plants, it is even more convenient,

since in general it is easier to perform the inspection without

the interruption of system operation.

The visual inspection of power lines, even if just searching

for hotspots in infrared images, is a very monotonous and

Authors are with the Department of Electrical Engineering, Federal Uni-versity of Rio Grande do Sul, Porto Alegre, RS, 90035-190 BRAZIL e-mails:[email protected], [email protected].

tedious task and therefore if done by human beings it is very

prone to fatigue of the operator. The method proposed in this

paper overcomes this problem by performing the analysis of

the infrared images automatically. Also, since the robot is

equipped with a differential GPS receiver, detected problems

are georeferenced so that they can be easily located by

maintenance teams. Furthermore, since the camera is mounted

on a robot moving on the line (aerial or underground), the

image is obtained very close to the line, thus avoiding the well-

known problem of averaging of thermal pixels. The averaging

of pixels can hide hotspots in images taken far from the object

under inspection.

This paper describes the procedures to acquire images from

the camera and the processing of the images to detect potential

problems on conductors and insulators. Details on the robot

can be found on [2]–[5].

The thermographic camera is an IP camera working as a

RTSP (Real-Time Stream Protocol) [6] server, thus the images

are provided as data streams. In order to have individual

images to be automatically processed, the images should be

captured from the received data stream.

The location of each object under risk of failure, obtained

by the robot GPS, is stored along with the image and details of

the analysis for further validation and schedule of maintenance

teams.

This article is organized as follows: Section II presents some

aspects that should be taken into account when processing

thermographic images. Section III addresses the main aspects

of the method developed in this work. Sections V and VI

present the experimental results and conclusions.

II. THERMOGRAPHIC IMAGE

A thermographic image is a digital image formed by elec-

tromagnetic waves in the infrared portion of the spectrum,

rather than in the range corresponding to the visible light, like

images generated by usual cameras. Each pixel represents the

temperature of a point in the objects in image or the energy

emitted by its material.

In this paper a thermographic camera from FLIR, model

A320 is used (see Fig. 1). Its operation is based on mi-

crobolometers and it generates images with resolutions up to

320×240 pixels. It has two temperature ranges: from -20 oC

to 120 oC or 0 oC to 250 oC, providing images in different

formats like standard composite video (NTSC or PAL) or

RAW and MPEG4 over an Ethernet connection.

The camera provides several different streams of the same

thermal image:

2010 1st International Conference on Applied Robotics for the Power IndustryDelta Centre-VilleMontréal, Canada, October 5-7, 2010

978-1-4244-6634-4/10/$26.00 ©2010 IEEE

2

Fig. 1. Thermographic camera FLIR A320.

1) MPEG4 compressed video in three resolutions

(640×480, 320×240, 160×128): Useful only for

presenting an operator view, since the MPEG4

compression imposes some loss on the information

contained on the captured image, usually observed as

pixellization of the image.

2) FCAM FLIR: This stream is in a proprietary format of

the manufacturer. Since details of the format are not

available, it is not useful for the general user.

3) RAW IR signal: This stream provides the raw data from

the infrared sensors. Since the data has not been cali-

brated for the sensors characteristics, and a calibration

pattern for the sensor is not usually available for the

user, this stream is not very useful too.

4) linear IR temperature image in two resolutions

(320×240, 160×120): These streams are thermal images

calibrated in degrees Kelvin with resolutions of 0.1 K or

0.01 K. The stream with resolution of 320×240 pixels

and 0.01 K is the one selected to be automatically

processed, since it can be easily interpreted and has not

distorted the captured data.

In this paper the linear IR temperature image with a

resolution of 320×240 is used. In this format, each pixel is

a 16-bit value representing the temperature in Kelvin of that

point. The precision can be selected to be either 0.1 K or

0.01 K. However, since the camera accuracy is ±2 oC, the

precision can selected much more as a function of the desired

temperature range.

III. IMAGE CAPTURE

The camera behaves as a RTSP server providing a stream

with thermographic images. Stream is a continuous flow of

images wrapped in the Real Time Protocol, RTP [7]. In order

to capture the stream, the processing software should talk to

the camera by using the stack of protocols shown in Fig. 2.

To receive streams, The client should in the first place

establish an RTSP session. This session configures some

parameters to be used for the streaming between the server

and the client. The RTSP protocol refers to the protocol used

to control the operation of the streaming server and provides

commands to select the stream, obtain a description the stream

properties, start, stop, pause and restart the data stream. Table I

Fig. 2. Protocol stack.

TABLE IRTSP COMMANDS SUPPORTED BY THE CAMERA

Command Description

OPTIONS list the supported optional commandsDESCRIBE list the streams available by the cameraSETUP set one RTP session using a specific streamGETPARAMETER get parameters like frame rate and file formatPLAY begin send the streamsPAUSE pause the sendingTEARDOWN close the RTP session

shows the RTSP commands supported by the camera used in

this work.

The connection between the server on the camera and

the client processing software on the robot to transfer

an image stream is called a multimedia session. Before

starting a multimedia session its parameters should be

configured. The Session Description Protocol, SDP [8] is

used to describe the desired session parameters such as

such as session name, media name, address and connection

information. The SDP protocol is used in the setup phase.

In this phase, the server describes the available streams

to the client, which then configures the parameters for

the desired stream. Figure 3 shows the reply sent by

the camera to a RTSP DESCRIBE command. The SDP

session description returned by the camera begins at the

line which starts with v=0. Notice the lines starting with

a=rtpmap: which starts the description of each stream

available from the camera. The stream used in this work

is the one described by a=rtpmap:103 raw/90000,

a=framesize:103 320-240 and a=fmtp:103

sampling=mono; width=320; height=240;

depth=16.

The stream itself is sent though the RTP (Real Time

Protocol) [7]. This protocol defines how the data is packed

to be sent. This protocol enables the client to interpret the

information that it receives and decode the stream in images.

Nowadays, the Ethernet, IP, UDP and TCP protocols are

standard features of most operating systems, but RTSP, RTP

and SDP should be provided by extra software. This work uses

the implementations of the RTSP, SDP and RTP protocols

available through the Live 555 library [9]. That library has

3

RTSP/1.0 200 OK

CSeq: 2

Date: 24 Jun 2010 20:57:18 GMT

Content-Type: application/sdp

Content-Length: 1028

Content-Base: rtsp://10.1.32.1/

v=0

o=- 0 0 IN IP4 10.1.32.1

s=IR stream

i=Live infrared

t=now-

c=IN IP4 10.1.32.1

m=video 13124 RTP/AVP 96 97 98 100 101 103 104

a=control:rtsp://10.1.32.1/sid=96

a=framerate:30

a=rtpmap:96 MP4V-ES/90000

a=framesize:96 640-480

a=fmtp:96

profile-level-id=5;config=000001B005000001B509000001010000012002045D4C28A021E0A4C7



a=fmtp:97

profile-level-id=5;config=000001B005000001B509000001010000012002045D4C285020F0A4C7



a=fmtp:98

profile-level-id=5;config=000001B005000001B509000001010000012002045D4C28282080A4C7

a=rtpmap:100 FCAM/90000


a=fmtp:100 sampling=mono; width=320; height=240; depth=16

a=rtpmap:101 FCAM/90000



a=rtpmap:103 raw/90000



a=rtpmap:104 raw/90000



Fig. 3. SDP session description returned by the camera.

C++ classes to start a session and send RTSP commands

to control the server (the camera). In order to receive an

thermographic image the client should send the commands

to choose the stream (in this case RAW stream with a

resolution of 320×240) and then send the command for the

server start sending the RTP packets. On reception the packets

are extracted from the stream. The payload of each RTP

packet is according to the RFC4715 (RTP Payload Format for

Uncompressed Video) [10]. Each of these arrays is a image

that will be analyzed to detect flaws.

The analysis of the infrared images is based on the detection

of meaningful hotspots and an assessment of the degree of risk

of failure that they represent. Depending on the temperature

of hotspots and its morphology it will be considered medium-

risk, serious-risk or imminent-risk of failure.

IV. FAILURE DETECTION

The purpose of this algorithm is to process the images to de-

tect hotspots and decide weather it represents a problem on the

transmission line or not. Each received image is processed for

inspection, searching for the existence and severity of failures.

The proposed method is based on the Infrared Thermography

Anomaly Detection Algorithm (ITADA) proposed in [11].

The original ITADA was proposed for inspection of elec-

trical equipment in operation and is based on detection of

high temperatures and/or extreme variations in temperature.

Furthermore, it uses a visual 8 bit palette-based image, while

in this paper a 16 bit radiometric image is used (see Fig. 4).

As in many computer vision methods, the first step is the

segmentation of the image. To segment an image is to separate

the target object(s) from the background. In this case, the

background are the areas with low temperatures. Therefore

the result of the segmentation step is an image where only the

interesting high temperatures areas have a temperature value

Fig. 4. Original radiometric image.

Fig. 5. Segmented image.

different from the background. Furthermore, the background

itself is adjusted to have a single value, which can be easily

distinguished from the high temperatures areas, usually 0. The

segmentation is done by thresholding the image with a value

determined in a similar way to [12]. Therefore, all pixels

with temperature below the threshold value is considered

background and has its temperature set to 0 (see Fig. 5).

Let α be the original image(see, for example 9(b)) and β

the binarized version of α, T is the threshold value from the

Otsu method, W and H the width and height of he image, x

and y the coordinates of a pixel. Then, the segmented image

is:

γ(x, y) =

α(x, y), if α(x, y) ≥ T

0, if α(x, y) < T

∀0 ≤ x ≤ W, 0 ≤ y ≤ H

The hotspots are then detected on the segmented image.

4

Fig. 6. Hottest pixels.

First the pixels with highest temperature value are located (see

Fig. 6).

Thot = max0≤W, 0≤H

γ(x, y) (1)

These pixels with highest temperature value are used to all

connected hot ares. Each Thot is used as a seed for a dilation,

denoted by Ω0.

Ω0(x, y) =

1 if α(x, y) = Thot

0 otherwise∀0 ≤ x ≤ W, 0 ≤ y ≤ H

(2)

Then, a dilation operation is used to find connected hot areas

around the hotspots, resulting in a set of hot areas as shown

in Fig. 7:

Ωk = (Ωk − 1 ⊕ B) ∩ C k = 1, 2, 3, .. (3)

where B is an 8-neighbors mask, C is a constraint representing

the experimental limit on the gradient between neighbors

in the foreground image γ (a value of 16 was adopted).

The algorithm converges when Ωk = Ωk−1, resulting in the

converged image Ω∗, which represents the all hotspots in the

thermographic image.

The very small areas with less than 5 pixels are regarded as

noise and are discarded for the purposes of subsequent anal-

ysis. The largest hotspot is used to characterize the situation

of the cable. The temperature of the largest hotspot is defined

by the mean value of its pixels:

Thot =1

D

(

W−1∑

x=0

H−1∑

y=0

γ(x, y), ∀ (x, y) ∈ A

)

(4)

where A represents all pixels of the largest hotspot and D the

number of pixels in the largest hotspot.

The criticality of each detected hotspot is determined by a

look-up table, with values based on standards such from IEC

Fig. 7. Hotspots.

(International Electrical Commission) or ABNT (Brazilian

Association of Technical Standards).

The criticality of a hotspot can be defined by a “quantitative”

or a “qualitative” analysis of its temperature. The quantitative

analysis considers the exact measured temperature of the

hotspot. This method is generally not as important, as the

accuracy of these values are often affected by environmental

factors such as the current ambient temperature, humidity and

emissivity, etc..

The “qualitative” analysis considers the temperature values

for a hotspot in relation to other parts of the equipment with

similar conditions, based on the delta of temperature defined

as

∆T = Thot − Tref (5)

where Tref is the temperature of the component if it were

operating normally. It is computed from the temperature of

pixels not present in any hotspot:

Tref =M

N(6)

where

M =

W−1∑

x=0

H−1∑

y=0

γ(x, y), if Ω∗(x, y) 6= 1 (7)

N =

W−1∑

x=0

H−1∑

y=0

γ(x, y)

γ(x, y), ifΩ∗(x, y) 6= 1 (8)

Table II presents absolute temperature values and Table III

presents relative temperature values used to decide on the crit-

icality of each hotspot. The reference temperature, Tref , takes

into account the voltage levels temperature of the background.

It is generally a value around 70 oC [13]. This value can vary

due to cable manufacturer and climatic conditions. Hence, the

actual temperature of the coldest parts of the image is used as

reference, which represents just a portion of cable that is free

of flaws.

5

TABLE IIQUANTITATIVE ANALYSIS

Condition Temperature Limits (oC)

Normal Thot ≤ 68.3

Not Serious 68.3 < Thot ≤ 76.7

Medium 76.7 < Thot ≤ 85.0

Serious 85.0 < Thot ≤ 98.9

Emergency 98.9 < Thot

TABLE IIIQUALITATIVE ANALYSIS

Condition Temperature Limits (oC)

Normal ∆T ≤ 10

Not Serious 10 < ∆T ≤ 20

Medium 20 < ∆T ≤ 30

Serious 30 < ∆T ≤ 40

Emergency 40 < ∆T

V. EXPERIMENTS

An aluminum cable with 25mm2 section (see Fig. 8(a))

is used in this experiment. It was submitted to a current of

500 A. The thermographic image is shown in Fig. 8(b). In

this image it is possible to see a warm region, detected as the

hotspot shown in Fig. 8(c), but this region have an acceptable

temperature. It is just about 5 oC warmer than the cold areas.

The qualitative and quantitative analysis results in a no fail

decision.

Then the very same cable, has been damaged, lowering its

conductivity, as shown in Fig. 9(a), and a new thermographic

image was obtained as shown in Fig. 9(b). Note that at the

failure point there is an increase in the temperature relative

to the previous image. The temperature on the cable in the

first image is around 27 oC, the temperature at the point

where the failure occurs is around 42 oC. The here proposed

method detected the region where the damage occurred as

a hotspot (see Fig. 9(c)). This temperature does not pass

the quantitative limits test. However, the qualitative analysis

results in a damage.

VI. CONCLUSION

In this paper a method for detection of failures from

thermographic images was presented. The next step is to

submit aluminum cables, like those used in transmission lines,

to currents on the order of hundreds of Amperes, simulate a

damage in it and automatically detect this fault.

ACKNOWLEDGMENT

The authors would like to thank to Companhia Estadual de

Energia Eletrica (CEEE), Coordenacao de Aperfeicoamento de

Pessoal de Nıvel Superior (CAPES) and Conselho Nacional de

Pesquisa (CNPq) for the financial support.

REFERENCES

[1] F. Chunli, S. Fengrui, and Y. Li, “Investigation on nondestructiveevaluation of pipes using infrared thermography,” in Proceedings of the

IEEE International Conference on Terahertz Electronics, vol. 2, Sept.2005, pp. 339–340.

(a) Visual image.

(b) Thermographic image.

(c) Hotspot.

Fig. 8. Aluminum cable without problems.

[2] V. M. de Oliveira and W. F. Lages, “Linear predictive control of abrachiation robot,” in IEEE Canadian Conference on Electrical and

Computer Engineering. Ottawa, Canada: IEEE, May 2006, pp. 1517–1520.

[3] ——, “Predictive control of an underactuated brachiation robot,” in

6

(a) Visual image.

(b) Thermographic image.

(c) Hotspot in the damaged aluminum cable.

Fig. 9. Damaged aluminum cable.

Proceedings of the 8th IFAC Symposium on Robot Control. Bologna:

Elsevier, 2006.[4] ——, “MPC applied to motion control of an underactuated brachiation

robot,” in Proceedings of the 11th IEEE International Conference on

Emerging Technologies and Factory Automation. Prague: IEEE Press,2006.

[5] ——, “Comparison between two actuation schemes for underctuatedbrachiation robots,” in Proceedings of the 2007 IEEE/ASME Interna-

tional Conference on Advanced Intelligent Mechatronics, Zurich, 2007.[6] H. Schulzrinne, A. Rao, and R. Lanphier, “Real time streaming protocol

(RTSP),” Network Working Group, RFC 2326, April 1998, available at〈ftp://ftp.ietf.org/rfc/rfc2326.txt〉.

[7] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, “RTP: Atransport protocol for real-time applications,” Network Working Group,RFC 1889, January 1996, available at 〈ftp://ftp.ietf.org/rfc/rfc1889.txt〉.

[8] M. Handley, V. Jacobson, and C. Perkins, “SDP: Session descriptionprotocol,” Network Working Group, RFC 4566, July 2006, available at〈ftp://ftp.ietf.org/rfc/rfc4566.txt〉.

[9] LIVE555 Streaming Media, Live Networks, Inc., Mointain View, CA,available at 〈http://www.live555.com/LiveMedia〉.

[10] L. Gharai and C. Perkins, “RTP payload format for uncompressedvideo,” Network Working Group, RFC 4566, September 2005, availableat 〈ftp://ftp.ietf.org/rfc/rfc4175.txt〉.

[11] Y.-C. Chou and L. Yao, “Automatic diagnosis system of electricalequipment using infrared thermography,” in Proceedings of the 2009

International Conference on Soft Computing and Pattern Recognition.IEEE Computer Society, 2009, pp. 155–160.

[12] N. Otsu, “A threshold selection method from gray-level histograms,”Systems, Man and Cybernetics, IEEE Transactions on, vol. 9, no. 1, pp.62–66, jan. 1979.

[13] I. E. C. (IEC), “IEC 60826: Design criteria of overhead transmission-lines,” 2006.

Jonathan Henrique Efignio de Oliveira receivedthe B.Sc. in Electrical Engineering from the Uni-versidade Federal do Rio Grande do Sul (UFRGS),Porto Alegre, Brazil, in 2007 and is working towardshis M.Sc. degree in Electrical Engineering at thesame University.

Walter Fetter Lages (S’91, M’99) received theB.Sc. in Electrical Engineering from Pontifıcia Uni-versidade Catolica do Rio Grande do Sul (PUCRS),Porto Alegre, Brazil in 1989 and the M.Sc. and D.Sc.degrees in Electronic and Computer Engineeringfrom Instituto Tecnologico de Aeronautica (ITA),Sao Jose dos Campos, Brazil in 1993 and 1998,respectively. From 1997 to 1999 he was an AdjointProfessor in the Physics Department of the FundacaoUniversidade Federal do Rio Grande (FURG), RioGrande Brazil. Currently he is an Associate Profes-

sor in the Electrical Engineering Department of the Federal University of RioGrande do Sul (UFRGS), Porto Alegre, Brazil. Dr. Lages is a member ofIEEE, ACM, Brazilian Automation Society and Brazilian Computer Society.

robotized inspection of power lines with infrared visionfetter/carpi2010_0699.pdf · 1 robotized...

Documents