Parallel and Distributed Processing Applicationsin Power System Simulation and Control

Djalma M. Falcão

COPPE'Universidade Federal do Rio de Janeiro

Abstract

Recent advances in computer technology will certainly have a great impact on the methodologiesused in power system expansion and operational planning as well as in real-time control. Pa~allel a~d

distributed processing are among the new technologies that present great potentllu for apphcatlon lU

these areas. Parallel computers use multipie functional or processing units to speed up computationwhile distributed processing computer systems are collections of computers joined together by high speedcommunication networks having many objectives and advantages. This paper discusses the potentialapplication areas af paralIeI and distributed processing in power system and presents. ao overview afthe current research effort in this subject. The papel also presents comments regardmg the researchwork in this area presently in developmewnt in Brazilian institutions and parallel computation facilitiesavailable in Brazil.

1 Introduction

Power system planning and operation in the 1990sface new challenges imposed by a new economic con­text (dereguiation, privatization, etc.), the praeticalutilization of fast aeting e!ectronic contro! devices(FACT's and HVDC controls), and the stretching ofthe e!ectric performance by operation e10se to sys­tem limits. Computation tools to simu!ate and con­trol power systems with these characteristics mustbe powerful enough to deal with increasingiy com­plex modeis and solution possibilities overlooked inthe pasto Moreover, the reduced number of powerengineers graduated in the past decade indicates alikely shortage of power system experts in the nearfuture. To overcome this personnel deficiency, thefew available experts will need eflicient computationtools to properly perform their duties.

Power system computation methods for simula­tíon and contro! have undergone a great deal of de­velopment in the last three decades or 50. Tech­niques for handling large sparse matrices, e1everadaptations like the fast decoupled power tlow meth­od, etc., made possible the analysis of large scalepower system models. A widely accepted guess isthat the introductiol! of new mathematical or pro­gramming methodologies will produce only marginalincrease in the performance of these methods [IJ.Moreover, single processing capacity, although dra­matically increased in the last years, seems to beapproaching physical limits [2, 3, 4J. Therefore,

-The author can be reached at COPPEjUFRJ, Caixa Pos­,al68504, 21945-970 Rio de Janeiro RJ, Brazil, Phone: (021)260 SOlO, Fax: (021) 290 6626, E-ma.il. Ealcao@coep.uírj.br.

substantial improvements are more likely to comefrom the adaptation of the already well developedpower system computation methodologies to the newhardware and software developments being offeredby the computer industry. Parallel and distributedprocessing appear to be among the most promis­ing ones of these new developments [5]-[7J. Parallelprocessing consists in the use of multiple hardwarecomponents to expioit concurrency in the computa­tion job. The main advantage of paraliei process­ing in power system applications is the speed upof computations in order to make viable the solu­tion of problems intractable in conventional comput­ers. Distributed computer systems are collectionsof independent computers joined together by highspeed communication networks having many objec­tives and advantages. Power system applications canbenefit from this form of decentralized computer ar­chitecture due to its geographically distributed na­ture and, as in many other application areas, fromits tlexibility, scalability, cost advantage, etc.

This paper introduces and discusses some ideasfor the use of parallel and distributed processing inpower system simulation and contro!. lnitially, it ispresented a general description of paraliei and dis­tributed processing schemes, their advantages anddisadvantages, as well as promising power systemapplication areas. Next, an overview of the researc!;work currently in development in these topics is pre­sented together with a perspective view of future de­velopments in this area. Finally, the paper presentscomments regarding the research work presently indevelopment in Brazilian institutions and paralieicomputation facilities available in Brazi!.

125

3 ParalleI and Distributed Processing

Fig. 1: Trends in computers and microprocessors perfor­mance

9085

Microprocesson

75 80Year

7065

perf 10or

manc Minicomputerae 1f-L.-r---.,,----,---.--..,

R 1000eIatIV 100e

power system simulation. The preventive controlstrategy, by and large adopted in modem EnergyManagement Systems (EMS), tries to avoid catas­trophic occurrences by a periodic simulation of cred­ible contingencies and system rescheduling and re­configuration. Obviously, in this case a fast solutiontime is essential to allow corrective measures to betaken in due time. The state-of-the-art in real-timecontrol hardware and software is so far only able todeai with steady-state models. Important dynamicphenomena, like transient, small signal, and volt­age stability, are almost completely ignored due tothe lack of powerful enough computation resources.Probabilistic analysis is also usually left out of thereal-time control strategy for the same reason.

Parallel and distributed processing are similarbut rely on different information processing con­cepts. Parallel computers are usually single frameunits in which the idea of concurrent processing isexploited mainly to speed up computations. Dis­tributed computer systems are collections of com­puters joined together with multiple objectives de­pending on the envisaged applications. These dataprocessing approaches have achieved practical imple­mentation mainly due to the relative large increasein microprocessor performance as can be seen in Fig­ure 1 [4) and Table 1.

3.1 Parallel Processing

ParalIei processing can be broadly defined as theutilization of multiple hardware components to per­form a computation job. Severallevels of parallelismcan be found in present day computers but the onesmost important for the applications discussed in thispaper are multiprocessors and multicomputers [3].

A multiprocessor is a computer composed of sev­eral processors sharing a common memory. Amulticomputer is a computer composed of sever­ai processor-memory pairs which communicate be­tween themselves by message passing. Figure 2

2 Power System Simulation and Con­troI

Simulation of power system behavior in the pres­ence of several types of predictable disturbances is afundamental tool in expansion and operational plan­ning and real-time control of these systems. Ow­ing to the complexity of the phenomena involved inpower system operation, simulation studies are usu­ally performed within different time frames: start­ing with the very fast electromagnetic transientscaused by switching operations and Iightning, pass­ing through the slower electromechanical oscilla­tions resulting from power unbalance on rotatingmachines, and eventually reaching the steady-stateanalysis corresponding to different loading condi­tions during the daily, weekly or yearly load cyeIe.Different models are used to represent power sys­tem components in these simulations but some basiccharacteristics appear in ali of them. For instance,the dynamic interaction between apparatus (genera­tors, compensators, etc.) occur via the transmissionnetwork whose representation introduces some sim­ilar structural properties into the models [8, 9, 10].These modeIs usually consist of sets of non-Iinearordinary differential equations, presenting direct in­teraction only to other few equations, and/or sets ofalgebraic equations whose linearized approximationsexhibit similar sparsity pattems.

The power system simulation problems, whencompared to simulation problems found in otherfields of engineering, may be eIassified as mediumsize problems. However, simulation cases are seidomperformed only once. In general, practical expan­sion and operational planning applications requirehundreds or thousands of simulation cases. In someconjectured probabilistic assessment studies, this fig­ure may reach several thousands. Other stringentrequirement of the power system simulation prob­lem is in the case of real-time simulators: unfold­ing the power system's response in the same ratethe physical system would do. These requirementsmake power system simulation a very time 'consum­ing and/or numerically intensive computer applica­tion. To circumvent the lack of enough computerpower to satisfy these high requirements, the usualpractice so far has been to sacrifice accuracy by theintroduction of simplifying assumptions and approx­imated models. This procedure sometimes leads tomeaningless results very often misunderstood by en­gineers not well acquainted with the limitations ofthe used models. Dedicated analog or hybrid de­vices (TNA's and HVDC simulators) have been usedfor the simulation of electromagnetic transients butthey are expensive and less f1exible than fully digitalsimulators.

The real-time control environment, apart fromspecific computation tools, also rely extensively on

126

..Table 1: Microprocessors used in microcomputers, workstations and paraliei computers

Micropro- Manufac- Type Clock Operating Peak performancecessars turer (MHz) systems (MIPS/MFLOPS)

Alpha 21064 DEC RISC 200 WNT,OSFl,VMSMIPS R4400SC MIPS RISC 150 IRIX,SRVR45

PA7100 HP RISC 100 HP-UXPentium Intel CISC 66 WNT,05.2,Unix,etc.

PowerPC 601 IBM/Motorola RISC 80 OS2,AIX,MacOSSuperSparc SUN/Texas RISC 60 Unix SVR4, Sun OS

i860 Intel RISC 66 WNT,OS2,Unix,etc.Transputer IRmos

presents a schematic view of these two forms of par­aliei computer architecture. Other interesting formsof parallelism are the pipelined processors, used inveetor supercomputers, specially tailored for concur­rent operation of loops present in vector and matrixoperations, and the superscalar processors that Usesspecial compilers to optimize the execution of con­ventional programs by executing concurrently someinstruct.ions in a single instruction stream.

Multicomputers, and less extensively multiproces­sors, can be built using commodity hardware in con­trast to, for instance, vector supercomputers thatrequire specially designed processors and memories.In particular, the microprocessors used in microcom­puters and workstations are becoming standard com­ponents of commercially available parallel computersas can be seen in Table 2. For this reason, multi­processors are able to present a price/performanceadvantage over supercomputers based on one or afew customized processors. This fact is driving thehigh performance computing industry towards mas­sive parallelism, i.e., computers with hundreds orthousands of processors. Scalability is another inter­esting feature of parallel computers. Table 2 presentsa sample of presently commercially available parallelcomputers ..

Programs developed for conventional computersusually require relatively exter.sive modificationsto be used in multiprocessars and multicoinputers.Moreover, the best performance of these machinesare usually achieved only with specially developedal'gorithms and programs. Automatic program par­allelization tools are rare in the massive parallel en­vironment and it is doubted weather they will becommonly available in the foreseeable future. For­tunately, a strong research e!fort in universities andindustry around the world in the last years madeavailable a large number of algorithms suitable forparallel implementation [6]. Moreover, parallel com­puter manufacturers are aI. present o!fering devel­opment tools much more efficient than they were afew years ago. On the other hand, it is usually eas­ier to convert programs to run on supercomputersand superscalar processors. Automatic vectorization

compilers are often available for supercomputers al­though the best performance are still achieved onlyby careful hand customization.

The gain obtained in moving an application toa parallel computer has been usually measured interms of speedup and efficiency of the paraliei im­plementation when compared to the best sequentialversion. Speedup is usually defined as the ratio be­tween the execution time of the best sequential codein one processor of the parallel machine to the timeto run the parallel code in P processars. EjJiciencyof the parallelization process is defined as the ratiobetwee'n the speedup achieved on P processors to P.In the early stages of applications development forparallel computers these two indexes were almost ex­clusively the determinants of the quality of parallelalgorithrns. As more parallel machines became com­mercially available, and practical applications beginto be actually implemented, other aspects of theproblem started to become important. For instance,the cost/performance ratio (Mflops/$; Mflops=106

f10ating point operations) attainable in real-life ap­plications. In other cases, although speedup and ef­ficiencies are not se high, the implementation in aparaliei machine is the only way to achieve the re­quired speed in the computations.

3.2 Distributed Processing

A distributed computing system is a collection ofautonomous computers, interconnected by a com­munication network, that work together to satisfythe information processing needs of a company, de­partment, laboratory, etc. [6, 7J. Figure 3 presentsa generic view of a distributed computing system.This type of decentralized computing approach isfeasible today due to the availability of very power­fui desktop computers (microcomputers and work­stations) which can be interconnected by very fastnetworks. Distributed systems are replacing main­frame based computer systems in both commercialand scientific applications generating a phenomenaknown as downsizing of applications.

Distributed computing systems may include a fewcomputers located in a single room, connected by a

127

----------

Mem 1 Mem 2 Mem p...

CPU 1 CPU 2 CPU p

I 1 1Interconnection Network

II I/O Oevices

Multícomputer(Oistributed Memory)

CPU 11 ICPU 2 . . . CPU P

Switching Mechanísm

Memory 1/0Oevices

Multiprocessar

(Shared Memory)

Fig. 2: Parallel Computer Architectures

local area network using coaxial cable as intercon­nection media, or a large number of computers scat­tered hundred of kilometers apart, interconnectedby a wide area network based on telephone lines,microwave channels, optical fiber cables, etc. Thecomputers may be of different types and capacities.No single operational system need to be shared be­tween processors.

Some of the advantages of distributed computingsystems over conventional ones are [7]:

• Cost: the average cost for MIP (million instruc­tions per second) on a mainframe is almost 100times the one for a workstation.

• Scalabi/ity: distributed systems can be expandedincrementally.

client-server system which allows a user to store, ac­cess and manipulate data transparently from manycomputers. This type of system will eventuallyevolve to a truly distributed operating system sothat the entire network appears to the user as a largecomputer system.

Computer

High Speed DataCommunication Network

Fig. 3: Distributed processing

In the last few years, workstation c/usters havebeen gaining acceptance as a platform to implementparallel computing in distributed systems. These aregroups of workstations connected by a local areanetwork which can exchange information by mes­sage passing. Parallel execution of programs can beachieved using client-server programming languagesand tools (Remote Procedure Call-RPC, for in­stance) or specially developed tools like PVM (Paral­lei Virtual Machine). The communication speed be­tween workstations is usually low what makes thesesystems indicated only for applications in whichcomputation on the processors are much more in­tense than communication between processors. Itis claimed that these systems may be able to usethe idle time of a cluster of workstations to perCormhighly intensive computer jobs in the background.

• Flexibi/ity and Configurabi/ity: distributed sys­tems have improved performance and reliabilitydue to redundant data and processing.

• Exploitation of special hardware: special graphicdevices, highly efficient number crunching mod­ules (vector or parallel units), etc., can be addedeasily to the system.

On the other hand, distributed processing is a newand unmarked field. The lack of standards, securityand int.egrity controi difficu!ties, the availability oftoa many options, etc., raise several technical andmanagement concerns.

Distributed applications can be developed accord­ing to several different paradigms [7]. In the sim­plest one, a terminal emulator makes a computerlook like a terminal which is connected to anothercomputer. A file transfer package allows files to betransferred between different computeIS. A moreclaborate modeI is the c/ient-server system in whichremotely located programs can exchange informa­tion in real-time. A distributed data and transactionmanagement system is a sophisticated version of the

ComputerComputer

128

Table 2: Commercially available parallel computers.

Name Manu- Classi- Proces- Topology No.of Peak Performancefacturer fication sor Proc. (GFLOPS)"

Paragon Intel MIND i860 XP Mesh 2d 8-480 36nCube 2 Ncube MIMD Custom Hipercube 2-1024 2.5

CM-5 Tinking MIMD Sun Fat Tree 1-16000 1000Machine Sparc

SPl IBM MIMD PowerRisc 1-64 8T3D Cray MIMD Alpha Torus 3d 32-2048

SIMD 21064 307Exemplar Convex MIMD PA7100 Tightly Coupled 1-128 25

SMM Crossbar

4 The Impact of Parallel and Distrib­uted Processing in Power SystemApplications

From the point of view of potential for parallelprocessing, power system applications can be elassi­fied as:

• Obviollsly Parallelizable: applications that con­sist in the solution of almost independent andcomputationalIy intensive numerical problems asstatic contingency analysis, Monte Carlo simu­lations, eigenvalue analysis, security constrainedoptima! power flow, etc.

• Not Obvioúsly Parallelizable: applications thatrequire ingenious problem decomposition in or­der to achieve a reasonable degree of concurrencyin the solution algorithrns. Most of the applica­tions in this category require the paralIei solutionof large sets of sparse linear algebraic equationsoriginated from different power network rhodelingapproaches. Examples of this elass of applica­tions are the power flow computations, dynamicsimulation for transient and long term stabilityanalysis, etc.

Some applications, like electromagnetic transientscomputation and state estimation, although concep­tually. belonging to the second category, may be in­eluded in the first one due to a nice structure of theequation set to be solved as wilI be discussed laterin this paper. It should be noted that the borderbetween the two categories above is nebulous andlikely to be changing with advancements in paralIelprocessing methods and architectures.

The brief history of parallel processing applica­tion to power system problems registers an appar­ent paradoxo although several important obviouslyparallelizable applications are anilable, most of theresearch effort has been concentrated on applica­tions in the not obviously parallelizable category.An explanation for this phenomenon may be found

in the fact that most of the research in this fieldhas been conducted in universities, were paraliei ma­chines have been available for quite a long time. Ap­plications with not obvious parallelism present moreinteresting challenges from both the theoretical andalgorithm design points of view and, therefore, aremore attractive to academic researchers. ~10re re­cently, this trend has been changing, even amongacademics, as the obviously paralIelizable applica­tions have shown to present as big chalIenges as theother category although seldom in the numerical al­gorithmic leveI.

Major impacto of paralIel processing in practica!terms are expected to occur in areas in which con­ventional computer methods and architectures havefailed so far to produce adequate results or in areasthat are likely to increase their complexity due tothe need of modeling newly introduced equípment.Some candidates are:

• Real- Time Control: dynamic security assessmentand correction, voltage stability assessment, secu­rity constrained optimal power flow, etc. [11,12).

• Real- Time Simulators: e1ectromechanical andelectromagnetic transient simulators to be usedin design and test of new equipment, control andprotection schemes, operators training, etc. [13J-[15].

• Probabilistic Assessment: composite reliabili.yand other probabilistic techniques based onMonte Carlo and enumeration methods using re­alistic static and dynamic models [16]-[18].

Another field that can benefit from paralIei pro-cessing is the automation of power engineering anal­ysis and synthesis studies required both in expan­sion and operational planning. These studies re­quire long and tedious cyeles of case preparation,runs, and analysis of load flow, transient stability,short-circuit, etc. As the availability of qualified en­gineers to perform these studies is diminishing, a

129

Fig. 4: Dynamic Simulation Model Decomposition

x Ax+Bu (3)

I(E, V) = YV (4)

u = h(E, V) (5)

-"-1

5.1 Dynamic Simulation for Transient Stabil.ity Analysis '

The mathematical mode! usually adopted fortransient stability analysis consists of a set of or­dinary non-linear differential equations, associatedto the synchronous machine rotors and their con·trollers, and a set of non-linear algebraic equationsassociated to the transmission network, synchronousmachine stators, and loads [9]. These equations canbe expressed as:

(1)

(2)

f(x,z)g(x,z)o

x

A

where f and 9 are non-linear vector functions: x isthe vector of state variables; and z is the vedor thealgebraic equations variables.

In the model defined in (I) and (2), the differentialequations representing one machine present interac­tion with the equations representing other macbinesonly via the network equations variables. From astructural point of view, this model can be visual­ized as shown in Figure 4: c!usters of generators areconnected by local transmission and subtransmissionnetworks and interconneded among themselves andto load centers by tie-lines.

In the sequential computer context, several solu­tion schemes have been used to solve the dynamicsimulation problem. The main differences betweenthese schemes are in the numerical integration ap­proach (implicit or explicit) and the strategy to solvethe differential and algebraic set of equations (simul­taneous or alternating). Implicit integration meth­ods, particularly the trapezoidal rule, have beenmostly adopted for this application. The most usedschemes are [9J:

1) Atternating Impticit Scheme (AIS): this solu­tion scheme is better understood if (I) and (2) arerewritten as:

• Corporation Integrated Information System: in­tegration of real-time operations with planningfunctions and other corporation functions, suchas accounting, customer services, and manage­ment information systems.

5 Power System Applications

This section introduces a review of paraliei anddistributed processing techniques applied to powersystem problems. This review contains mainly thework developed by the author and his associates.Research work developed by other groups are alsomentioned and reviewed to some extent.

• Distribution A utomation: distribution automa­tion projects have been proposed based on theavailability of processing power distributed along­side the power distribution network with the oh­jective of supervised switching operations, volt­age/var control. load management, customer in­terface automation functions, etc. [5, 21, 22J.

New areas which may benefit from distributedprocessing are [23]:

• Distributed EMS: integration of the traditionalsupervision and control system of the main trans­mission network with the distribution automationfunction and down to the customer leveI.

130

result of both the utilities personnel policies and re­duced interest of electrical engineering students forthe power engineering area, the deve!opment of au­tomated tools to help engineers to speed up thepower engineering studies is becoming an importantrequirement. This type of application exhibits alarge amount of obvious parallel computations (eachpower flow or transient stability case is almost com­plete!y independent from each' other) but requiresa central coordination tool most likely based on in­tel/igent processing techniques like expert systems,neural networks, fuzzy logic, etc.

Two distinct reasons motivate the utilization ofdistributed processing in power system applications:the geographically distributed nature of the powersystem and the recognition by the electrical energyindustry of the cost and operational advantages ofusing networks of microcomputers and/or worksta­tions, based on the open system concept, insteadof the expensive and less flexible proprietary main­frame based systems.

Two applications are already established as viableand being implemented in practice:

• New Architectures for EMS: the traditional dualand similar proprietary architectures of controlcenters, using mainframe or supermini comput­ers, are being substituted by more versatile andcost effective networks of high performance work­stations [5, 19, 20].

[F(x, V') ] k __ [Q 1!-] k [ ~x ] k+1G(x, V') - S Y ~V' (8)

where F is a vector function associated with the dif­ference algebraic equations, G is a vector functionassociated with the algebraic equations, and V' isa vector of nodal voltages expanded in its real andimaginary components, are lumped together in anenlarged set of equations which are, then, solvedsimultaneousiy. The Newton method is the usualchoice of method to solve this set of equations. Inthe k-th iteration of the Newton algorithm, the fo1­lowing set of linear equations has to be solved:

where Q, R, S, and Y are Jacobian submatrices, TheJacobian matrix in (6) cao be ordered in a BlockBordered Diagonal Form (BBDF) [25, 26J and thesystem of equations solved by Block Gauss elimina­-tion .. The elements of the J acobian matrix may bekept constant for several iterations and/or integra­tion steps in a process known as the Very DishonestNewton Method (VDHN).

The difficulties for the parallelization of the dy­namic simulation problem in the AIS are concen­trated on the network solution. The differentialequations associated with the synchronous machinesand their controllers are naturally decoupled andeasily parallelizable. On the other hand, the net­work equations constitute a tightly coup/ed problem

where A is a square, real valued, block diagonal ma­trix in which each block is associated to one machine;B is a rectangular, real valued, blocked matrix inwhich each block is associated to one machine; uis a vector of interface variables; I is a vector of in­jected nodal currents; Y is a square, complex valued,nodal admittance matrix; V is a vector of complexnodal voltages representing the steady-state networkbehavior; E is a subvector of x required to calculatecurrent injections in generation nodes; and h is anonlinear vector function.

The alternating implicit scheme consists in trans­forming the differential equations in difference equa­tions, by the application of an implicit integrationmethod, and to solve these equations iteratively andalternately with the algebraic equations. A mod­ified version of this scheme, called the InterlacedAlternating Implicit scheme [9, 31, 32], is obtainedrelaxing the convergence requirements on the net­work equation solution. The convergence test isperformed in the state variables. This method usu­ally presents better results than the conventional ap­proach.

2) Simultaneous Imp/icit Scheme (5/5): in thisapproach, the network algebraic equations and theaigebrized differential equations

5.1.1 Spatial Parallelization

Methods in this category exploit the structuralproperties of the the network or J acobian equationsto be solved in each integration step of conventionalsimulation schemes (AIS or SIS). Four methods arebriefiy described below:

1) The Parallel VDHN [27J: it consists in astraightforward parallelization of the VDHN methodsimply identifying tasks that can be performed con­currently and allocating them among the processors.This method was implemented in the paraliei com­puters Intel iPSC/2 (distributed memory) and Al­liant FX/8 (shared memory) and tests performedwith the IEEE 118 bus and US Midwestern systemwith 662 buses. The results show speedups slightlysuperior for the iPSC/2 with a strong saturationwith the increase in the number of processors. Themaximum obtained speedup was 5.61 for 32 proces­sors (efficiency = 17.5%).

2) The Parallel Newton- W matrix Method [28]:the main feature of this approach is the use o a par­aliei version of the Sparse Matrix Inverse Factors[29, 30] in the SISo The methodology was tested inthe shared memory Symmetry parallel computer thesame test system used in the work cited in the previ­ous item. The results show a worse performance of·this method wheo compared to the parallel VHDNwith ao slowdown of 10% to 30% depending on thechosen partitions.

requiring ingenious decomposition schemes and solu­tion methods suitable for parallel applications. TheSIS also requires the parallel solution of sets of linearalgebraic equations in every integration step, withdifficulties similar to the ones described for the AIS.

A basic numerical problem in both simulationschemes is the parallel solution of sets of linear alge­braic equations. Direct methods, like LU factoriza­tion, have been dominating this application in con­ventional computers. If parallel computers are con­sidered, however, the hegemony of direct methodsis no more guaranteed. In several other engineer­ing and scientific fields, parallel implementations ofiterative methods have shown superior performance.Among the most successful iterative methods are theones belonging to the Conjugate Gradient (CG) cat­egory [6]. The parallelization of the solution of thenetwork equations requires the decomposition of theset of equations in a number of subsets equal to thenumber of processors used in the simulation. An ad­equate decomposition is fundamental to the successof the parallel solution and need to take into con­sideration factors like computation load balancing,convergence rate of the iterative algorithms, etc.

In the last decade or so, several parallel methodswere proposed for the solution of the dynamic sim­ulation problem. In the following sections, some ofthese methods are reviewed.(6)

(7)

= O= O

F(x, V')

G(x,V')

~-131

=

h(E1, Vilh(E2 , V2 )

"

(11)

(13)

(12)

1,2, ... ,p, solve

Yi Vi = li - Y; V,

p'""' -, -\l, = l, - L..J Y; Y; lii=l

p .'""' -, -1-Y, = Y, - L."Y; Y; Y;i=l

Phase 2: For i

• The number of subnetwork connections is keptto a minimum in order to reduce communiea­tion requirements.

• The subnetworks have approximately the sainedimensions and complexity.

Two approaches were used to obtain the NBDF's ..with the above characteristics: a trial and erro~.... :~<.method based on a careful analysis of the systel!!,!I" :

'-:", .

The solution of the network equations in the form(10-13) can be performed in different ways. Consid­ering the implementation in paraliei machines, themain issue is how to solve Phase 1 efficiently sincePhase 2 presents an inhe~ently paraliei structure.The explicit formation of YI and II and the soiutionof (la) by a direct method could be a straightfor­ward approach to this problem. However, it shouldbe noted that Y, usually presents a less sparse struc­ture than Y,. Moreover, as stated before, the paral­lei implementation of direct methods is not an ea.sytask.

The solution of (la) by the .CGM does no~ re­quire the explicit formation of Y, but only of y! V.0and Í, in the ca1culation of the residual rO and Y,dk

(dk 0,1, ... , are the conjugate directions) ineach iteration. Moreover if the p independent sys­tems given in (13) are solved by a direct method likeLU factorization, the factors of Y;, i = 1, ... , p, can

• o' . k'be efficiently used to ca1cu!ate Y, VI , II and Y, d !n ,_paralieI. 'I''.'.

5) The Full CG Approach [33]: In this method thenetwork equations are solved as a whole by a block- .parallel version of the Preconditioned CG method.The network matrix is decomposed in such a waythat the blocks in the diagonal are weakiy coupiedto each other, i.e., in a Near Block Diagonal Form(NBDF) [25, 26J. The NBDF is equivalent to thedecomposition ofthe network in subnetworks weaklycoupled. A block-diagonal matrix, obtained from theNBDF neglecting the off-diagonal blocks, is used asa preconditioner. To optimize the algorithm perfor­mance, the NBDF should be obtained according,tothe following:

• The number of subnetworks is fixed and equalto the number of processors used in the simu­lation.

(la)

Vp

V,

=y,V,

where:

Phase 1: Solve

(9)The BBDF of the network equations can be

achieved by re-ordering the network nodes in p + 1sets in which the p first sets correspond to subnet­works connected to each other by the boundary busesin the (p + 1)'h set. A semiautomatic decompositionmethod to obtain the BBDF for large networks IS

described in the next section of this paper.Equation (9) can be solved in a two-phase scheme

by Block-GalLssian Elimination as follows:

9) The Parallel Real- Time Digital Simulator [14]:The main objective of this work was the deveiopmentof a real-time power system s,imulator for equipmenttesting based on a massive parallel architecture. Theparallel algorithm is based on the AIS using thetrapezoidal integration method, One processor IS

associated to each network bus, The differentialequations corresponding to each generator and itscontrollers are solved in the processor assigned tothe bus in which the generator is connected, Thenetwork equations are solved by a Gauss-Seidel likemethod aiso allocating one equation to each proces­sor, Therefore, the number of processors requiredto perform the simulation is equal to the numberof buses in the network. Reported results with a261 bus network in a 512 node nCube parallel com­puter show that the required cpu time is not affectedby the system dimensions. However, it is doubtfulweather this property can be kept valid for largersystem taking into consideration that the numberof iterations required by the Gauss-Seidel algorithmincreases considerably with system size. This ap­proach exhibits low speedup and efficiency measuredby the traditional indexes. However, it has. had atremendous impact in the research commumty be­cause it has demonstrated the usefulness of parallelprocessing in solving a real world problem. .

4) The Hybrid CC-LU Approach [31, 32]:. Thlsmethod is based on the AIS, the decomposltlon ofthe network equations in a BBDF, and a hybrid 50­

lution scheme using LU decomposition and the CGmethod. An efficient parallel implementation of theCG method depends on the existence of a convenientstructure of the coefficient matrix. One such struc­ture is the BBDF which is shown in (9) for equation(4).

132

.,

-----------

133

• Each processor may be a cluster of q other pro­cessors which could be used to solve each subsetof equations in parallel in the 50 called space andtime parallelization.

Two CG methods suitable for asymmetric sets of lin­ear equations were experimented in the implementa­tion of the this method: These implementations arereferred to as:

• The Bi-Conjugate Gradient method (Bi-CG) inwhich two sequences of mutually orthogonalresiduais are generated by simple relations sim­ilar to the one used in the CG method [41].

2) The Spaee and Time Gauss-Jaeobi-Bloek­Newton Approaeh [38, 39]: In this approach thewaveform relaxation concept is used with a slightlydifferent formulation. The discretization of the dif­ferential equations is performed for ali integrationsteps simultaneously resulting in an extremely largeset of algebraic equations. In a first work [38), thisset of equations is solved by a paraliei version of theGauss-Jacobi method with a poor performance. Ina second work [39] a method called Gauss-Jacobi­Block-Newton Approach that consists essentially in

• Low communication requirements. Except for theconvergence test, each processor needs to commu­nicate only with processors assigned to integra­tion steps before and after itself.

• the recently introduced more robust and efli­cient variant of the Bi-CG method called the Bi­CGSTAB [42].

Both methods were implemented usingpreconditioning. The preconditioning matrix usedis the following block-diagonal matrix:

where p is the number of integration steps takensimultaneously. Matrix A is asymmetric and verysparse. Assuming that each one of the m syn­chronous machines and its controllers is modeled byK differential equations and that the network has nnodes, the dimension of (7) is (Km + 2n).

1) The Spaee and Time CG Approaeh [33]: Thecalculation of the e!ements of the mismatch functionsF; and C;, the e!ements of the J acobian matrix, andthe updating of the components of x; and V;' in (14)elin be performed with perfect parallelism. There­fore, an obvious mapping of these operations in themultiprocessar system is to aliocate the equationscorresponding to each integration step to a singleprocessor. The decompositionof(14) should be donein the same way. The advantages of this decomposi­tion approach are:

• Perfect processing load balance.

ç -Ab (14)

where

ç= FI CI F2 C2 Fp Cp ]T

b= ~Xl ~V{ ~X2 ~v.' ... ~xp ~Vp' f2

QI RISI y!

Q2! R 21 Q2 R2O O 52 Y2

A=

Qpp-I Rpp-IO O

one-line diagram and the semiautomatic methodol­ogy described in the next sedion.

The Hybrid CG-LU method has shown a fairlygood performance on actual parallel implementa­tions but presents the disadvantages of applying theCG method to a system of equations with rela­ti vely small dimensions and the need of an optimizedBBDF. The Full CG method an attempt to overcomethese difficulties. The CG gradient method is sup­posed to present best performance, both in paralleland vector computers, for very large problems.

The use of preconditioning techniques is advisedfor a better performance of the Hybrid CG-LUmethod and essential for an adequate performanceof Full CG method. Truncated Series precondition­ing [40] is used for Hybrid CG-LU method. For theFull CG method, the Incomplete LU Factorization[6) was chosen as the preconditioning approach afterpreliminary tests with several methods available inthe literature.

5.1.2 Waveform Relaxation

This method [35, 36] consists in the decomposi­tion of the set of equations describing the power sys­tem dynamics into subsystems weakly coupled andto solve each subsystem independently for severalintegration steps to get a first approximation of thetime response. The results are, then, exchanged andthe process repeated. The advantages of this methodare the possibility of using different integration stepsfor each subsystem (multirate integration) and toavoid of the need to solve large sets of linear alge­braic equations.

5.1.3 Space and Time Parallelization

This elass of methods follows the idea introducedin [37J in which the differential equations given in (1)are algebrized for several integration steps, calledintegration windows, and solved together with thealgebraic equations of this window by the Newtonmethod, similarly to the conventlonal one-step ap­proach shown in (56). The resulting enlarged systemof linear equations has the following structure:

t

t

~ ...

the of the VDHN method to the equations associ­ated to each integration step and, then, to apply theGauss-Jacobi globally to all integration steps, is usedwith better results. Both works present results onlyfor simulations of parallel implementation.

5.1.4 Conjugate Gradient Approach Results

The Hybrid CG-LU, Full CG, and Space andTime CG methods described above were tested us­ing different test systems, inciuding a representationof the South-Southern Brazilian interconnected sys­tem with 80 machines and 616 buses. The testswere performed on the iPSC/860 computer and ina prototype parallel computer using the TransputerT800 processor. Despite the difliculties in paralleliz­ing this application, the results obtained in thesetests showed a considerable reduction in computa­tion time. The CG methods presented adequate ro­bustness, accuracy, and computation speed estab­lishing themselves firmly as an alternative to directmethods in parallel dynamic simulation. Moderateefliciencies and speedups were achieved, particularlyin the tests performed on the iPSC/860, which arepartially explained by the relatively low communica­tion/computation speed ratio of the machines usedin the tests. It is believed that in other commerciallyavailable parallel machines, the studied algorithmswill be able to achieve higher leveis of speedup andefliciency.

5.2 Network Decomposition for ParallelBlock-Iterative SolutioDS

A basic step in the solution of sets of linear equa­tions by block-iterative methods is the decomposi­tion of the coeflicient matrix in order to have itmapped onto the parallel architecture. In the caseof electrical networks, this is equivalent to decom­pose or tear apart the network into subnetworks ofsmaller size.

Reference [24J introduces an enhanced version ofa semiautomatic network decomposition method forblock-iterative solutions on parallel computers pre­liminary reported in [34]. The basic operating prin­cipie of the proposed ;nethod is the build ui> of sub­networks from a given number of starting nades (seednades). The subnetworks are forined by aggregatingnodes one by one to these seed nodes. The choiceof which node to aggregate to a given seed node de­pends on a node ranking criterion based on weightsassigned previously to all network nodes. Fig. 5shows an overview of the decomposition method.

The weights used for the node ranking are calcu­lated using the following criterion:

Sul>­Network

Formatian

SeedNadesCnoice

Sul>­Network

Formatian

Selectionof the

Decom positions

Sul>­Network

Formatian

Fig. 5: Decomposition method overview

NadeRanking

where ni is the set of ali branches connected to nodei; nT is the set of ali network branches; B1 is thesusceptance of branch I; nb is the number of networkbranches. The value of Wi indicates the magnitude ofthe electrical interconnection between a node and itsneighbors. The exponent ft is used to increase theinfluence of branches with high susceptance values inorder to differentiate between nodes with severallowsusceptance value interconnections from nodes withfew interconnections with high susceptance value.

In 'each step of the process, the next node to beaggregated to the subnetwork is the one with thelargest weight and still not yet aggregated to othersubnetworks. The process terminates when the sub­networks can aggregate no mOfe nodes. In the al­gorithm to generate the NBDF, ali nodes are aggre­gated to the subnetworks while in the one for theBBDF, the nodes left out of the subnetworks formthe interconnection subnetwork.

The choice of the seed nodes set is a relevantstep in the decomposition method outlined above.The number of seed nodes determines the number ofsubnetworks and their location strongly influencesthe suitability of the decompositions for parallel 50­

lution. Most of the generated decompositions arelikely to be adequate for a solution by a block­iterative method. However, some of these decompo­sitions require less iterations and computation timethan the others, for the same solution method andconvergence tolerance. Therefore, it is desirable tobe able to spot the most promising decompositionsbefore the effective network equation solution. Thiscan be achieved using a Decomposition Selection

(15)

(16)M

Wi

134

method.The decomposition method outlined above, us­

ing a node ranking based on weights calculated ac­cording with (15) and (16). is conducive to block­diagonal dominance of the NBDF's and BBDF'sand, therefore, to the convergence of associated iter­ative processes for the network equation solutions.

5.3 Simulation of Electromagnetic Transients

In the usual model of the power network for elec­tromagnetic transient studies ali components, excepttransmission lines, are modeled by lumped parame­ter equivalent circuits composed of voltage and cur­rent sources, linear and non-linear resistors, induc­tors, capacitors, ideal switches, etc. These elementsare described in the mathematical modei by ordi­nary differential equations which are solved by step­by-step numerical integration, often using the trape­zoidal rule, leading to equivalent circuits consistingof resistors and current sources.

Transmission lines often have dimensions compa­rable to the wave-length of the high frequency tran­sients and, therefore, have to be modeled as dis­tributed parameter elements described mathemati­cally by partial differential 'equations (wave equa­tion). For instance, in a transmission line of lengthl, the voltage and current in a point at a distance xfrom the sending end, at a time t, are related throughthe following equation:,

processing, often correspond to a geographical map­ping of the power system onto the multiprocessortopology as shown below for a two subnetwork ex­ample:

Fig. 6: Natural Decomposition of the ElectromagnelicTransienls Problem Model

where GA and GB are conductance matrices relatedto linear branch elements; TA and TB are non-linearfunctions related to non-linear branch elements;EA

and EB are vectors of the unknown node voltages;l~, I~ are nodal current injections corresponding toindependent sources, li, l~, IX, liJ are the nodalinjection currents related to the equivalent circuits ofinductors and capacitors, and If, 111 are the nodalcurrent injections present in the transmission linemodels. Figure 6 is a graphical representation ofthis modeL

Since IS(t) is known and IH(t), IL(t), and IC(t)depend only on terms computed in previous inte­gration steps, EA(t) and EB(t) can be computedindependently in different processors. The compu-'tation of the terms I~(t), I~(t), I~(t), IMt), andIX(t), liJ(t) can also be executed in parallel, sincethe equations related to branches in a particular sub­network depend only on nodal voltages belonging tothe same subnetwork. However, the term If (t) de­pend on the past terms lff (t - r) and EB(t - r). aswell as lff (t) depend on the past terms If (t - r)and E.(t - r). Since such terms have already beenevaluated in previous integration steps, the proces­sors must exchange data in order to each one be ableto compute its part of the vector IH (t).

An electromagnetic transients simulation method­ology, based on the above model, was developed andimplemented on a prototype of a parallel machineusing the Transputer T800 processor [43J. Using acareful handicraft load balancing procedure, it waspossible to achieve very high efficiencies (98%' insome cases) in the simulation of real size networks.

L 8l~, t) + R I(x, t) (17)

C 8E~, t) + G E(x, t) (18)

8E(x, t)8x

8I(x, t)8x

where E(x, t) and I(x, t) are p x 1 vectors of phasevoltage and currents (p is the number of phases); R,G, L and C are p x p matrices of the transmissionline parameters.

The wave equation does not have an analytic so­lution in the time domain, in t.he case of a lossyline, but it has been shown that it can be adequatelyrepresented by a trave!ing wave model consisting oftwo disjoint equivalent circuits containing a currentsource in paraliei with an impedance in both endsof the line. The value of the current sources aredetermined by circuit variables computed in past in­tegration steps (history terms).

This model is nicely structured for parallel pro­cessing: subnetworks of lumped parameter circuitelements connected by transmission !ines, represent­ing a group of devicee in a substation for instance,can be represented by sets of nodal equations thatinterface with other groups of equations by the vari­ables required to calculate the current sources in thetransmission !ine equivalent circuits. The exploita­tion of this characteristic of the network model, inthe partitioning of the set of equations for paraliei

135~~~-------- -----

5.4 Small-Signal Stability

Operational problems caused by e1ectromechani­cal oscillations weakly damped, or even negativelydamped, must be detected and eliminated to avoiddeterioration in the quality of the supplied electri­cal energy and damage to equipment. This type ofproblem can be studied using linearized versions ofthe power system dynamic mode!. The great advan­Lage of this approach is the possibility of the perfor­mance assessment of control schemes without timesimulation. This assessment is conducted throughlinear control systems analysis and design methods.A large scale numerical problem resulting from theapplication of these techniques is the computationaf eigenvalues and eigenvectors associated with thestate matrix of the linearized system mode!.

A linearized version of (1) and (2) at an operatingpoint (xo, zo) is given by

where lI,,'" l4 are Jacobian matrices evaluated atthe linearization poinl. The power system statetransition equation can be obtained eliminating Llzfrom (21):

Lli: = (lI - hl4-l h) Llx = A Llx (22)

where A is the system state matrix whose eigenval­ues provide information on the singular point sta­bility of the nonlinear system. Efficient algorithmsto obtain the dominant eigenvalues and eigenvectorsof A for large scale systems do not require the ex­plicit calculation of this matrix [44, 45]. These algo­rithms can be directly applied to (21), named theaugmented system matrix, whose sparse structurecan be fully exploited to reduce both cpu time andmemory requirements. These methods require re­peated solutions of linear equation sets of the form[46J:

where UI, vare unknown vectors; q is a complex shiftused lo make dominant the eigenvalues e10se to q;1 is lhe identity matrix; r is a complex vector; andk is the iteration counter. These equation sets areindependent and their solution can be obtained con­currently in different processors. This property makelhe eigenvalue problem well suited for parallel pro­cessmg_

In the work reported in [46] and [47], algorithmsfor the parallel solution of the eigenvalue problemfor small-signal stability assessment, using the aboveformulation. are described and the results of testswith models of a large practical power systems arepr~sented. A first investigatory line of research was

based on the parallelization of the Lop-sided Simul­taneous Iterations method [46]. The obvious paral­leI stratagem used was to carry out each trial vectorsolution on a different processor. Results obtainedin tests performed on the iPSCj860 parallel com­puter, using two large scale representations of theBrazilian South-Southem interconnected power sys­tem, presented computation efficiencies around 50%.A second approach to the problem uses a HybridMethod [47] resulting from the combination of theBi-Iteration version of the Simultaneous Iteration al­gorithm and the Inverse Iteration method. The Hy­brid algorithm exploits the fast eigenvalue estima­tion of the Bi-Iteration algorithm and the fast eigen­vector convergence of the Inverse Iteration algorithmwhenever the initial shift is e10se to an eigenvalue.In the Inverse Iteration stage, the Hybrid algorithmallows perfect parallelization. The results obtainedindicate a superior performance of this method bothin terms of computation speedup and robustness.

5.5 State Estimation

State estimation is a basic module in the EnergyManagement System (EMS) advanced applicationsoftware. lts main function is to provide reliableestimates of the quantities required for monitoringand control of the electric power system. In al­most ali state estimation implementations, a set ofmeasurements obtained by the data acquisition sys­tem throughout the whole supervised network, atapproximately the same time instant, is centrallyprocessed by a static state estimator at regular in­tervals or by operator's requesl. Modem high speeddata acquisition equipment is able to obtain new setsof measurements every I-lO seconds but the presentEMS hardware and software alIow state estimaÚonprocessing only every few minutes. It has been ar­gued that a more useful state estimation operationalscheme would be achieved by shortening the time in­terval between consecutive state estimations to allowa e10ser monitoring of the system evolution partic­ularly in emergency situations in which the systemstate changes rapidly. Another industry trend is toenlarge the supervised network by extending stateestimation to low voltage subnetworks. These trendspose the challenge of performing state estimation ina few seconds for networks with thousands of nodes.

The higher frequency in state estimation execu­tion requires the development of faster state estima­tion algorithms. The larger size of the supervisednetworks will increase the demand on the numericalstability of the algorithms. Conventional centralizedstate estimation methods have reached a develop­ment stage in which substantial improvements in ei­ther speed or numerical robustness are not likely tooccur. These facts. together with the technical de­velopments on distributed EMS, based on fast datacommunication network technology, opens up the

136

possibility of parallel and distributed implementa­tions of the state estimation function.

In the work reported in [48), the possibility of par-oalieI and distributed state estimation implementa­tion was exploited leading to a soJution methodologybased on conventional state estimation algorithmsand a coupling constraint optimization technique.

The information modeI used in power system stateestimation is represented by the equation

where z is a (m x 1) measurement vector, x IS a(n x 1) true state vector, h(.) is a (m x 1) vedorof nonlinear functions, w is a (m xl) measurementerror vector, m is the number of measurements, andn is the number of state variables. The usual choicefor state variables are the voltage phase angles andmagnitudes while the measurements are active andreactive power f10ws and node injections and voltagemagnitudes.

Like in load f10w calculations, it has been foundthat state estimation algorithms based on decoupledversions of (24) behave quite adequately for the usualpower networks·. Therefore, the foilowing decoupledmodel has been mostly adopted:

z = h(x) + W

Zp hp(O, v) + Wp

z. = h.(O, v) + W q

(24)

(25)

(26)

where

z; and z; are vectors of active and reactive mea­surements in area k; dimensions: (m; x 1) and(m; xl), respectively.

Ok and vk are vectors of voltage phase angles andmagnitudes in area k, including the ones corre­sponding to the boundary buses; dimensions:(n~ x 1) and (n~ xl), respectively.

The WLS state estimat~ for the distributed estima­tíon problem depicted in (27) and (28) can be ob­tained solving a constrained optimization plOblemwith a separable objective function and a set of lin­ear constraints introduced to force the state variablesin the overlapping areas to assume the same values.The WLS plOblem is stated as follows:

r

L 1/2{ [z; - h~()f[R;]-I[z; - h;()] +k=1

+[z; - h:(.)nR~rl[z; - h:()]} (29)r

S. to L.1kOk - O (30)p -k=1

r

L.1;vk' = O (31)k=1

where O (nq xl) and v (n. xl) are the vectors oftrue voltage magnitudes and phase angles; p and qare subscripts indicating partitions of vectors andmatrices corresponding to active and reactive mea­surements, respectively; nq = n - 1, n. = n, and nis the number of network nodes.

Assume that the power network is decomposed inr areas. The areas are connected through boundarybuses which belongs simultaneously to both adjacentareas. Therefore, there are overlapping areas whichmay be observed using both adjacent measurementsets. The number of boundary buses may be kept toa minimum or incorporate a few extra buses in or­der to facilitate some estimation functions·like baddata processing, for instance. A further assumptionis that there are no injection measurements in theoverlapping area buses. This assumption does notnecessarily represent a practical limitation to theproposed methodology as actual injection measure­ment buses in overlapping areas can be replaced byfictitious buses with no injection measurement con­nected to the actual buses, now placed outside theoverlapping area, by zero impedance lines.

Under the above assumptions, the state estima­tíon plOblem introduced in (25) and (26) can be de­composed as follows

where 'P and "Iq are (I xl) vectors of Lagrange mul­tipliers and

calculated at the solution point.As in the integrated state estimation applOach,

the Gauss-Newton method combined with the usual

ohk(Ok. vk)H k = q .

q avkand

ohk(Ok vk)H k = p ,

P OOk

aL-[H;f[R~rl[z;- h~(Ok, vk)J +=

aok

+[.1;f "Ip = O , k = 1, ... , r (32)

oL-[H;f[R~]-I[z;- h;(Ok, vk)] +

ovk =

+[.1~]T "Iq = O , k = I, '.'IT (33)

oLr

= L.1~Ok =0 (34)a"lp

k=l

aLr

LA~vk = O (35)O,q k=1

where A; and A; are (I x n~) and (I x n~), respec­tively, matrices (I is the number of boundary buses)whose nonzero elements are either 1 or -1.

The necessary conditions for the solution of theabove problem, derived flOm the corresponding La­grangian function L, are:

(27)

(28)

k=l, ,r

k=l, ,r

= h~(Ok, vk) + w~,

h~(/fk, vk) + w;,

137

Fast Decoupled assumptions can be used to solve(32)-(35) leading to the following algorithm:

Õk(i+l) = Ok(i)+[C;t'tlb;(i),k = 1, ... ,r (36)

(37),p(i + 1)

Ok (i + 1)

r

Np-I L .1;Õk(i + 1)

k=1Õk(i + 1) - [C;]-I[A;f-{p(i + 1),

k=I, ... ,r (38)

A straightforward implementation of the algo­rithm given in (36)-(47) leads to a hierarchical es­timator. As explained earlier, this kind of estima­tor is not suitable for parallel or distributed pro­cessing. A version of the algorithm more adequateto this type of processing can be obtained neglect­ing the off-diagonal elements in matrices C; and C;in equations (37,38) and (40,41). In this case, lheseequations can be merged and rewritten, respectively,as

r

,q(i+l) Nq- ' L A;vk(i + 1) (40)k=1

vk(i + 1) = vk(i + 1) - [C;]-I[A;f ,q(i + 1),

k = 1, ...,r (41)

where

tlb; (i) [H;f[R;]-'[z; +

- h;(Ok(i),v.(i))] (42)

tlb; (i) [H;f[R;]-'[z; +

- h;(Ok(i + 1),v.(i))] (43)

Ck = [H;f[R;l-IH; (44)p

Ck [H;f[R;]-1 H; (45)qr

Np = L.1;[C;t1[.1;f (46)k=1

r

Nq L .1;[C;t1[.1;f (47)k=1

and H p and H q are calculated at nominal conditionsand kept constant in the iterative processo

Matrices N p and N q defined in (46) and (47) playa key role in the distributed state estimation formu­lation. These matrices present the following relevantcharacteristics:

• Slructure: represented by a graph whose nadescorrespond to the boundary buses: from a partic­ular node, there are links to ali nodes correspond­ing to boundary buses of the adjacent areas.

• Elements: the diagonal elements are the sum ofthe diagonal elements of the two local area inversegain matrices (C; and C;) involved in the corre­sponding constraints; the nonzero off-diagonal el­ements correspond to the off-diagonal elements ofthe inverse local area gain matrices; in the case ofmore than one boundary bus in the overlappingarea, the off-diagonal elements related to the in­terconnection of these buses are the sum of thecorresponding e!ements of the two local area in­verse gain matrices.

k

± k 9rrgi [Õ~(i + 1) - õ;.(i + 1)150)grr + rr

(48)

(49)

Õk(i + 1) + tlÕk(i + 1)

vk(i + 1) + tlvk(i + 1)

Ok (i + 1)

vk (i + 1) =

where g:r and gtr are diagonal elements correspond­ing to bus r of the inverse gain matrices of the neigh­boring areas k and j, respectively, and the sign isset to + or - according to A;. The elemems oftlvk(i + I) are calculated similarly.

The algorithm introduced in the previous sectioncalculates the e and v updates at every iteration ina synchronous way, i.e., it has to wait until the statevector is updated in ali areas (ar processors) beforeit starts a new iteration. This approach presentsdrawbacks regarding both parallel and distributedprocessing. In paraliei processing, synchronous al­gorithms usually cannot achieve high elficiency dueto processar idle time unless a perfect load balanc­ing is obtained which is not easily achievable in mostpractical applications. In distributed processing, itrequires dilficult synchronizing operations and reli­able communication systems which is not usuallypresent in geographically distributed systems. Asyn­chronous algorithms, on the other hand, are moref1exible for the implementation of both parallel anddistributed computation processes.

To achieve asynchronous mode operation, the al­gorithm defined above has to be slightly modified.The modifications concem the way the componentsof the vectors defined in (48) and (49) are updated.In the synchronous version, these vectors are calcu­lated using values of the state variables obtained inthe i-th iteration. In the asynchronous algorithm,these vectors are updated using the latesl availablevalue of the state variables. Therefore, whenever aparticular processar finishes the updating of its statevector, it transmits this information to its adjacentprocessors (logical adjacency), reads in' the values ofthe boundary buses state variables from its buffer,regardless of lhe age of Ihal informQllOn, and pro­ceeds to the calculation of the next area ,;tate vectorupdate.

where ali the elements of tlÕk(i + 1) are null exceptthe ones corresponding to boundary buses that aregiven by

(39)

Vk (i) + [C;t 1tlb; (i),

k=I, ... ,r

vk(i + 1)

and

138

Owing to the local based nature of the stateestimation algorithm, simulated experiments haveshown that the computation can continue even inthe absence of information from other areas. Thisfact can easily be understood considering that (36)and (39) alone are in fact a local decoupled stateestimator. Obviously, in this case the matching ofboundary bus state variables would not be achieved.This property make the algorithm suitable to asyn­chronou!! implementation. Based on the above prop­erty, several iterative schemes can be derived, incor­porating different degrees of asynchronism, as willbe discussed in latter sections.

The results of computational experiments indicatethat the algorithm described above is accurate, ro­bust and efficient in terms of reducing the requiredcomput.ation time. An important by-product. of thework is the indication that state estimation is proba­bly a naturally decoupled problem. Important paral­lei and distributed state estimation issues, like database access and communication between processors,were not addressed in this work. Also, bad data pro­cessing and observability analysis, in the distributedcontext, were not studied. These issues are topicswell suited for further research. Finally, a strongpractical advantage of the methodology introducedin this paper is the use of standard state estimationalgorithms.

5.7 Team AIgorithms

Parallel asynchronous implementations of itera­tive algorithms are now establishing themselves asgood choices for high performance computation indistributed-memory environments, in view of sev­eral attractive features that they possess, such asease of implementation, facility with load balancing,shorter convergence times and so on [6J. A new con·text for asynchronism arose with the introduction ofthe so-called Team Algorithms, which are hybridsof different methods. Such combinations of algo­rithms were first proposed in the context of powersystem applications in [52], for sequential comput­ers, and later again in [53, 54, 55], where they werefirst called Team Algorithm, in the context of asyn­chronous parallel computing, in which such methodshave a natural implementation.

The main idea behind a Team Algorithm (TA)is to partition a complex problem in several sub­problems and to solve each subproblem in a differentprocessor of a distributed computer system. choos-­ing for each subproblem one or more methods thatbest solve it. That way, if there are subproblemswith different characteristics for which it has beenchosen different methods, the resulting combinationof methods is a TA.

As an illustration for the TA approach, considerthe problem of solving a seI. of n nonlinear algebraicequations given by:

5.6 Load Flow Computations <I>(x) = O x E R" (51)

<I>,(X) = O (54)

<I>2(X) = O (55)

An asynchronous implementation of a TA to solve(51), implemented in a distributed memory comput­ing system, can be represented as:

Assume that the elements of (51) can be partitionedas:

<I>(x) = [<I>,(x) <I>2(X) f (52)

x = [Xl X2 f (53)

Then, equation (51) may be rewritten as:

9dx ' (k)]92[x2(k)]cx(k) + W, X~(k)

=X1(k+ 1)X2(k + 1)x(k + I) +w2X~(k)

(56)where, 9i (i = 1,2) are functions representing theiterative processes used in each processor. respec­tively; xi(k) represents the most recently receivedvalue of vector x in processor i aI. iteration k, whilexr(k), (i = 1,2), represents the most recent valueof Xi aI. iteration k, available to processor 3 (theAdministrator) .

The TA given by (56) works as follows: each pro­cessor i (i = 1,2) tries to solve problem (4) using

• The practicalload-flow problem is much more dif­ficult to parallelize than other similar problemsowing to constraints added to the basic system ofnon-linear algebraic equations.

• Very efficient algorithms are already availablewhich can solve large load-flow problems (morethan 1000 nodes) in a few seconds on relativelyinexpensive computers [1].

More interesting investigatory lines are the par­allelization of multipie load-flow solutions (contin­gency analysis, for instance) [49, 50J and the speedup of load-flow programs in vector processors[51].

Load-flow is a fundamental tool in power sys­tem studies. It is by far the most often used pro­gram in evaluating system security, configuration ad­equacy, etc., and as a starting point for other com­putations such as short circuit, dynamic simulation,etc. Its efficient solution is certainly a fundamen­tal requirement for the overall efficiency of severalintegrated power system analysis and synthesis pro­grams. Therefore, it should be expected a great re­search effort in the parallelization of the 10ad-f1owalgorithms. Tbat has not been the case, however,for two main reasons:

139

tx txProcessor 1 Processor 2

Xl = Çl (x) X2 = Ç2(X)

Xl X2

Processor 3: Administrator

~ X = C X + Wl Xl + W2 X2

Fig. 7: Example cf Team Algcrithm.

an iterative algorithm represented by the map ç;,using the most recent value of X received from theAdministrator, and transmitting the updated valueXi. At the same time, the Administrator updates X

combining most recentl)' received values of Xi, usingproperly chosen non-negative weíghts (Wl, W2 2: O).The constant c is chosen to ensure that for a solutionx' , the following condition holds:

which. in turno implies that:

c = 1 - Wl - W2·

This solution process is illustrated in figure (7).The advantages of using a TA in distributed load

flow computations is demonstrated in the researchwork reported in [56, 57, 58]. In this work, twoload flow example problems are studied. For theseexamples, neither the Fast Decoupled method northe block Jacobi version of the Y matrix methodcan properly solve the problem; however, a TAcombining both methods solves the problems withthe additional advantage of being easily parallelizedin an asynchronous environment with an excellentspeedup. That way, it was possible to exploit ali thepotentia! of distributed memory computers, as theInte! iPSCj860 hypercube of 8 nodes used for theimplementations described in the referred work.

6 Research Activities l.n Brazilian In­stitutions

In Brazil, research actlvltles in parallel process­ing application to power system problems have beenshowing a fairly fast pace in the last ten years or so.At least three Brazilian institutions have researchgroups working in this area: COPPE I , CEPEV,and UNICAMp3 .

l The Postgraduate School of Engineering of the FederalL;niversitv of Rio de Janeiro.

2The Brazilian Utilities Research Center.3The University of Campinas.

CO PPE started a research project in the appli­cation of paraliei processing to engineering prob­lems in 1986. The project joined together profes­sors, researchers and graduate students of the Com­puter Science, Civil, and Electrical Engineering De­partments. The objective of the project is the de­velopment of a complete cycIe of parallel process­ing technology comprising hardware, basic and ap­plication software. The project has been financedby the Brazilian Government Agencies FINEP andCNPqjRHAE. An Intel iPSCj860 parallel com­puter, with hypercubic topology and eight process­ing nodes, has been used in this project. AnIBMjSP-2 parallel computer with eight cpu·s anda Cray J90j8 supercomputer are in the process ofacquisition by COPPE and are planned to becomeavailable for research activities in early 1995. As aresult of this project, several software methodolo­gies and programs for parallel processing have beendeve!oped. The ones related to power system ap­plications have already been mentioned in previoussection of this paper. Also, a prototype of a paral­Ie! machine based on the Transputer T800 processorwas built and has been used in the development ofapplications.

CEPEL and UNICAMP started their activities inthis research area using a parallel machine knownas PP (Preferential Processor) developed at theresearch center of the Brazilian TelecomunicationCompanies (CPqDjTELEBRAS). This machine isbased on a hybrid sharedjdistributed memory ar­chitecture and the Intel 80286 processor. A few pro­totypes of this machine were bui!t, some of whichare still in operation, but the project was later dis­continued.

CEPEL developed a parallel programming envi­ronment for the PP, based on the MS- DOS oper­ating system extended with facilities for commu­nication and synchronization, which was used inpower system applications such as multi-area reli_oability evaluation. hydrothermal production costingand reliability evaluation. and securitv constraÍned. .optimal power flow [59. 60]. This programming en­vironment was also used at UNICAMP to develop aparallel methodology to solve sets of network equa­tions based on the matrix inverse factors (W-matrix)approach [29]. After the discontinuity of the PPproject. the UNICAMP power system group ac­quired a nCUBE-2 with 64 processors that has beenused in research projects in security constrained ap­timal power flow, probabilistic short-circuit analv­sis, etc. [18, 61]. CEPEL is also developing a longterm project in the application of distributed archi­tectures and open system concepts to EMS design[20J.

The availability of parallel machines for researchand development activities in Brazilian institutianshas also increased considerably in the !ast few years.

140

Most of the ideas contained in this paper are theresult of a long and fruitful collaboration with Prol.E. Kaszkurewicz and ProL A. Bhaya (COPPE) andour present and former students I.e. Decker, H.L.S.Almeida, M.H.M. Vale, B. Ba.ran, J.M. Campagnolo,C.L.T. Borges, and D.Q. Siqueira. More recent workhas been developed in collaboration with Prol. F.F. Wuand Dr. L. Murphy (U .C. Berkeley, USA) in Pala.lIel andDistlibuted State Estimation, and DI. Nelson Martins(CEPEL) and Prol. J.L.R. Pereira (UFJF) in PaIalieISma.ll Signal Stability Assessment. Pa.rt of this papeltext and some figures were extracted from thesis and pa­peIS wlitten by OI with people referred to above.

Acknowledgments

[1] J.V. Mittsche, "Stletching the Limits of PowerSystem Analysis", IEEE Computer Application inPower, vol. 6, no. 1, pp. 16-21, Janua.ry 1993.

[2] IEEE Spectrum, Special I.sue: Supercomputers,September 1992.

[3] T.G. Lewis and H. EI-Rewini, Introduction to Par­aliei Computing, Prentice Ha.lI, New YOlk, 1992.

[4] M.J. Quinn, Parollel Computing: Theory and Proc­tico, McGlaw-Hill, New York, 1994.

References

tle bit less uncertain, it should be pointed out thetendency in the paralieI computer industry to maketheir products follow open system standards and thepossibility of developing applications less dependentof a particular architecture.

Distributed processing is a prevailing technologyin many information processing applications. Com­bined with Lhe concept of open system, it offers abetter performance/cost ratio, more reliability andf1exibility, scalability, etc. In the power utility en­vironment, it has been used in distribution automa­tion and is substituting the minis and mainframes inthe supervisory and control centers. In the future,networks of microcomputers and workstations willprobably integrate operation and planning informa­tion systems with accounting and billing processingsystems allowing direct information transfer and re­source sharing.

The most likely scenario for the electric utility in­formation processing system of the future will bea conglomerate of local area networks serving spe­cilic functions (control centers, expansion and op­erational planning, billing, substation control, etc),built around sta'ldard hardware and software, inwhich each computer has specilic characteristics likesophisticated graphic terminais, data base servers,etc. Some of the computers may actually be paralieImachines that will act as number crunchmg serversfor highly intensive numerical applications. Theselocal area networks will be joined together by a widearea network connecting computers in the corpora­tion headquarters and widespread in the whole areaserviced by the utility.

Avatlable In early 199<l.

Make Number Institi-Model of cpu's tution

Intel iPSC/860 8 COPPEnCube 64 UNICAMP

IBM SP-l 16 LNCCIBM SP-l UNICAMPCray YMP 2 UFRGSCray J90/8 8 COPPE"IBM SP-2 8 COPPE"

" -

ParalieI processing may be the only way to makeviable some power system applications requiringhigh performance computing like real-time dynamicsecurity assessment, security constrained optimalpower f1ow, real-time simulation of electromagneticand electromechanical transients, composite reliabil­ity assessment using realistic models, etc. ParalieIcomputers are presently available in a price rangecompatible with power system applications and pre­senting the required computation power.

Two main factors are still impairments to the wideacceptance of these machines in power system ap­plications: the requirements for reprogramming orredevelopment of applications and the uncertaintyabout the prevailing parallel architecture. The lirstproblem is inevitable, as automatic parallelizationtools are not likely to become practical in the nearfuture, but has been minimized by the research effortin parallel algorithms and the availability of more ef­ficient programming tools. The second difficulty isdestined to disappear as parallel computers leave thelaboratories and arrive in the showrooms of comput­ers' manufacturers.

Likewise in the history of sequential computer evo­lution, a unique and overwhelming solution to theparalieI computer architecture problem is not to beexpected. It is more likely that a few different ar­chitectures will be successful in the next few yearsand the users will have to decide which one is themost. adequate for their application. Moreover, it isprobably that commercial processing applications,which are now turning towards paraliei processing,are the ones that will ~hape the future parallel com­puter market. However, to make this scenario a lit-

7 Concluding Remarks

Table 3: Parallel computers and supercomputers avail­able in Brazil

Table 3 gives a partial relation of parallel comput­ers and supercomputers available in Brazil. Mo.stof these machines are open to use for the academiccommunity and accessible via computer communica­tion networks.

1"I

141

[5] D.J. Tylavsky, A. Bose, et ai., "Parallel Processingin Power System Computation", IEEE Transaetionson Power Systems, voI. 7, no. 2, pp. 629-637, May1992.

[6] D.P. Bertsekas and J.N. Tsitsikilis, ParaI/c/ andDistributed Computahon, Prentice Hall, New York,1989.

[7] A. Umar, Distributed Computing: A Praetical Syn­thesis, Prentice-Hall, New Jersey. 1993.

[8] B. Stott, "Review of Load Flow Calculation Meth­ods", Proeeedings of the IEEE, voI. 62, no. O?, pp.916-929, July 1974.

[9] B. Stott, "Power System Dynamic Response Calcu­lations", Proeeedings of the IEEE, voI. 67, no. 2, pp.219-241, February 1979.

[10] H.W. Dommel and W.S. Meyer, "Computationof Electromagnetic Transients", Proceedings of theIEEE, voI. 62, no. ???, pp. 983-993, July 1974.

[11] B. Stott, O. Alsac, and A. Monlicelli, "SecurityAnalysis· and Optimization", Proeeedings of theIEEE, voI. 75, no. 12, pp. 1623-1644, December1987.

[12] N.J. Balu, et ai., "On-Line Power System SecurityAnalysis", Proeeedings of the IEEE, voI. 80, no. 2,pp. 262-280, February 1992.

[13] Y. Sekine, K. Takahashi, and T. Sakaguchi, "Real­Time Simulation af Power System Dynamics" I Pro­eeedings of the 11th Power Systems ComputationConferenee, Avignon, France, Aug. 30 - Sep. 3 ,1993.

[14] H. Taoka, I. lyoda, H. Noguchi, N. Sato, and T.Nakazawa, "Real-Time Digital Simulator for PowerSystem Analysis on a Hypercube Computer", IEEETransactions on Power SY4ter.u, vai. 7, no. 1, pp.1-10, February 1992.

[15] D. Brandt, R. Wachal, R. Valiquette, and R. Wier­ckx, "Closed Loop Testing of A Joint VAR Con­troUer Using a Digital Real-Time Simulator", IEEETran.!oction$ on Power Sy.!tem8, vaI. 6, no. 3, pp.1140-1146, August 1991.

[16J M.V.F. Pereira and N.J. Balu, "Composite Genera­tion/Transmission Reliability EvaJuation" I Proceed­ings of lhe IEEE, voI. 80, no. 4, pp. 470-491, April1992.

[17J A.M. Leite da Silva, J. Endrenyi and L. Wang, "In­tegrated Treatment of Adequacy and Security inBulk Power System Reliability Evaluation". IEEETran.!octio"" on Power Sydems, vaI. 8, no. 11- pp.275-285, February 1993.

[18] F. Sato, A.V. Garcia, and A. Monticelli, "ParallelImplementation of Probabilistic Short-Circuit Anal­ysis by the Monte Carlo Approach", Proceedingsof the 18th Power Industry Computer AppliealionsConferenee, Scottsdale, AZ, May 1993.

[19] IEEE Tutorial Course on Fundamentais of Supervi­sory Syslems, Publication 91EH0337-6-PWR, 1991.

[20] L.C. Lima et ai., "Design and Development of anOpen EMS", Proeeedings of th, Joint InlemalionalPower Conference (Athens Power Teeh), Athens,Greece, September 1993.

[21J IEEE Tutorial Course on Distribulion Automation,Publication 88H0280-8-PWR, 1988.

[22] W.R.Cassel, "Distribution Management Syslems:Functions and Payback", IEEE Transaetions onPower Syslems, voI. 8, no. 3, pp. 796-801, August1993.

[23] L. Murphy and F.F. Wu, "An Open Design Ap­proach for Distributed Energy Management Sys­tems", IEEE Transactions on Power Sy"tems, vaI.8, no.3, pp. 1172-1179, AugusI 1993.

[24] M.H.M Vale, D.M. Falcão, and E. Kaszkurewicz,"A Semiautomatic Nelwork Decomposition Methodfor Block-Iteralive Solutions on Paraliei Compul­ers" , submitted lo the 1995 IEEE Power Indus­Iry Computer Applications Conference, Salt Lake,USA, May 1995.

[25] W.L. Hatcher, F.M. Brasch, and J.E. Van Ness,"A Feasibility Study for lhe Solution of TransientSlability Problems by Multiprocessor Structures",IEEE Transaelions on Power Apparalus and Sys­tems, voI. 96, no. 6, pp. 1789-1797, Nov./Dec. 1977.

[26] F.M. Brasch, J.E. Van Ness, and S.C. Kang, "Sim­ulation of a Multiprocessor Network for Power Sys­tem Problems", IEEE Transactions on Power Ap­paratus and Systems, voI. 101, no. 2, pp. 295-301,1982.

[27J J .S. Chai, N. Zhu, A. Bose, and D.J. Tylavsky,"Paraliei Newton Type Methods for Power SystemStability Analysís Using Local and Shared Mem­ory Multiprocessors", IEEE Transactions on PowerSyslems, voI. 6, no. 4, pp. 1539-1545, November1991.

[28] J .S. Chai and A. Bose, "Bottlenecks in ParalleIAlgorithms for Power System Stability AnalysiB",IEEE Transaetions on Power Systems, voI. 8, no. 1,pp. 9-15, February 1993.

[29] A. Padilha and A. Morelato, "A W-Matrix Method­ology for Solving Sparse Network Equations onMultiprocessor Computers", IEEE Transaetio!,s onPower Systems, voI. 7, no. 3, pp. 1023-130, August1992.

[30] A.F.C. Canto e J.1.R. Pereira, "Solução deEquações da Rede Elétrica Utilizando Processa­mento Sequencial e Paralelo", Anais do 10 Con­gre880 Bra.!ileiro de Atomática, Rio de Janeiro, RJ, Setembro 1994.

[31] I.C. Decker, D.M. Falcão, and E. Kaszkurewicz,"An Eflicient Parallel Method for Transient Stabil­ity Analysis", Proceedings of the 10th Power Sys­tema Computation Conference, Graz, Austria , pp.509-516, August 1990.

[32] I.C. Decker, D.M. Falcão, and E. Kaszkurewicz,"Parallel Implementation of a Power System Dy­namic Simulation Methodology Using the Conju­gate Gradient Method", IEEE Transactions onPower Systems, vol. 7, no. 1, pp. 458-465, Febru­ary 1992.

[33] I.C. Decker, D.M. Falcão, and E. Kaszkurewicz,"Conjugate Gradient Methods for Power SystemDynamic Simulation in Parallel Computers" I sub­mitted to the 1995 IEEE PES Summer Meeling.

[34] M.H.M Vale. D.M. Falcão, and E. Kaszkurewicz,"Electrical Power Network Decomposition for Par­alieI Computations", IEEE Symposium on Circuit, T':.

142

I

I ,I

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43J

[44]

[45]

[46]

[47]

and Systems, San Diego, CA, pp. 2761-2764, May1992.

M.Ilié-Spong, M.L. Crow, and M.A. Pai, "TransientStability Simulation by Waveform Relaxation Meth­ods'" 1 IEEE Transaetions on Power Systems, vaI. 2,no. 4, pp. 943-952, November 1987.

M.L. Crow and M. Ilié, "The Parallel Implementa­tion af the Waveform Relaxation Method for Tran4

.

sient Stability Simulations", IEEE Transaetions onPower Systems, vol. 5, no. 3, pp. 922-932, August1990.

F. Alvarado, "Parallel Solution of Transient Prol>­lems by Trapezoidal Integration", IEEE Transae­tions on Power Apparatus and Systems, vol. 98, no.3, pp. 1080-1090, MayfJune 1979.

M. LaScala, A. Bose, D.J. Tylavsky, and J.S. Chai,"A Highly Parallel Method for Transient StabilityAnalysis", IEEE Tran.soction" on Power Sydem.!,vol. 5, no. 4, pp. 1439-1446, November 1990.

M. LaScala, M. Brucoli, F. Torelli, and M. Trovato,"A Gauss-Jacobi-Block Newton Method for ParallelTransient Stability Analysis", IEEE Transaetionson Power Systems, vol. 5, no. 4, pp. 1168-1177,November 1990.

J. M. Ortega, Introduetion to ParaI/e! and VeetorSolution of Linear Systems, Plenum Press, NewYork, 1988.

J.J. Dongarra, I.S. Duff, D.C. Sorensen, and H.A.van der Vorst, Solving Linear Systems on Vectorand Shared Memory Computera, SIAM, 1991.

H.A. van der Vorst, "Bi-CGSTAB: A Fast andSmoothly Converging Variant of Bi-CG for the 5<>­lution of Nonsymmetric Linear Systems", SIAM J.Sei. Stal. Comput., vol. 13, no. 2, pp. 631-644,Mareh 1992.

D.M. Falcão, E. Kaszkurewicz and H.L.S. Almeida,"App!ication of Parallel Processing Techniques tothe Simulation of Power System ElectromagneticTransients", IEEE Transactions on Power Appara.tus and Systems, vol. 8, no. 1, pp. 90-96, February1993.

N. Martins, "Eflicient Eigenvalue and FrequencyResponse Methods Applied to Power System Small­Signal Stability Studies", IEEE Transactions onPower ApparatuAI ond System!, vaI. 1, no. 1, pp.217-226, February 1986.

N. Martins, 1.T.G. Lima, H.J.C.P. Pinto, and N.J.P. Macedo, "A State-of·the-Art Computer ProgramPackage for the Analysis and Control of Small Sig­nal Stability of Large AC/DC Power Systems", Pro·eeedings of the lERE Workshop on New Issues inPower System Simulation, Caen, France, pp. 11-19,March 1992.

J.M. Campagnolo, N. Martins, J.1.R. Pereira, L.T.G. Lima, H.J.C.P. Pinto, and D.M.Falcão, "FastSmall-Signal Stability Assessment Using ParallelProcessing", IEEE Tran"oetioo., 00 Power Apparo~

tu.! and Systems , vaI. 9, no. 2, May 1994.

J .M. Campagnolo, N. Martins, and D.M. Falcão,"An Eflicient and Robust Eigenvalue Method forSmall-Signal Stability Assessment in Paraliei Com-

puters", presented at the 1994 IEEE PES SummerMeeting, San Francisco, CA, Ju!y 1994.

[48] D.M. Falcão, F.F. Wu, and L. Murphy, "Paralleland Distributed State Estimation", presented at the1994 IEEE PES Summer Meeting, San Francisco,CA, July 1994.

[49] D.Q. Siqueira, Power System Statie ContingeneyAnalllsis Using Paral/el Processing, M.Sc. Thesis,COPPE, 1991, (In Portuguese).

[50J D.P. Pinto e J.L.R. Pereira, "Um Método Integradopara Análise de Contingências Usando Processa­mento Paralelo", Anais do 10 Congresso Brasileirode Atomática, Rio de Janeiro, RJ , Setembro 1994.

[51J C.L.T. Borges, Performance Assessment of PowerFlow Solution Methods for ParaI/e! and Vector Pro·eessing, M.Sc. Thesis, .COPPE, 1991, (In Por­tuguese).

[52] Y.P. Dusonchet, S.N. Talukdar, H.E.Sinnot. andA.H. El-Abiad, "Load Flows Using a Combina­tion of Point Jacobi and Newton's Method", IEEETranJaetion$ on Power Apparatu$ and Sy.dem!, vaI.90, pp. 941-949, 1971.

[53] S. N. Talukdar, 5.5. Pyo, and R. Mehrotra, "DesignAIgorithms and Assignments for Distributed Pr<>­eessing", EPRI Reparl EL.9917, 1983.

[54] R. Mehrota and S.N. Talukdar, "Task Scheduling onMultiprocessors for Power System Problems". IEEETransaetions on Power Apparatus and Systems. vaI.102, pp. 3590-3597, 1983.

[55] S.N. Talukdar, 5.5. Pyo, and T.C. Giras, "Asyn­chronous Proeedures for Paraliei Processing", IEEETran$oction" on Power Apparatu", and SYJtemJ, voI.102, pp. 3652-3659, 1983.

[56] B. Barán, An Study of Asynehronous Toam AI.gorithms, D.Sc. Thesis, COPPE, 1993, (In Por­tuguese).

[57] B. Barán, E. Kaszkurewicz, and A. Bhava. "Par­aliei Asynchronous Team Algorithms: Co~vergenceand Performance Analysis" , submitted to SeientijieComputing, SIAM, 1994.

[58] B. Barán, E. Kaszkurewicz, D.M. Falcão, "Team AI­gorithms in Distributed Load Flow Computations",submitted to the 1995 IEEE PES Winler Meeting.

[59] M.J. Teixeira, H.J.C. Pinto, M.V.F. Pereira, andM.F. MCCOYl "Developing Concurrent ProcessingApplications to Power System Planning and Oper­a.tions", IEEE Tran.saction.s on Power Sydem.s, voI.5, no. 2, pp. 659-664, May 1990.

[60J H.J.C. Pinto, M.V.F. Pereira, and M.J. Teixei­ra, "New Parallel Algorithms for the Security­Constrained Dispatch with Post-Contingency Cor­rective Actions", Proceedings of the 10th Poteer Sys.temJ Computation Conference, Graz, Austria. I Au­gust 1990.

[61J M. Rodrigues, O.R. Saavedra, and A. MonticeUi,"Asynchronous Programming Model for the Con­current Solution of the Security Constrained Opti­mal Power Flow Problem", IEEE/PES 1999 Sum­mer Meeting, Vancouver, Canada, July 1993.

143