myrda vision
TRANSCRIPT
-
8/6/2019 Myrda Vision
1/13
32 IEEEpower & energy magazine may/june 20071540-7977/07/$25.002007 IEEE
TTHE RETAIL INDUSTRY IN THE UNITED STATES KNOWS THAT
every year between Thanksgiving and Christmas the hustle and bustle is on
to sell one more video game or one more sweater. For the North American
electric utility industry, every year between the beginning of May and the
beginning of June the hustle and bustle is on to get one more transformer
humming or one more substation online. It should come as no surprise then
that the livening of a small transmission substation on a blue-sky Saturday
late in May of 2006 in a small town in Michigan could go unnoticed. This
substation features the use of an Ethernet local area network (LAN), reliance
on IEC 61850, the capture of nonoperational data, and a station human-
machine interface (HMI). This is pretty obviously an example of substation
automation and yet that term may not tell the whole story.
To help appreciate how much substation automation has become a part of the
lexicon, a recent Google of the phrase got the same approximate number of hits as
the phrase protective relay (145,000). Though the definition of protective relay
can be debated, the term substation automation can be actively deceptive. Itimplies that this concept is within the walls of the substation and seems to refer to
the automatic operation of things like voltage control, load transfer, and tap
changes. The Michigan substation is meant to realize the organic implementation
of automation technologies. It reflects how a system would evolve around these
technologies as opposed to having substation automation added to a system. This
substation was an attempt at realizing the vision of automation.
Strategy and MethodologyIn 2004, Michigan Electric Transmission Co. embarked on a program to devel-
op a sweeping business and technical strategy to replace the aging protection
Authorized licensed use limited to: IEEE Xplore. Downloaded on January 21, 2009 at 09:13 from IEEE Xplore. Restrictions apply.
-
8/6/2019 Myrda Vision
2/13
may/june 2007 IEEEpower & energy magazine 33
and control equipment. This aggressive investment and replacement strategy
can be the most cost-effective solution for system-wide upgrading.
The implementation methodology, addressing impact on capital and oper-
ating costs, is described in more detail and is based on the following tasks:
integrating all relaying, control, monitoring, automation, and enterprise
functions through Ethernet LANs in the substations and EPRIs com-
mon information model (CIM)
introducing IEC 61850 LAN integration system and protocol as rapidly
as is practical, to replace control wiring, and to simplify integration and
data flow
organizing protective functions using the newest generations of relays to
improve dependability and security, while drastically reducing the num-
ber of units required and complying with or exceeding all agency
design requirements
looking at recent operating issues, relaying problems, industry trends,
and recent wide-area system events; and designing a solution that aimssquarely at improving performance on these specifics.
Key steps to implement the standardized substation protection and control
design are being carried out. The first step was to develop a practical and inno-
vative technical strategy for system-wide wholesale upgrading, including
studying the existing system design and operating issues, recent industry
events (e.g., the August, 2003 Northeast U.S. blackout), and the design
requirements of North American Electric Reliability Council (NERC). The
strategy describes how the functions in the latest microprocessor relays can be
arrayed for the most cost-effective and fully redundant protection, while
drastically reducing the amount of equipment from what was needed with
EYEWIRE
Authorized licensed use limited to: IEEE Xplore. Downloaded on January 21, 2009 at 09:13 from IEEE Xplore. Restrictions apply.
-
8/6/2019 Myrda Vision
3/13
preceding designs. The strategy shows how the IEC 61850
LAN protocol is to be introduced into the system design.
Monitoring functions that feed directly into the asset manage-
ment program are included in the design.
Levels of Implementation
These key steps in the strategy can be viewed at three levelsof implementation at the substation. The first level of imple-
mentation is that the substation must be designed to operate
successfully. Recent technological advances were assessed
for the ability to be immediately and successfully used to
operate the system. The second level of implementation is to
insure reliability of the substation; how well it can continue
to operate after the loss of any component. The third level of
implementation is the visibility of the substation system to
being able to track the other two levels. For example, when
substations supervisory controls were limited and the pro-
tection and control system was electromechanical, the sys-
tem was adequately operable, often reliable enough, but
almost completely opaque. Presence on-site might increase
the visibility of the substations operations, but postevent
analysis often involved deductive reasoning based on clues
gathered, such as the overcurrent relay didnt have a target,
the negative sequence relay had a target, and the operator
said he smelled something in the yard.
OperabilityThe core function of a transmission substation is to facilitate
the flow of power through a bulk electrical system. Therefore,
whatever design is used, be it fully automated or not, the
breakers have to close when power is needed to flow. Trans-
formers need to be energized when the load requires it. The
newest technologies may promise much, but if they cant be
implemented to reliably operate a system then they shouldnt
be called on to do so. The design philosophy for the program
was summed up by the phrase this design needs to be lead-
ing edge, not bleeding edge. Figure 1 summarizes the result-
ing design reflecting the strategy.
The substation operating system features an EthernetLAN that allows data gathering along with protection and
control commands to be exchanged. This LAN connects all
the relays with most of the other intelligent electronic devices
(IEDs) in the substation. The operating commands sent to the
yard equipment are sent over hard-wire connections. The
design for the off-site operation of yard equipment uses a cor-
porate wide-area network (WAN) via data communication
services. Operation by personnel on-site can be performed at
the HMI by mousing over the elements featured on a one-line
representation of the yard (Figure 2). Personnel can also
operate the 138-kV circuit breakers from buttons on the front
of relays (Figure 3).
Multifunction relays were used so even redundant protec-
tion could be afforded without using large amounts of panel
space. The relays, being IEDs, not only performed the crucial
function of protection but also specific features were required
to execute the design of operation. These IEDs are enabled
for IEC 61850 and featured programmable front-panel but-
tons (Figure 3) that could functionally take the place of test
switches. In addition, the design of the IEDs was flexible
enough that the core functions could be changed without
removing the relay from the panel. In other words, a trans-
former relay can become a line relay by reconfiguring the
relay and addressing the external connections; replacing the
hardware platform is not required.
34 IEEEpower & energy magazine may/june 2007
figure 1. New substation protection and control LAN architecture.
Managed Optical Ethernet Switches - LAN 1
Managed Optical Ethernet Switches - LAN 2
Line A Relay 2IEC 61850
and DNP 3.0
Local HMI
GPS Clock
RoutersPhysical and Electrical Isolation of Redundant Protection Systems
Xfmr Relay 2IEC 61850and DNP 3.0
Xfmr Relay 1IEC 61850
and DNP 3.0
Bus Relay 2IEC 61850
and DNP 3.0
Bus Relay 1IEC 61850
and DNP 3.0
PMU 2COMTRADE/IEEE
1344
SubstationAutomation Host
Local Historian
dDFR Host
METC Enterprise Service Providers
Monitoing IEDsSerial Comms Protocol
Connections for 1 msTime Stamp Synch
Corporate WANvia
Primary and Hot Standby DataCommunications Services
Other ControlCenters
SCADA/EMS
PMU 1COMTRADE/IEEE
1344
Line A Relay 1IEC 61850 and
DNP 3.0
Authorized licensed use limited to: IEEE Xplore. Downloaded on January 21, 2009 at 09:13 from IEEE Xplore. Restrictions apply.
-
8/6/2019 Myrda Vision
4/13
may/june 2007 IEEEpower & energy magazine
The end result of all these features was great freedom in
the physical design of the relay panels since the necessary
functions could be either hard wired, programmed into the
relay, or executed over the LAN. With this great flexibility in
physical design, standards could be created that would be
applicable to more scenarios and wouldnt change with every
new relay feature. Add to this a standardized entrance ofcables to the control house and how the cables are run to each
panel and now standardization can go beyond the panel. The
layout of the building could be standardized based on the
future build of the site. Any changes to the bulk electrical sys-
tem that may happen in the yard (the addition of a line or
transformer or a reassigned bus position) can be accommo-
dated with minimal physical construction.
No matter how far technology can take us, it must be
assumed that people will have to be able to operate the sys-
tem on-site. To help insure that the implementation of the
strategy would be able to be operated, a human factors engi-
neering evaluation was performed. The
objective was to inform the final design
and build out of the control houses and
their control panels of any problems
with the human factors and ergonomics
of the workspace, as well as the user
interface with the control-house controls
and displays. Recommendations of the
study were incorporated into the design.
ReliabilityIt is not enough that a substation operates
properly; it must also operate reliably
under credible contingency situations. In
the U.S.-Canada Power System Outage
Task Force report from April of 2004
titled Final Report on the August 14,
2003 Blackout in the United States and
Canada: Causes and Recommendations,
it was identified that one of the common
causes of the significant outages of the
last 30 years on the bulk electrical system
level was a lack of safety nets where
A safety net is a protective scheme that
activates automatically if a pre-specified,
significant contingency occurs. This is
an important concept at the substationlevel as well and any good design needs
to address safety nets. There are two
ways to fulfill the requirement for ade-
quate safety nets: either install those nets
or eliminate the credibility of a signifi-
cant contingency. It must be further iden-
tified that there are two types of threats to
the substation that require safety nets.
One is the internal threat to the system
due to normal failure over time of power
system and control-house components. The second is the
external threat to the substation instigated by forces from the
outside. The NERC identifies two types of electrical sector
threats: cyber and physical (see http://www.nerc.com/
cip.html). Cyber security addresses the attacks on the corpo-
rate WAN that pose external threats to the communication
between the bulk operating system and the substation. Theproject strategy was to emphasize cyber security and design a
comprehensive on-site security system to address the external
threats to the substations physical plant.
Internal Threats
The goal was to maintain the reliable operation of the substa-
tion by eliminating the effects of any single credible contin-
gency on the control house. The decision was that at the
345-kV level the control systems were to be fully redundant.
At the 138-kV level the systems werent required to have full
redundancy, but they must have backups in place for each
35
figure 2. An HMI displaying the substation one-line diagram.
figure 3. Front view of an 11-1/RH30 IED.
Authorized licensed use limited to: IEEE Xplore. Downloaded on January 21, 2009 at 09:13 from IEEE Xplore. Restrictions apply.
-
8/6/2019 Myrda Vision
5/13
credible contingency. The reason for the difference is that a
redundant system reflects the criticality of the electrical sys-
tem components at any one substation at this voltage. There-
fore, a redundant system is one where even extreme
contingencies have no effect on the operability of the substa-
tion, whereas a backup system may have reduced operability
under equally extreme circumstances. These redundant sys-tems can be referred to as System 1 and System 2.
Redundancy
What was found during design was that the technologies
adopted allowed the benefits of a standardized solution for
both levels of operation to outweigh the costs of most of the
redundancy required at 345 kV. In the world of multifunction
protective IEDs, the marginal cost between a redundant device
and a backup device, one that may have fewer functions, is not
significant; therefore, redundant devices were installed. Fur-
thermore, it had been decided that the benefits of diversity of
manufacturer was not significant enough to preclude evaluat-
ing benefits gained from using the same manufacturer on all
relays. So the design was free to use an identical device as the
redundant, driving standardization farther.
One of the early concepts behind the design was the identi-
fied value of redundant battery systems. A common practice in
transmission system protection is to protect for all credible sin-
gle contingencies including battery failure. Since it is common
for a substation to have one battery, its failure would leave the
substation unable to take any action to clear a fault condition.
Therefore, all remote sites have to act in place of the site withthe failed battery. With this requirement in place, distance
relays at the remote sites (sometimes referred to as Zone 3) had
to be set to see the other remote sites, which can be a very large
setting. The operation of the Zone 3 distance function during
periods of high load was identified as a contributing cause for
more than one of the major outages covered in the U.S.Cana-
da Power System Outage Task Force report. The redundant
battery minimizes the likelihood of a single credible event dis-
abling all operations at a substation, removing one of the needs
for the Zone 3 distance relay to be set high.
In the final design, the largest difference between the 138-
kV and the 345-kV systems is this requirement of two sepa-
rate batteries (Figure 4) at the higher voltage substations and
the physical separation of the redundant systems. Beyond the
need for two batteries and physical separation, the differences
between the systems using redundancy
or backup were subtle.
Since the control and protection sys-
tem relies heavily on the Ethernet LAN,
the failure of one of the switches or of a
fiber connection must have no effect on
either the ability or speed of communica-
tion. To accomplish this, a design strate-
gy similar to the redundant dc system
was adopted. The implementation of
redundancy can be seen in the two views
of the system architecture presented in
Figure 1 and Figure 5, specifically in the
application of redundant LANs.
Redundancy isnt enough, though.
You can have two eggs, but if theyre in
the same basket then the second egg may
not be worth much. An effort was made
to have physical separation between the
redundant elements. In this design,
redundant relays have a 6-ft aisle
between them. The two batteries are not
only in two different rooms but there aretwo battery chargers and two tray sys-
tems for getting cables from the batteries
to the relay panels. The redundant Ether-
net switches are also separated by an
aisle. However, whereas pains were taken
to eliminate the close proximity of wires
from System 1 to wires from System 2, it
was recognized that the communication
infrastructure had to be different. LAN 1
needs to know the status of LAN 2 and
36 IEEEpower & energy magazine may/june 2007
figure 4. Detail of the building layout showing two batteries.
EyeWash
125 VDC Battery Bank (Half)1 - 7 ft-0 in 1 ft-81/2 in Rack
125 VDC Battery Bank (Half)1 - 7 ft-0 in 1 ft-81/2 in Rack
125 VDC Battery Bank (Half)1 - 9 ft-0 in 1 ft-81/2 in Rack
125 VDC Battery Bank (Half)1 - 9 ft-0 in 1 ft-81/2 in Rack
ExhaustFan
ExhaustFan
1 in Conduit(Weather Station)
FanControl
EyeWash
FanControl
Weather
Station
Heater
System 1Battery Room
System 2Battery Room
Louver
Louver
Heater
Authorized licensed use limited to: IEEE Xplore. Downloaded on January 21, 2009 at 09:13 from IEEE Xplore. Restrictions apply.
-
8/6/2019 Myrda Vision
6/13
may/june 2007 IEEEpower & energy magazine
vice versa. Therefore, the design required a physical connec-
tion between the two LANs.
This application of physical separation was extended to the
termination cabinet design. The termination cabinet is the
point where all the cables come in from the yard that are
assigned to a system. This is a large wall-mounted box filled
with columns of terminal blocks. It was observed that theeffects of the failure of any wire termination followed by a fire
could wipe out an entire system. To address this, the physical
design of the box was revised to include metal plates between
the columns of terminal blocks. The blocks themselves were
mounted on plates that raised them from the back of the box.
This allowed the physical access to the blocks that the separat-
ing plates took away. The results are that the separating plates
will limit the effect that the heat and smoke of a fire at the ter-
minal blocks has on adjacent columns of blocks. Figure 6 is a
close shot of terminal block in the termination cabinet. As
youll notice, there is space on either side of the block closer
to the back wall of the cabinet. The separating plate is visible
as is a portion of a second block on the other side of the plate.
The pursuit of reliability has led us to the following: redun-
dant dc systems: redundant relays and redundant communica-
tions. Combine these with the prevalence of redundant trip
coils on transmission system circuit breakers and the need for
physical separation between redundant elements and a virtual-
ly complete System 1/System 2 approach is a natural out-
growth. This redundancy is complete to the point that the
entire System 1 could theoretically be taken out of service and
the bulk power system could still be operated through System2 at no loss of efficacy or speed. Redundancy not only brings
the system to a high level of reliability but also further enables
the modularity of design and increases the benefits of stan-
dardization. A line panel on System 1 is identical to a line
panel on System 2. The panel line up for System 1 reflects the
panel line up for System 2. The termination cabinet where the
System 1 cables come into the control house can be nearly
identical to the termination cabinet for System 2. This concept
simplifies the design efforts significantly.
External Threats: Cyber Security Plans for the Project
The many aspects and dimensions of cyber security for a
project like this are like the multiple ugly heads of the
mythical monster called the Hydra. Worst of all, when you
think youve dispatched one, another one grows to take its
37
figure 5. System architecture with greater connection detail.
Existing Control Building
ControlCenter 1
ControlCenter 2
Modem Modem
ModemSplitter
ModemSplitter
Existing RTU
SubstationNo
NewRTU
SecureServer
Master SiteServer
Runtime
MQTTMPLS
Network
Switch
T1 Relay
T1 Relay
Red BusRelay
Blue BusRelay
TransformerMonitors
Pots
PhoneSwitch
Telephone
T2 Relay
T2 Relay
B kr Relay
B kr Relay
B kr Relay
B kr Relay
B kr Relay
B kr Relay B kr Relay
B kr Relay
L7 Relay L4 Relay
L4 Relay
L5 Relay
L5 Relay
L7 Relay
L8 Relay
L8 Relay
138 kV
L3 Relay
L2 Relay
L2 Relay
L1 Relay
L1 Relay
Switch
HMI No
MISC No
(SecuritySystemAlarms)
Inverter
ServerRuntimeLocalSurveillanceMonitor
Switch
GPS Clock
DVMre Receiver
Switch
Server
Legend
Copper
10/100 Base FL10/100 Base TX100 Base FX
Gateway
WirelessBackup
Substation
PLC
PLC
PLC
PLC
PLC
PLC
PLC
PLC
PLC
345 kV
L3 Relay
Authorized licensed use limited to: IEEE Xplore. Downloaded on January 21, 2009 at 09:13 from IEEE Xplore. Restrictions apply.
-
8/6/2019 Myrda Vision
7/13
place. Some of the ugly heads include security management
practices, access control systems, network and telecommu-
nications security, security architecture, encryption, applica-
tion security, and physical security to name a few.
This section will focus on the network/telecommunica-
tions security and the overall security architecture. Figure
5 gives you an overall view of the architecture of the proj-ect. As with most cyber security architectures, much of
the defense in depth comes from the multiple security
levels or zones. The most secure area of the architecture
is the local substation network that connects directly to
the relays. As designed, each relay has its own IP switch
connection. Not only does this eliminate any potential
collisions but these switches have media access control
(MAC) address filtering capability, thereby adding anoth-
er level of security in the overall architecture. The relays
have multiple levels of passwords and audit logging to
insure access only by authorized personnel. The next level
up includes the computer that is responsible for scanning
the relays and then reporting the results to the world.
This security layer isolates the relays from communicat-
ing with multiple clients, leaving them free to do their job
of protecting the grid.
So how does the information go from the IEDs to the
Boardroom? Well, enter the multiprotocol label switching
(MPLS) network. Since a corporate SONET network was not
available throughout Michigan, the use of a public/private
network is the next logical alternative. Enter MPLS or net-
works that have now become a fundamental building block
used by many of the large Internet service providers as their
backbone. The key to the security of these networks is that
the entire IP address space is available to each client that sub-
scribes to the network service and that the core network rout-
ing protocols are completely invisible
to the client and visa versa. MPLS
networks have quietly been providing
IP connectivity for several years to
business-critical applications. The
next layer in the security architecture
are the boundary routers that are pro-
grammed to route only specific IP
addresses from one place to another.
This network connects all the substa-
tions in a many-to-many communica-
tions network that includes thecorporate data center.
We now enter the corporate data
center (Figure 7) and the notorious
corporate firewall or, more appropri-
ately, the corporate wall of Swiss
cheese. The classic firewall model is
to close all ports until it is demon-
strated that there is a need to have the
port open. While this is a good strate-
gy and represents our next security
layer, closing down all ports is a real pain operationally. It
virtually guarantees that any new application wont run until
that port/service is enabled. However, cyber security was
never about making life easy but rather minimizing risk, so
no pain no gain. Recognize that once all the applications
are implemented and the firewall is properly configured,
there are numerous holes punched through the firewall,and hence the notion of a wall of Swiss cheese. Typically,
the firewalls are used to implement several additional layers
of security. First, the real time systems are fire walled off
from the corporate network and the corporate network is fire
walled off from the Internet. One of the key techniques to
support this isolation is the use of dedicated/fixed IP
addresses for all servers. This allows firewall rules to be
written that expressly allow traffic to and from a specific
server on a specific port. Corporate applications traffic such
as e-mail, file sharing, terminal server, etc., are prohibited
from entering the real-time/substation network. Most of
the real-time data are stored in relational or specialized time-
series databases, thereby further isolating the substation
LAN from direct contact with the outside world.
Finally, we get to the dreaded Internet with all its virus-
es and cyber threats. At a minimum, only those servers that
absolutely need to serve Web traffic have a connection to the
outside world. Also, where possible, weve established a
DMZ (demilitarized zone, a semiprotected LAN segment)
where the Web servers operate, communicating through one
of the firewall ports back to the corporate servers.
This design provides layer upon layer of security. One
more step in keeping the ugly Hydra/cyber security heads
dispatched: AUDIT, AUDIT, AUDIT! No matter how many
sleepless nights you have spent designing the most bullet-
proof architecture, you dont really know until you bring in
the white hats (friendly hackers)
to attack the network and look for
vulnerabilities on how good your
design really is. A key part of the
plan is to perform these audits using
an independent firm and one that is
familiar with recent work going on
at the national labs. In particular, the
Department of Energy jointly estab-
lished the National Supervisory
Control and Data Acquisition
(SCADA) Test Bed program atIdaho National Laboratory and San-
dia National Laboratory.
Physical Security
Among the requirements NERC sets
in CIP-006, which addresses physical
security, are physical access controls,
monitoring physical access, and log-
ging physical access. The substation
design leveraged technology to effec-
38 IEEEpower & energy magazine may/june 2007
figure 6. Detail of a termination cabinet.
Authorized licensed use limited to: IEEE Xplore. Downloaded on January 21, 2009 at 09:13 from IEEE Xplore. Restrictions apply.
-
8/6/2019 Myrda Vision
8/13
-
8/6/2019 Myrda Vision
9/13
check the surrounding area for indications of why the trip
took place.
One of the first devices that greatly improved visibility
in substations was the digital fault recorder (DFR). DFRs
are stand-alone devices connected to representative inputs
that provide crucial information regarding system condi-
tions during events. The project design strategy has an
alternate approach. Since the relays installed already
record oscillographic data, a computer was installed dedi-
cated to the retrieval of these files from all the relays.
These files were then stored by time. As more substations
come online, the implementation plan for these computers
was to upload the oscillography to a central server organ-
ized by substation. Figure 8 is a screen shot showing
events gathered from numerous sources arranged chrono-
logically. Office analysis of system-wide events could now
happen within minutes of the event.
Digital data acquisition affords a level of visibility never
considered possible in the electromechanical era of substa-
tions. Exhaustive routine maintenance of every connection
between relays and auxiliary relays was a requirement
because this was the only way the failure of wires or their
terminations could be discovered. With the adoption of IEC
61850 for the delivery of protection related commands,
every data connection between relays is under constant
scrutiny. If, for whatever contingency, that protection func-
tion is unavailable, not only is there a redundant function in
place but also this failed state is alarmed immediately. This
is the equivalent of having every wire tested every few
minutes in an older substation.
With the dependence on IEDs for protection, control,
and communication, it is crucial that the IEDs themselves
are visible. As described, the design of the project reduces
physical installation. Auxiliary relays have been eliminated,
test switches are nearly extinct, and instead of hundreds of
wires strung between relays, there are now two pairs of
fibers from each relay to Ethernet switches. However, with
the elimination of the physical comes the proliferation of
the digital. What was once communicated with detailed dc
schematics and recorded relay settings now must be accom-
plished with settings files and logic diagrams. And control
of the configuration of the relays is critical to the reliable
operation of the substation. These newer functions are being
supported by IED manufacturers efforts to make the work-
ings of multifunction microprocessor relays increasingly
visible. The IEDs used in these substations have software
available that automatically converts relay configuration
into logic diagrams and easily understandable settings
reports. The software will also document inter-relay rela-
tionships, reducing the time spent on documentation. Figure
9 shows a small portion of the logic diagram representation
of the configuration for one of the relays.
Even though all this visibility exists, it is at a resolution
that is not easily understood. It would be an input overload
if someone were trying to assess the data in a real-time
manner. So once you access the information, it must be
stored for later analysis. It is at this point that the invest-
ment in automation really comes through and the vision of
automation is realized. The next step is to convert the data
into business intelligence.
40 IEEEpower & energy magazine may/june 2007
The design philosophy for the programwas summed up by the phrase this designneeds to be leading edge, not bleeding edge.
figure 9. Relay software-generated logic diagram.
BKR FAIL 1 TRIP OP 30H9BFR ON (VO39)
CONTROL PUSHBUTTON 2 ON
LATCH 1 ON
AND
116
BFR RST On (VI21)
LATCH 1 OFF
117
OR
118
30H9BFR OFF (VO40)
AND
Authorized licensed use limited to: IEEE Xplore. Downloaded on January 21, 2009 at 09:13 from IEEE Xplore. Restrictions apply.
-
8/6/2019 Myrda Vision
10/13
may/june 2007 IEEEpower & energy magazine
Data Warehousingand Information AccessThe newest relays and communications systems selected for
the project present the enterprise with a massive stream of
substation data that must be automatically stored, managed,
analyzed, and presented in useful forms that improve busi-
ness and technical operations. The data warehousing fea-
tures of modern information architectures are essential to
provide end users with easy access to the wealth of data and
information substation devices provide. Three types of data
are being created and stored for later retrieval as needed:
sequence of events records
high-speed time-series data records such as COM-
TRADE oscillography and phasor measurement files,
including binary status such as relay trip and close or
unit line protection communications signals
analog power system measurements and reports from
equipment monitors such as transformer analysis IEDs.
All the data are collected, organized, and archived at the
data-hosting center using the modeling standard CIM and
providing easy access by the staff. Today, CIM is embodied
within IEC standards 61968 and 61970 and the project is
benefiting from these standards through its use of readily
available adapters that can be used to rapidly integrate data
from various applications. Various diagnostic tools, including
automatic preprocessing and dashboard reporting, are being
developed to aid the end user in analyzing this wealth of
information. In conjunction with the upgrade project is an
41
It is not enough that a substation operates properly;it must also operate reliably under crediblecontingency situations.
figure 10. The basic architecture of the decision support system.
Portlets
FinancialDB's
Utility Databases
EventDB
Performance
DB
GISDB
CMMSDB
EMSDB
Sources:StaffContractorsSuppliersCustomersField ForceRegulators
DataHistorian
MonitorPoints
MonitorEvents
CalcEvents
CalcPoints
Substation Devices
Utility Real Time Data
Remote AssetMonitoring
Tools
ExpertGrid
Analyzers
Portal Server
WebPages
Dashboards,Scorecards,
OLAP, Reports
ApplicationServer
IEC 61968/61970 Compliant Middleware
CIMData
Warehouse
Real-TimeEvent
Analyzers
Analytics Stack
OptimizationComputations
DashboardTools
OLAP and
Hypercubes
CustomAnalytics
Data Miner
NotificationServer
ETL
Authorized licensed use limited to: IEEE Xplore. Downloaded on January 21, 2009 at 09:13 from IEEE Xplore. Restrictions apply.
-
8/6/2019 Myrda Vision
11/13
integral effort to develop information systems for the staff.
These systems will benefit asset management, system plan-
ning, and operations.
Information dashboards that support each of these busi-
ness areas are currently under development. For example,
operations will have ready access to fault location informa-
tion, SER, lightning strikes, transformer monitors, weather
and video streams, and operating performance information.
Planners will benefit with more accurate system and device
loading information, better system event capture, weather
profiles, and improved system models.
Asset management will have access to an immense
array of information that will be distilled from the
detailed operating data. For example, consider trans-
former monitoring and life management. The transformer
relays report currents and voltages. Oil condition is moni-
tored with installed gas-in-oil detectors. Top-oil tempera-
ture sensors and accessory alarms connect to data-
collection IEDs. A weather station reports ambient condi-
tions. This body of data can support life assessment and
emergency operating decisions. To get these results, the
key data is extracted and analyzed with modeling algo-
rithms and stored as trend results for each transformer.
Self-organizing neural networks preprocess the vast
amounts of operational data for the operating and mainte-
nance management personnel who get prioritized succinct
information on which they can act, quickly if needed.
Also, data or alarms indicating maintenance problems or
repair issues can act as triggers. These can originate in the
substation or with back-office processing functions. These
drive notices to business partners; create work orders and
status tracking, map issues to geographic information sys-
tems, update asset management records, and search for
patterns or issues requiring broad action.
Converting Data intoBusiness IntelligenceRecently there have been a number of initiatives in the U.S.
power industry around the notion of an intelligent grid.
Along these lines, one of the core elements of this project
is to capture all of the data available at the substation and
stream it to a central location for decision making as well
as operational support.
The substation data sources are the following:
advanced IEDs for circuit breaker operation
digital fault recorder
transformer monitoring systems
security system
weather station (not shown on the diagram)
phasor measurement units (PMUs)
The automated substations contain distributed data his-
torian servers that collect data from the substation data
collection system (DCS) and send them to the central data
historian server. Unlike a relational database, data histori-
ans provide an efficient means of storing temporal data
(time-series data) using various algorithms to essentially
compress the data. One needs to use caution in their
selection of historian vendors to assure that the compres-
sion method is adequate for the intended use. We chose to
use a vendor that implements a lossless method. The com-
munications network and quality of service controls are
used to prioritize data traffic from the substations, with
SCADA having the highest priority. The data historian
real-time service receives IED data from the DCS and for-
wards them to the historian server. This is the means by
which real-time data are made available to the centralized
data warehouse providing a means for business analytics
to be performed.
The project identified four levels of analytics:
Level 1 Simple thresholds and alarms: monitoring trans-
former oil temperature.
Level 2 Financial trends, basic system performance met-
rics: budget versus actual, TSAIDI, TSAIFI, etc.
Level 3 Real-time event analysis, interpretation of event
sequences: diagnosing circuit breaker failure
modes from DFR waveform data.
Level 4 Analytics for optimization purposes: prioritiza-
tion of asset maintenance, asset replacement.
The decision support system (Figure 10) implements the
data integration, analytics, and information distribution
functions. The central data repository is a data warehouse
that is structured in compliance with the CIM for utilities.Use of CIM is central to the concept of open standards.
Information is disseminated from the data warehouse and
analytics stack throughout the enterprise via Web services
and portals.
Figure 11 shows how the analytics provide decision sup-
port for operations and business functions. The Level 1 ana-
lytics (parameter thresholds and notifications) are quite
extensive and require sophisticated management to permit
each authorized user to subscribe to only those notifications
that are of interest in the users job role. Any authorized user,
42 IEEEpower & energy magazine may/june 2007
Redundancy not only brings the system to a high level ofreliability but also further enables the modularity of designand increases the benefits of standardization.
Authorized licensed use limited to: IEEE Xplore. Downloaded on January 21, 2009 at 09:13 from IEEE Xplore. Restrictions apply.
-
8/6/2019 Myrda Vision
12/13
may/june 2007 IEEEpower & energy magazine
from maintenance engineer up to chief operations officer, can
subscribe to notifications as desired. This is likewise true for
higher-level analytics, key performance indicator dashboards,
and decision support analyses.
The CIM data warehouse and data historian jointly sup-
ply the data to drive a variety of analytics. The project has
defined a large set of analytics, including 19 system per-formance metrics and many financial and operational
measures. The analytics architecture supports both opera-
tional metrics and business key performance indicators and
each person in the organization can receive the relevant
analytics and support data and can customize his or her
portal to show preferred information in preferred locations
and formats.
Through careful consideration of the relevant key busi-
ness drivers, the project arrived at a suitable intelligent
grid strategy. Having created a business model that relies
heavily upon outsourcing, advanced automation, and the
use of analytics to support operational and business deci-
sions, the team developed an architecture that uses tech-
nology to support both the outsourcing strategy and
minimal staffing by making maximum use of information
sources and tools.
The business case for this approach shows the value of
advanced automation and decision support tools to be con-
tained in the following:
reduced operation and maintenance expenditures
reduced capital expenditures low staffing requirements
increased transmission system reliability
preservation of the value of infrastructure through use
of open standards.
Realization of these benefits is achieved by maintaining
a low headcount, using remote monitoring to reduce field
manpower through reduction of both scheduled and
unscheduled visits to their widespread collection of substa-
tions, reducing CAPEX and operation and maintenance
costs through improved information-based asset manage-
ment and reliability-centered maintenance, and using open
standards to guide the selection of equipment, systems, and
architectures that minimize the future impact of changes in
any one system, component, or supplier.
43
figure 11. Analytics support for business functions.
Utility Databases
FinancialDB's
PerformanceDB's
MaintenanceDB
GISDB
CMMSDB
IncidentDB
Source:StaffContractorsSuppliersCustomersField ForceRegulators
DataHistorian
MonitorPoints
MonitorEvents
CalcPoints
CalcEvents
Utility Real Time Data
Analytics L.3
Real Time Event AnalysisAdvanced Diagnostics
Analytics L.1
Thresholdsand
Alarms
Integer Programming ML Estimators and Classifiers
Integral Maximization, Linear and Nonlinear Programming,Dynamic Programming, ACO/PSO, Simulated Annealing,Search Techniques
Optimization Tools
Analytics L.4
Analytics L.2
Data Mining, Clustering,Regression, CART,Model Construction
Grid Meta Data
CIMDataWarehouse
OLAP,
Simple Meterics
AssetNormalized
Models
Constraints
BudgetCash FlowRegional
ManpowerRegulatory
Inter-Department
Dashboards,Scorecards,Cube Views,
Reports
Solutions:Prioritization,Subsetting
Strategic Functions
Asset Life Cycle ManagementGrid Expansion Planning
System Performance AnalysisCAPEX OptimizationFault Mitigation PlanningPerformance Metric ImprovementPost-Fault Analysis
Operational Functions
Work ManagementPredictive MaintenanceReal Time Event InterpretationGrid ControlAsset Utilization OptimizationEvent-Based Maintenance
Utility Front and Back
Office Functions
Implement Decisions and Control
Transform Data into Information
Collect Low Level Data and Events
E
TL
Authorized licensed use limited to: IEEE Xplore. Downloaded on January 21, 2009 at 09:13 from IEEE Xplore. Restrictions apply.
-
8/6/2019 Myrda Vision
13/13
Future SightInvesting in a robust communication infrastructure, a nimble
LAN-based operating system and the computing power of
IEDs put into place a system that is flexible enough to be
enhanced in the future, especially in the area of visibility.
PMUs are part of an emerging technology. The project
implementation included the installation of PMUs on all
345-kV buses. This arrangement provided for monitoring of
all lines and transformers associated with the station.
Improved visibility and operator situational analysis were
key factors in implementing this technology. One of the key
findings of the August 2003 blackout was the lack of opera-
tor awareness during the time leading up to the blackout.
PMUs offer substantially improved intelligence not only for
real-time operations but also for postevent analysis and sys-
tem model validation and more.
Farther down the path of substation automation is the
use of IEC 61850 GOOSE messages over the corporate
WAN to other substations. If this method doesnt reach the
speed of communication over power line carrier or fiber
communications for carrying pilot protection data, then it
might be used as a backup. This might also be a way to
perform inter-substation remedial action schemes and wide
area protection or monitoring schemes.
The Vision of Substation AutomationRecognizing the limitations of the term substation automa-
tion, the concept it represents is realizing all the benefits
that digital technology can bring to the substation. The
overall strategy is not only to automate substations but to
optimize them. Substation automation is not only a protec-
tion issue, its not a metering issue, and neither is it a super-
visory control or a data acquisition issue. The vision of
substation automation is not only system wide but system
deep. In this case, it not only runs from the Mackinac
Bridge to the wrist of the Michigan mitten, substationautomation affects the system from the terminal block to the
345-kV circuit breaker. That three-breaker substation east of
the Michigan Dunes is the beginning of the realization of
the vision of substation automation.
AcknowledgmentsIEC is a registered trademark of Commission Electrotech-
nique Internationale. Google is a registered trademark of
Google Technology, Inc. SONET is a registered trademark of
SONAT, Inc.
For Further ReadingR. Brantley, K. Donahoe, J. Theron, and E. Udren, The
application of IEC 61850 to replace auxiliary devices includ-
ing lockout relays, presented at the 60th Annual Georgia
Tech Protective Relaying Conference, Apr. 2006.
U.S.-Canada Power System Outage Task Force, Final
report on the August 14, 2003 blackout in the United States
and Canada: Causes and recommendations, Apr. 2004
[Online]. Available: http://www.nerc.com.
R. Krutz and R. Vines, The CISSP Prep Guide: Mastering
the CISSP and ISSEP Exams, 2nd ed. New York: Wiley, Apr.
2004.
Biographies Paul Myrda has 30 years of experience in electrical power
systems engineering. Most recently he was director of opera-
tions and chief technologist for Trans-Elect Inc. He was
instrumental in developing an overarching strategy in asset
management and championed an innovative protection and
control system upgrade project for the Michigan Electric
Transmission Company, a former affiliate of Trans-Elect. This
project fully leveraged the capability of IEC 61850-based
microprocessor relays, physical security, telecommunications,
and data warehousing technologies using EPRIs common
information model. His diverse background includes planning,
engineering, information systems, and project management.
He has an M.B.A. from Kellogg Graduate School of Manage-
ment and an M.S.E.E. and a B.S.E.E. from Illinois Institute of
Technology. He is a licensed professional engineer in Illinois,
a member of CIGRE, and a Senior Member of the IEEE.
Kevin Donahoe has spent the last 25 years working in the
electric utility industry. The last 22 of those years have been
spent testing, installing, trouble shooting, specifying, setting,
estimating, designing, reviewing, documenting, and setting
standards for protection and control schemes. He spent
20 years with Commonwealth Edison, an Exelon company,before moving to GE Energy. Though the majority of his
experience has been with transmission and distribution sub-
stations, he has significant experience with generation protec-
tion and distribution protection with specific experience with
interconnection requirements. He received his B.S.E.E. from
the Illinois Institute of Technology in 1981 and in 1993
received an M.B.A. from Lewis University. Donahoe is a
member of the IEEE Power System Relaying Committee and
the IEEE Standards Advisory. He is a licensed professional
engineer in Illinois, Oklahoma, and Michigan.
44 IEEEpower & energy magazine may/june 2007
p&e
The many aspects and dimensions of cyber securityfor a project like this are like the multiple ugly heads ofthe mythical monster called the Hydra.