Power Systems/Communication

System Co-Simulation and

Experimental Evaluation of Cyber

Security of Power Grid

Yi Deng, Sandeep Shukla,

Hua Lin, James Thorp

February 5, 2014

9th Electricity Conference at CMU

Outline

1. Introduction

2. Power Systems and Communication System Co-Simulation: GECO: a

Modulized Global Event-driven CO-simulation Platform

3. Cyber Attack Simulation on PMU-based State Estimation

4. Co-simulation Case Study on PMU-based Out-of-step Protection

5. Conclusion & Future Research

2

1: Introduction

3

Power Plant

Transmission Substation

Distribution Substation

Individual Consumer

Public Consumer

Industry Consumer

Urban Area

Rural Area

Monitor & Control

Operation & Marketing

1

Wind

Nuclear

Solar

Hydro

2

Software Infrastructure

New Strategy

3

3

4

GE’s Solution on Wide Area Monitoring and

Control – Synchrophasor Techniques

4

* From GE’s Industrial Solution Website

Dominion Synchrophasor Project

5 ** From Dominion project and VT project report 21 500kv station, 5 230kv station, 115kv station

6

Hua Lin; Veda, S.S.; Shukla, S.S.; Mili, L.; Thorp, J., "GECO: Global Event-Driven Co-Simulation

Framework for Interconnected Power System and Communication Network," Smart Grid, IEEE

Transactions on , vol.3, no.3, pp.1444,1456, Sept. 2012

2: Global Event-Driven Synchronization

7

Dynamic Simulation Procedure of Power Systems Communication Network Simulation Procedure

t0 t'

Δt

t

Initialize

all state

variables

Calculate

network

boundary

variables

Calculate

next

variables

Calculate

state

variable

derivatives

Integration

step

t = t' + Δt

One simulation round

t1 t2

Δt

Time Hopt

Ev

ent

1

Ev

ent

2

Ev

ent

3

Node 1 Node 2

Event 1: node 1 sends packets to node 2

Event 2: node 2 receives packets from node 1

Node 3

Node 4

Event List Queue:

t0

t

t1 t2

t

Even

t 1

Even

t 2

Even

t 3

Even

t 4

Δt = T

Error 1

Error 2

Start Synchronization Point 1

Synchronization Point 2

Power System Time Scale

Communication Network Time

Scale

t0

t

t1 t2

t

Ev

ent

1

Ev

ent

2

Ev

ent

3

Ev

ent

4

Start

Power System Time Scale

Communication Network Time

Scale

Ev

ent

1

Ev

ent

2

Ev

ent

3

Ev

ent

4

Global Event Queue

Two types of synchronization errors Event-driven synchronization without errors

GECO (Global Event-driven CO-simulation): Platform

Structure

8

Basic

Model

Dynamic

Model

PSLF Simulation

PSLF

Interface

Power

Communication

Protocols

NS2 Simulation

NS2

Interface

Power

Applications

Power

Applications

…………

………………

………………

Global

Scheduler

Global

Event List

9

GECO: A Modulized Global Event-driven CO-simulation platform

Power System Simulator Platform

Application-Specific Physical System Simulators

GE’s Positive Sequence Load Flow

(PSLF)

Cyber Network Simulators

Network Simulator 2 (NS2)

Basic ModelDynamic

Model

Power System Interface Middleware “epcmod”

Global Scheduler Global Event Queue

Communication Network Simulator Middleware “tcl_PSLF”

Messages, Shared Memory,

Formatted Files, Mediators, etc.

Simulator Integration Layer

Communication Network System Simulator Platform

Global Scheduler Global Event Queue

SCADA Communication Protocol Package Layer:

Modbus, DNP3, ICCP, Profibus, Ethernet, TCP/IP, IEC 61850

Physical System Application Packages

State

Estimation

Cyber Events Applications

Cyber

Attacks

Network

Contingency

Out of Step

Protection

Electric

Marketing

3: Problem Statement: Attack ModelMalicious Data Injection attack on State Estimation

10

G

P1, Q1

v1, 0

P3, Q3

P12, Q12

1 2

3 4

P13, Q13

P23, Q23

P34, Q34

P31, Q31

v2, Ɵ2

v4, Ɵ4v3, Ɵ3

RTU

RTU

RTU

RTU

RTURTU

RTU

SCADA Master

Operator

𝒛 = 𝑯𝒙 + 𝒆

𝒙 = 𝑯𝑻𝑾−𝟏𝑯−𝟏𝑯𝑻𝑾−𝟏𝒛

Bus location

Vo

lta

ge

ma

gn

itu

de

𝒛𝒂 = 𝒛 + 𝒂

𝒂 = 𝑯𝒄

𝒛𝒂 −𝑯 𝒙𝒇 = 𝒛 − 𝑯 𝒙 ≤ 𝝉

We can’t detect the attacks

The injected data will modify

the state estimation results

The Placement of PMUs

11

1

23

4

9785

6

12 1314

11

10

IEEE 14-Bus Example

PMU1PMU2

PMU3Test system PMUs Number

IEEE 14-bus 3

IEEE 24-bus 6

IEEE 30-bus 7

New England 39-bus 8

IEEE 57-bus 11

Minimum number of critical

places for installing PMUs

Secured PMUs installed in these

places make the system observable

Case study: New England 39-bus test system

12

PDC1

SPDC

PDC2

PDC3 PDC4

Cyber attack Simulation: on network channels

13

Single Network Link Failure

Saturation attacks

Bus16-Bus17 (Tp=50ms) Bus16-Bus17 (Tp=60ms)

Network saturation 85%Network saturation 50%

14

Denial of Service Attack

Data Spoofing

DoS attack on the router at Bus 16 Enhanced DoS attack

Cyber attack Simulation: on network nodes

PMU spoofing on Bus 3 PMU spoofing in contingency

4: Out-of-Step Protection

15 Equal Area Criterion

Out-of-Step (OOS) means a

generator or a group of generators

lose synchronism with the rest of

the system.

Cyber attack on power generator by Idaho lab

Out-of-Step Protection

16

• Out-of-Step (OOS) means a

generator or a group of generators

lose synchronism with the rest of the

system.

Fault cleared in 0.3 second, OOS condition is observedFault cleared in 0.1 second, system back to normal condition

• One effective method is to run time-

domain dynamic simulations and

monitor the generator angles.

PMU-based Out-of-Step Protection

• Protection Scheme

• Four Steps

17

Islanding Algorithm

Measure Rotor Angles using adequate PMUs

Identify Coherent Generator Groups using

offline simulations

Predetermine Islanding Locations

Island Asynchronous Generator Groups

Real-Time Generator Clustering Algorithms

Input Output

Two Coherent Generator

Groups

Threshold

Algorithm 1: Sorting, then check neighboring

element distance

Algorithm 2: Match elements into existing

clusters sequentially

Group 1

Group 2

Rotor Angles of the Generators

Equivalence of islanding to s t min-cut problem

Clustering Algorithm for Coherent Groups

• Clustering algorithm refers to a

group of algorithms whose goal

is to divide data into subsets

based on certain criteria.

18

• The first algorithm sorts the

measured rotor angle and

traverse the measured rotor angle

sequentially. If the gap between

two neighbors is greater than 120

degrees, then the OOS condition

is identified.

• An alternative second algorithm

processes the measured rotor

angle one by one.

Islanding Algorithm

19

Equivalence of islanding to s − t min-cut problem

• As long as we have found two

coherent generator groups S

and T, the next step is to find

a minimum cut of the entire

power system that can

separate S and T.

• Edmonds-Karp algorithm

which is O(|V ||E|2)

A max-flow example Find the min-cut on the residual network

Simulation Results

20

0 2 4 6 8 10

-20

0

20

40

60

80

100

120

Ge

ne

rato

r A

ng

le (

de

gre

e)

Time (s)

0 2 4 6 8 10

-0.1

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Ge

ne

rato

r R

ea

l P

ow

er

Ou

tpu

t (p

.u.)

Time (s)

0 2 4 6 8 10

-20

0

20

40

60

80

100

120

Ge

ne

rato

r A

ng

le (

de

gre

e)

Time (s)

0 2 4 6 8 10

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

Ge

ne

rato

r R

ea

l P

ow

er

Ou

tpu

t (p

.u.)

Time (s)

Generator angels showing OOS condition

(BW=1Gbps, D=5ms)

Generator real power outputs

(BW=1Gbps, D=5ms)

Generator angels with link failure

(BW=100Mbps, D=10ms)

Generator real power outputs with link failure

(BW=100Mbps, D=10ms)

5: Conclusions & Future Research

• Implemented a co-simulation platform GECO, and integrated the

dynamic state estimation and the out-of-step protection modules in

the platform.

• Launched two case studies (all-PMU based state estimation and

PMU based out-of-step protection) to reveal the cyber security

vulnerabilities on co-simulation platform.

• Cloud-based virtual SCADA testbed for cyber security research

• Centralize & Modulize computing and communication resources

• Replaceable different communication protocols for security research

• Seamlessly interact with power/control system simulators.

21

Virtual SCADA Testbed for Cyber Security

Research

22

SCADA MasterServer

RTUs HMI

MatrikonOPC serverL1

OPC I/O drivers in iFixD1

assigns the data to a tag in the iFix database manager

OPC I/O drivers in iFixD2

monitors the tag in D1 s database

Data Source Attack!

Database Attack!

OPC I/O drivers in iFixD3

Access Control

Cloud-based Virtual SCADA Infrastructure in VT

23

Linux OS

VM_1Windows

iFIXTCP/IP

VM_2Windows

iFIXTCP/IP

VM_NWindows

iFIXTCP/IP

Usr1 Usr2 UsrN

User_1

User_2

RTUs/OPC Servers

iWebSpace Server

Windows Server

Hyper-ViFIX

TCP/IP

Hyper-ViFIX

TCP/IP

Hyper-ViFIX

TCP/IP

Usr1 Usr2 UsrN

Admin/TAs

SCADA Server

VT Private Cloud

References1. Hua Lin, Yi Deng, Sandeep Shukla, James Thorp, Lamine Mili. "Cyber Security Impacts on All-

PMU State Estimator - A Case Study on Co-Simulation Platform GECO", Third International IEEE

Conference on Smart Grid Communications (SmartGridComm), November, 2012, Tainan City,

Taiwan.

2. Yi Deng, Sandeep Shukla, “Vulnerabilities and Countermeasures - A Survey on the Cyber

Security issues in the Transmission Subsystem of a Smart Grid”, Journal of Cyber Security and

Mobility, invited paper, 2012

3. Yi Deng, Hua Lin, Arun G. Phadke, Sandeep Shukla, and James S. Thorp, “Networking

technologies for wide-area measurement applications” book chapter, “Smart Grid

Communications and Networking” to be published, Cambridge University Press, UK, 2012

4. Yi Deng, Hua Lin, Arun G. Phadke, Sandeep Shukla, James S. Thorp, Lamine Mili,

“Communication Network Modeling and Simulation for Wide Area Measurement Applications”

IEEE PES Conference on Innovative Smart Grid Technologies, Jan. 2012

5. Yi Deng, Shravan Garlapati, Hua Lin, Santhoshkumar Sambamoorthy, Sandeep Shukla, James

Thorp, Lamine Mili, “Visual Integrated Application Development for Substation Automation

Compliant with IEC 61850” PAC World Conference 2011, Dublin, Ireland, June 2011

6. H. Lin, S. Sambamoorthy, S. Shukla, L. Mili, J. Thorp, “GECO: Global Event-Driven Co-

Simulation Framework for Interconnected Power System and Communication Network”. IEEE

Transactions on Smart Grid, accepted, 2012

7. 1: Yi Deng; Hua Lin; Shukla, S.; Thorp, J.; Mili, L., "Co-simulating power systems and

communication network for accurate modeling and simulation of PMU based wide area

measurement systems using a global event scheduling technique," Modeling and Simulation of

Cyber-Physical Energy Systems (MSCPES), 2013 Workshop on , vol., no., pp.1,6, 20 May 2013

24

Thanks for your attention!

25

{yideng56, birchlin, shukla, jsthorp}@vt.edu