proceedings of the ieee, vol. x, no. xx, pp. 1 -23, 2017 1 … · 2017-08-07 · proceedings of the...

PROCEEDINGS OF THE IEEE, VOL. X, NO. XX, PP. 1 -23, 2017 1

Smart Grid Simulations and Their SupportingImplementation Methods

Aranya Chakrabortty, Senior Member, IEEE, Anjan Bose, Fellow, IEEE(Invited paper)

Abstract—In this tutorial we present state-of-the-art as well asnew methods for simulating various planning, operation, stability,reliability, and economic models of electric power systems. Thediscussion is driven by both first-principle models and empiricalmodels. First-principle models result from the fundamentalphysics and engineering that govern the behavior of variouscomponents of a grid. Empirical models, on the other hand, aremodels that result from statistics and data analysis. We overviewa wide spectrum of applications starting from planning modelswith a time-scale of simulation in years to real-time modelswhere the time-scale can be in the order of milliseconds. Wepresent a list of simulation software that are popularly used bythe power engineering research community across the world. Theincreasingly important roles of power electronics, communicationand computing, model aggregation, hybrid simulation, faster-than-real-time simulation, and co-simulation in emulating thedaily operation of a grid are enumerated. The importanceof research testbeds for testing, verification and validation ofcomplex grid models at various temporal and spatial scales isalso highlighted. The overall goal is to provide a vision on howsimulations and their supporting implementation methods canhelp us in understanding the evolving behavior of tomorrow’spower networks as a truly intelligent cyber-physical system.

Index Terms—Simulations, modeling, stability, planning, oper-ations, power electronics, electric markets, cyber-physical systems

I. INTRODUCTION

Simulation is defined as the process of using a mathe-matical model to study the behavior and performance of aphysical system. When it comes to understanding the behaviorof an enormous, complex, multi-time-scale system such asan electric power system, simulations are the primary toolthat one has to rely on. Accurate simulations can guide theoperation, planning, and control of power systems at all levels -generation, transmission, distribution, consumers and markets[1]-[3]. They are essential for analysis and evaluation of powersystem operations. They improve system planning and design.They help in recognizing and in mitigating potential hiddenproblems in the grid, avoid unforeseen errors, and preventsystem interruption, and can determine under-utilization ofsystem resources. They accelerate engineer and operator train-ing, reduce design and commissioning time, and help powersystem operators in avoiding inadvertent plant outages causedby human error and equipment overload. Simulation is alsoone of the most effective ways to teach power engineering toundergraduate and graduate students.

Currently, power systems in different parts of the worldare undergoing revolutionary changes in both architecture and

A. Chakrabortty is with the School of Electrical and Computer Engineering,North Carolina State University, Raleigh, NC, 27606, USA. A. Bose is withthe School of Electrical and Computer Engineering, Washington State Univer-sity, Pullman, WA, 99164, USA. E-mails: [email protected], [email protected]

operating principles [4]-[6]. The modern grid requires a tightintegration of physical assets with cyber information to allowefficient transportation of data and control signals, and also todevelop an open and real-time market. The envisioned digitalgrid of tomorrow must facilitate the open competition and in-novation needed to accelerate the adoption of renewables whilesatisfying various challenges related to grid stability, markets,and dynamics. It must also allow for active participation ofcustomers in day-to-day grid operations through data-drivenprograms such as demand response. None of these changes,however, can be implemented on the grid unless we have areliable simulation capability. Simulations will be essential tounderstand all complex interactions between cyber, physical,and social layers of tomorrow’s grid.

Motivated by this challenge, in this paper we review a listof methods that are currently used as well as new methodsthat will potentially be used for simulating various planning,operation, stability, reliability, and economic models of powersystems. We base our presentation on simulations of bothphysical models and heuristic data-based models. Physicalmodels describe the behavior of physical components in thegrid such as synchronous generators, induction motors, trans-mission lines, control equipment, loads, transformers, relaysand circuit breakers. Heuristic models, on the other hand, areemployed to describe uncertainties as well as the behaviorof many other engineered and human-driven infrastructuresunderlying the grid [7], [8]. A list of simulation software ispresented. New models arising from non-conventional genera-tion sources and loads, with an emphasis on power electronics,are presented. The need for larger-scale simulations in order totransition from local monitoring to system-wide monitoring ishighlighted, together with a discussion on how new methodsof measurement-based model aggregation can help these wide-area simulations. The increasingly important roles of co-simulation, faster-than real-time simulations, predictive sim-ulations, and the associated challenges in communication andcomputing on daily operation of the grid are also presented.

The rest of the paper is organized as follows. Section IIreviews fundamental models for power system simulations.Section III summarizes the current approaches for simulation.Section IV presents the challenges in computational complex-ity, while Section V describes the impact of power electronicsin conventional simulation models. Section VI describes theneed for co-simulation of transmission and distribution systemmodels. Section VII describes co-simulation of power gridwith other critical infrastructures. Section VIII presents cyber-physical modeling challenges, while Section IX covers faster-than real-time simulations. Section X discusses the future ofremote simulation testbeds. Section X concludes the paper.


II. SIMULATION MODELS AND APPLICATIONS

We start our discussion with a comprehensive list of appli-cations that power engineers need to simulate for everydayoperations of the grid. Many of these applications lead tooptimization problems whose solutions decide the setpointsfor decision variables that are needed to operate the grid ina stable, efficient, economic, and reliable way. These opti-mizations need to be executed over large number of variableswhile following different constraints arising from the physicalbehavior of the grid, thereby necessitating simulations. Thefollowing list captures the most important applications, but,of course, is not complete. We classify these applications withrespect to their ‘time step’ of simulation as follows.

1) Steady-state simulations:• Power flow• State estimation• Steady-state security assessment• N-1, N-1-1 contingency analysis• Fault analysis, fault location analysis• Volt/VAr optimization• Symmetrical components and unbalanced faults• Induction motor analysis• Device coordination and selectivity• Reliability assessment• AMR/AMI integration• Power system markets

2) Long-term planning (in years)• Production costing• Model validation of generators• Reliability evaluation

3) Scheduling (in hours)• Economic dispatch• Optimal power flow (OPF)• Unit commitment• Hydro-thermal Coordination• Preventive Security Constrained OPF• Weather modeling and predictions

4) Slow dynamic simulations (in minutes)• Voltage stability• Loadability limit calculations

5) Electro-mechanical simulations (in seconds)• Automatic generation control (AGC)• Operator training simulator (OTS)

6) Fast electro-mechanical simulations (in milliseconds)• Modal analysis of oscillations• Oscillation damping control• PSS, FACTS controller tuning• Transient Stability• Synchronous machine transient analysis, includingharmonics and unbalanced short-circuit faults

7) Electro-magnetic simulations (in microseconds)• Electro-magnetic transient program (EMTP)• Geomagnetic Induced Currents (GIC) calculations• Arc flash hazard analysis

Each of these applications come with their own first-principle physics-based models that enable their simulations.These models are mostly nonlinear, and described by a large

set of algebraic and differential equations whose evolution varyover different time-scales as noted in the list itself. Most ofthese models exhibit complex nonlinearities such as saturation,deadband, if-then logic rules, and non-smooth behavior [9]-[11]. We start our discussion by recalling a selected setof models that represent the various fundamental buildingblocks of a power system. These models are often used forrunning simulations on academic examples of a grid, mostly atgeneration and transmission levels. Depending on the accuracyand resolution of the simulations that one may be interestedin, these models by themselves can quickly become quitecomplicated. To avoid this difficulty, we limit this section toonly the most basic mathematical equations that capture themain functionalities of the system, and point the reader toappropriate references for more details.

A. Synchronous Generator Models

Consider a power system network with n synchronousgenerators. Synchronous generators are modeled by a combi-nation of Gauss’ law, Newton’s second law of angular motion,Kirchoff’s law, Ohm’s law, and Faraday’s law of electro-magnetic induction. A suite of such models with varying levelsof complexity can be found in [11]. A convenient modelthat is often used for transient stability analysis, small-signaloscillation analysis, as well as for design and simulation ofgenerator control systems such as Automatic Voltage Regula-tors (AVR) and Power System Stabilizers (PSS) is the so-calledflux-decay model. This model assumes that the time constantsof the d- and q-axis flux are fast enough to neglect theirdynamics, that the rotor frequency is around the normalizedconstant synchronous speed, and that the amortisseur effectsare negligible. The dynamic circuit of this model is shown inFig. 1. Using the phasor variables used in this figure, the modelis described by the set of differential-algebraic equations [12].

÷ø

öçè

æ-

+- 2' ]')[(

pd ij

qiqidiqi ejEIxx

dijx'

sir÷ø

öçè

æ-

+ 2)(

pd ij

qidi ejII

ii

j

i

j

qidi eVejVVq

pd

=+÷ø

öçè

æ-2)(

+

-

Fig. 1. Dynamic circuit model of a synchronous generator

1) Differential Equations:

δi = ωi − ωs (1)Miωi = Pmi − E′qiIqi − (xqi − x′di)IdiIqi

−Di(ωi − ωs) (2)T ′doi Eqi = −E′qi − (xdi − x′di)Idi + Efdi (3)

TAi Efdi = −Efdi +KAi(Vref,i − Vi) (4)

for i = 1, . . . , n.2) Stator Algebraic Equations: Applying Ohm’s law to

the stator circuit shown in Fig. 1, and after a few


calculations one obtains the following pair of algebraicequations:

Vi sin(δi − θi) + rsiIdi − xqiIqi = 0 (5)E′qi − Vi sin(δi − θi)− rsiIqi − x′diIdi = 0, (6)

for i = 1, . . . , n.3) Network Equations: These equations couple the active

and reactive power produced by the generator to thepower flowing through the transmission lines, and thepower consumed by the loads. They will be providedshortly in section IIC where we discuss transmissionline models.

Here, i is the generator index, i = 1, . . . , n. Equations(1)-(2) are referred to as the swing equations, (3)-(4) as theexcitation equations, and (5)-(6) as the stator current balanceequations. As shown in Fig. 1, the terminal voltage at theith generator bus is denoted as Vi = Vi∠θi where Vi is themagnitude (volts) and θi is the phase (radians). The symbolsδi, ωi, E′qi, Efdi respectively denote the generator phase angle(radians), rotor velocity (rad/sec), the quadrature-axis internalemf (volts), and field excitation voltage (volts); ωs is thesynchronous frequency, which is equal to 120π radian/secfor a 60-Hz power system such as the North American grid.Mi is the generator inertia (seconds), Di is the generatordamping, Pmi is the mechanical power input from the ith

turbine (Megawatt). T ′doi and TAi are the excitation timeconstants (seconds); xdi, x′di, and xqi are the direct-axis salientreactance, direct-axis transient reactance, and quadrature-axissalient reactance (all in ohms), respectively. Vref,i is thesetpoint value for the terminal bus voltage (volts) of thegenerator, and KAi is a proportional controller that regulatesthe voltage Vi to this setpoint in steady-state. This controlleris referred to as a Automatic Voltage Regulator (AVR). Ad-ditional control signals can also be added to the RHS of (4)through the field voltage Efdi. These controllers are referredto as Power System Stabilizers (PSS), which takes feedbackfrom the generator speed, and passes it through a lead-lagcontroller for producing damping effects on the oscillations inthe power flow after disturbances. More detailed descriptionof the various parameters of this model can be found in [12],[5]. Besides the swing and excitation equations, the generatorcircuit model can have many additional state variables arisingfrom AC and DC excitation, flux linkage, subtransient effects,washout filters, governor equations, automatic generator con-trollers (AGC), connections to other control devices such asFACTS and HVDC, and so on.

B. Models of Renewable Energy Sources

The modern day grid is seeing significant penetration ofwind and solar power, both of which come with interfacingpower electronic converters. Fig. 2 shows a schematic diagramon how these non-conventional distributed energy resources(DERs) are typically integrated with the main grid. We nextsummarize the state-space simulation models of these DERsto understand how their dynamic signal flows couple with thegrid variables.

AC/DC

Internal Controller

Wind Turbine DFIG

Wind Power Plant

Battery

Storage

Wind Bus

duty cycle

duty cycle

PI

DC/DC PI

Fig. 2. Schematic diagram of grid model with DERs

1) Wind Generator Models: A wind power system com-prises of wind turbines, their associated control systems suchas pitch and yaw controllers, doubly-fed induction generators(DFIG), and their associated internal voltage and currentregulators. The turbine can be modeled by a single-mass ordouble-mass unit with gearboxes that couple its shaft withthe rotating shaft of the DFIG. The turbine is driven by anaerodynamic torque, which is typically a cubic function of thewind velocity. The DFIG is modeled through the dynamics ofits stator and rotor currents (Amperes), expressed in a rotatingd-q reference frame. A linear model for the DFIG is oftenwritten as [13]

{i = A(ωg)i+Rv1 +Bv2

Ew = γ (v1q iqs + jv1d ids), i :=

idriqridsiqs

. (7)

The linearization is generally done at the operating pointthat corresponds to the wind velocity that produces maximumwind power output in steady-state. A control scheme calledmaximum power-point tracking control (MPPT) is used tocompute the setpoints for the DFIG currents correspondingto this maximum power. The variable ωg (rad/s) is the rotorangular velocity whose dynamics are decided by the turbinemechanics [13], and skipped here for brevity. The states idrand iqr are the d- and q-axis rotor currents (Amps), ids and iqsare the d- and q-axis stator currents (Amps), v1 = [v1d v1q]

T isa column vector of the d- and q-axis stator voltage magnitudes(volts) at the bus where the wind power plant is connected tothe grid, v2 = [v2d v2q]

T represents the d- and q-axis rotorvoltages (volts), and Ew = Pw+jQw is the total effective windpower injected into the grid. Explicit mathematical expressionsof the matrices A, R and B can be found in [14]. Thepositive constant γ is the number of wind turbines in the plant,which when multiplied to the power output of each generatorprovides the total power output of the wind farm. Typically,both the d- and q-axis currents of the rotor need to be regulatedto pre-computed setpoints based on MPPT. This is usuallyachieved by simple PI control.


The ability to control the mechanical torque of a windturbine applied to the generator using pitch control, and elec-tromagnetic torque using rotor current control enables a windpower system to avoid any mismatch between mechanicaltorque and electromagnetic torque, and therefore, to avoidany rotor deceleration under network frequency decline. Sinceinertia of a rotating machine is equivalent to the time constantof its frequency dynamics, this property of a wind powersystem is equivalent to having zero rotor inertia [16]. Thus,the wind power model (7) is often referred to as a low inertiamodel. The same holds for a solar cell, whose model doesnot involve any rotational dynamics [15]. Simulations of gridmodels integrated with these DERs thus often show verysharp declines in the grid frequency due to this low inertialeffect. Several examples can be found in [17], [18]. Impact ofthis effect on electro-mechanical oscillations were shown viasimulations in [19].

2) Converter Models: Three-phase pulse-width-modulated(PWM) converters are used for integrating DERs with the grid.Assuming a direct-quadrature synchronous reference frame,the converter model can be written as [20] Lid = ωsLiq − ρvdc

Liq = −ωsLid + eq − ρvdcidc = eqiq/vdc

(8)

where id and iq are the d- and q-axis currents (Amps) flowinginto the converter, eq is the q-axis voltage (volts) at the busconnecting to this converter, vdc and idc are the converted DCvoltage (volts) and current (Amps), ρ is the duty cycle, and L isthe filter-inductance (milli-Henry), and ωs is the synchronousfrequency (rad/s). Generally, a PI controller is used to regulatethe output DC current of the converter to a desired setpoint.

3) Storage models: Today’s power grid is also experiencingpenetration of electrical battery storage at various scales.Typical models of batteries involve electro-chemical equationsthat reflect their internal physical principles, coupled withpower flow balance that connect them to the grid. Manyvariants of battery models with different levels of complexityexist in the literature [21]. A simple model can be consideredas [22]

q = idc/C, vdc = g(q) (9)

where q is the state of charge, C is the capacity of thebattery (microFarad), and g(q) is a monotonically increasingfunction such that g(q) > 0 if q > 0, following from theelectrochemistry of the battery cell.

C. Transmission Line Models

The state variables of the DERs, converters, and storagemodels are coupled to those of the main grid via power balanceacross the transmission lines in the grid. A transmission lineis modeled using series resistance, series inductance, shuntcapacitance, and shunt conductance. A line is defined as ashort-length line if its length is less than 50 miles. In thiscase, the capacitive effect is negligible, and only the resistanceand inductive reactance are considered. Assuming balancedconditions, the line can be modeled by an equivalent circuit ofa single phase with a resistance. If the line is between 50 miles

and 150 miles long, it is considered to have a medium length.The single-phase equivalent circuit is modeled as a π or Tcircuit. The shunt capacitance is divided into two equal parts,each placed at the sending and receiving ends of the line. If theline is more than 150 miles long, then it is classified as a long-length line, whose model, unlike the lumped parameter modelsof the above two types, must consider parameters uniformlydistributed along the line. Those models are described bypartial differential equations. For a medium-length line, thepower balance equations at any bus i can be written as [11]m∑k=1

ViVk(Gik cos(θik) +Bik sin(θik))︸︷︷︸Pi

−PGi + PLi = 0

m∑k=1

ViVk(Gik sin(θik)−Bik cos(θik))︸︷︷︸Qi

−QGi +QLi = 0

where m is the total number of buses in the network, Vi and θiare the voltage (volt) and phase angle (radian) of the ith bus,θik = θi − θk, PGi and PLi are the active power (Megawatt)generated and demanded at bus i, QGi and QLi are the reactivepower (MegaVAr) generated and demanded at bus i, and Gikand Bik are the conductance and susceptance (mho) of theline joining buses i and k. For example, for the flux-decaymodel (1)-(6), the network equations linking the generatorstate variables and the transmission line flows can be writtenaccordingly as

0 = IdiVi sin(δi − θi) + IqiVi cos(δi − θi) + PLi

−m∑k=1

ViVk(Gik cos(θik) +Bik sin(θik) (10)

0 = IdiVi cos(δi − θi)− IqiVi sin(δi − θi) +QLi

−m∑k=1

ViVk(Gik cos(θik)−Bik sin(θik)). (11)

for i = 1, . . . , n. Since one may write Gik = Yik cos(αik) andBik = Yik sin(αik), where Yik and αik are the magnitude andphase of the element (i, k) in the admittance matrix, a morepopular representation of (10)-(11) is often written as [12]

0 = IdiVi sin(δi − θi) + IqiVi cos(δi − θi) + PLi

−m∑k=1

ViVkYik cos(θik − αik) (12)

0 = IdiVi cos(δi − θi)− IqiVi sin(δi − θi) +QLi

−m∑k=1

ViVkYik sin(θik − αik). (13)

For non-generator buses, the network equations reduce to

PLi −m∑k=1

ViVkYik cos(θik − αik) = 0 (14)

QLi −m∑k=1

ViVkYik sin(θik − αik) = 0. (15)

If any bus i has a wind power generator, or a solar powergenerator, or battery storage connected to it, then the active


and reactive power generated (or consumed) by them will beadded on the LHS of (14) and (15), respectively. Transmissionline models also include transformers, circuit breakers, phaseshifters, and shunt and series compensators.

D. Load Models

The active and reactive power drawn by the jth load canbe modeled as [11]

PLj = ajV2j + bjVj + cj (16)

QLj = ejV2j + fjVj + gj (17)

where, (aj , bj , cj) and (ej , fj , gj) are load-specific constantcoefficients of appropriate dimension, Vj is the voltage mag-nitude at the load bus j, and the three terms in each equationexplicitly represent the contribution of each type of load,namely - constant impedance, constant current, and constantpower load, respectively. Although constant impedance loadsgreatly simplify the derivation of the final state-space modelof the entire power system network in an explicit closed-form, retaining this explicit form is not always possible forthe two other types of loads. Thus engineers have to resort tosoftware programs to numerically simulate the implicit modelequations, linearize them over a given equilibrium point, andcompute a small-signal state-space model. More complicatedload models such as frequency-dependent loads have beendeveloped by the IEEE task force in [23]. Several papers suchas [24], [25] have also considered dynamic loads of the form

TpdP

dt+ P = Ps(V ) + kp(V )

dV

dt(18)

where, Tp is the load time constant, Ps(·) is called the staticload function, which is applicable in steady-state, and kp iscalled the dynamic load function. In today’s grid operationsespecially, loads play an extremely important role by virtue ofcustomer-side applications such as demand response. Recentpapers such as [26], for example, have developed aggregatedmodels for a population of air conditioning loads including sta-tistical information of the load population, load heterogeneity,and their transient dynamics. Various other types of static anddynamic load models such as building loads and industrialmachine loads can be found in [27], [28].

E. Models of Communication Systems

Power system operations today are becoming increasinglydependent on information and communication technology(ICT), thereby necessitating accurate modeling of communi-cation networks. Communication is needed for mainly threeclasses of applications - namely, wide-area networks (WAN)used for wide-area control of transmission systems, local-area networks (LAN) used for smart metering, and homearea networks (HAN) used for control of smart homes.Coming up with just one single representative mathematicalmodel, however, is impossible, if not impractical. Researchers,therefore, often tend to couple actual communication systemswith offline physical models of the grid, and emulate theinteraction between the two in real-time, rather than usingtheoretical models. Examples include the Internet (or exper-imental variants of it such as Internet2) for long-distance

data communication such as in wide-area monitoring andcontrol of transmission networks, and wireless platforms suchas Zigbee for short-range data communication such as inpower distribution systems in university campuses and homeautomation. One important deciding factor behind the choiceof these models is the communication delay that a messagemay experience while something critical is happening in thegrid. For example, recent papers such as [29], [30] have useddelay models arising in Internet-based wide-area communi-cation networks for testing the efficacy of just-in-time wide-area control actions. Typically, these delays consist of threecomponents:

1) the minimum deterministic delay, say denoted by m,2) the Internet traffic delay with Probability Density Func-

tion (PDF), say denoted by φ1, and3) the router processing delay with PDF, say denoted by

φ2.The PDF of the total delay at any time t is written in termsof these three components as

φ(t) = p φ2(t)+(1−p)∫ t

0

φ2(u)φ1(t−u)du, t ≥ 0. (19)

Here, p is the probability of the open-period of the path withno Internet traffic. The router processing delay is approximatedby a Gaussian density function

φ2(t) =1

σ√

2πe−

(t−µ)2

2σ2 , (20)

where µ > m. The Internet traffic delay is modeled by analternating renewal process with exponential closure periodwhen the Internet traffic is on. The PDF of this delay is givenby

φ1(t) = λe−λt, (21)

where λ−1 models the mean length of the closure period. Thecumulative distribution function (CDF) of this delay modelcan then be derived as

P (t) =

∫ t

−∞φ(s)ds =

1

2[erf(

µ√2σ

) + erf(t− µ√

2σ)]+

(p− 1)

2e(

12λ

2σ2+µλ)e−λt[erf(λσ2 + µ√

2σ) + erf(

t− λσ2 − µ√2σ

)].

(22)

Random numbers arising from this CDF were used in [29],[30] for simulating delays in a distributed communicationnetwork for wide-area monitoring and control [31], [32]. Moredetails of these distributed networks will be presented inSection III when we describe the NASPInet.

The majority of the state-of-the-art communication models,however, are used for smaller-scale simulations that involveinformation exchange over a single channel. The challenge isto translate these single-path models to multi-path, multi-hop,shared network models where background traffic due to otherparallelly running applications over shared resources may poseserious limitations in latencies. Recent references such as[33], [34] have provided interesting theoretical tools such asMarkov jump process, Poisson process, multi-fractal models,and Gaussian fractional sum-difference models for modeling


delay, packet loss, queuing, routing, load balancing, and trafficpatterns in such multi-channel communication networks.

III. CURRENT METHODS OF SIMULATIONS

We next discuss how simulations are currently handledby power system engineers and dispatchers. The discussionincludes examples of several common software packages usedfor simulations in today’s grid, and also hardware-in-looptype simulations using real-time digital simulators (RTDS)and Opal-RT simulation facilities [35], [36]. List of currentlyused open-source software include UWPFLOW, TEFTS, Mat-Power, PST, InterPSS, GridLAB-D, OpenETran, OpenPMU,rapid61850, etc., while commercial software includes Pow-erWorld [37], PSS/E, PSLF, Aspen, E-Tap, PSCAD, Plecs,Open-DSS, Sim-Power Systems, and so on. Educationalsoftware programs include PSAT, which is based on Mat-lab/Simulink, and DOME, based on Python and public-domainC, C++, and Fortran libraries [38]. An excellent survey onopen-source software can be found in [39]. The primaryapplications that these software are used for are as follows.

A. Power Flow Analysis

The power flow equations (12)-(15) can be expressed in acompact form as

f(x) = 0 (23)

where

f =

P2(x)− PG2 + PD2

P3(x)− PG3 + PD3

...Pm(x)− PGm + PDmQ2(x)−QG2 +QD2

Q3(x)−QG3 +QD3

...Qm(x)−QGm +QDm

, (24)

x = col(V1, V2, . . . , Vm, θ1, θ2, . . . , θm), PGi and PLi arethe active power generated and demanded at bus i, QGi andQLi are the reactive power generated and demanded at busi, i = 1, . . . ,m. In (23) we consider bus 1 to be the slackbus, and hence it’s power balance equation is not includedin f(·). Solution of these nonlinear equations is referred toas power flow. The majority of software packages these dayscomes with their own sets of power flow solvers includingaccelerated Gauss-Seidel, Newton-Raphson, and perturbation-theory iterative techniques. Fast Decoupled and DC techniquesare also often used [9]. The advantage of Newton-Raphsonmethod is that it guarantees fast convergence as long as initialguess is close to the actual solution, and also that it enhancesthe region of convergence. The difficulty, however, is comput-ing and inverting the Jacobian matrix J(xk) at every iteration.Factorizing a full matrix is an order n3 operation, and henceextremely expensive. The order can be decreased, however, byexploiting the sparse structure of Y , which induces sparsity inJ as well. It has been shown through examples in [9] that byusing sparse matrix methods results in a computational orderof approximately n1.5, which can be a substantial saving whensolving systems with tens of thousands of buses.

Power flow solution for distribution system models is donequite differently than in transmission models. Transmissionsystems have meshed networks, while distribution systemsare mostly comprised of radial circuits. Software such asDistribution System Simulator (DSS) these days, however,can solve meshed distribution systems as well because theyare intended to be used for distribution companies that mayalso have transmission or sub-transmission systems [40]. Thecircuit model employed can be either a full multiphase modelor a simplified positive-sequence model. The power flowexecutes in numerous solution modes including the standardsingle snapshot mode, daily mode, duty-cycle mode, MonteCarlo mode, and several other modes where the load varies asa function of time. For planning purposes the simulation timeperiod is chosen as a 24-hour day, a month, or a year.

B. Dynamic Simulations

Transient stability simulations for large power system mod-els are based on the differential-algebraic (DAE) model (1)-(5) for synchronous generators, and the algebraic power flowequations (12)-(15). The DAE can be written as

dx

dy= f(x, y, p), x ∈ Rn, y ∈ Rm, p ∈ Rq (25)

0 = g(x, y, p). (26)

The state vector x corresponds to the internal dynamic statesof generators, exciters, governors, PSS, dynamic loads, powerelectronic devices, and so on. The output vector y containsthe network power-flow variables e.g., bus voltage magnitudesand angles. The vector p describe machine parameters suchas time-constants, inertia and damping, transmission line pa-rameters such as line reactances and network topology, aswell as operational parameters such as the status of shuntcapacitor banks. The current trend in general-purpose DAEsimulation, for example in Powerworld, PSCAD, and PSAT,follows hierarchical system modeling, which in addition tosimplifying the conceptual and model-management tasks, alsoprovides a good basis for incremental testing and validationof models and simulations. Traditional power system literaturelists two broad modeling approaches for simulations - namely,explicit modeling and implicit modeling. Explicit models fordynamical systems use ordinary-differential equations (ODE)as the primary mathematical abstraction. The word ‘explicit’here refers to causality, i.e., equations representing the physicsof the system are written so that known quantities suchas inputs and state variables are used to define unknownquantities. Languages associated with explicit modeling areblock-oriented, such as Simulink and Ptolemy II [41]. Implicitmodels, on the other hand, directly use DAE as the primarymathematical abstraction. DAE based models emphasize therelationships between elements without regard to a particularcausal ordering, and hence the name implicit. Example of suchlanguages include Modelica [42], Hydra [43], and Sol [44].

C. Economic Dispatch and Optimal Power Flow

Linear and Quadratic programming techniques are the mostcommon tools for both of these critical applications. Power-world, for example, uses ‘Quadprog’ and ‘Rglpk’ packages


for optimal power flow (OPF), both of which are basedon quadratic programming. A sparse solver for OPF usingmoment-based convex relaxation has been proposed in [45].The classical OPF problem aims to minimize the function [9]

J =∑i∈G

(ci1P2Gi + ci2PGi + ci0), (27)

which is the sum of the quadratic costs accounting for thecost of active power generation at every generator bus (theset of all generator buses being denoted as G) over the busvoltages, active and reactive power generations, subject todifferent system-level constraints. In general, this problemis non-convex, and difficult to solve. Recent results in [46],[47] have derived conditions on power system models thatguarantee zero duality-gap so that semidefinite programming(SDP) can be used to retrieve the global optimum solution tothe OPF problem. Sufficient conditions for global optimalityhave been presented in [48]. The results have been extendedto meshed transmission network topologies in [49].

D. Fault Analysis

Fault simulation packages simulate single-phase and 3-phase bolted faults for interconnected power system models,and make provision for complex fault impedance. The Y -bus building algorithm is commonly implemented to constructthe admittance matrix [50], followed by sparse matrix inversecomputations. Several fast algorithms exist for this - for ex-ample, direct methods via LU factorization, Recursive Green’sFunction (RGF) algorithm, and more advanced newer andfaster methods such as Fast Inverse using Nested Dissection(FIND). A list of simulation tools for comprehensive faultanalysis can be found in [51].

E. Load Frequency Control

Load variations in a power system cause drifts in frequencyand voltage from their nominal values, resulting in lossof generation due to tripping of transmission lines. Thesedrifts can be minimized, and kept within tolerable limits byload frequency control (LFC) [9], [11]. In LFC simulationsusers can change loads in selected areas in the grid, andproportional-integral (PI) control is automatically initiatedbased on the speed droop settings of the areas. The areas sharethe load by changing their generation levels based on theirpreset percentage contributions. With increasing penetrationof renewable power LFC will be required to compensatefor not only short-term fluctuations in frequency but alsolong-term fluctuations that are conventionally dealt with byeconomic dispatch. Recent papers such as [52] have proposedthe use of storage as a solution to this problem. EnhancingLFC algorithms to compensate for uncertainties introduced byintermittent renewables is currently an ongoing area of activeresearch.

F. Operator Training Simulator

The simulation engine for the Operator Training Simulator(OTS) has to simulate the behavior of the power system in realtime so that the operator trainee gets the feel of a real powersystem [53], [54]. As mentioned before, such a simulation

consists of solving differential equations (the dynamic part)as well as the algebraic equations (the steady-state part). Tosolve this in real time the dynamic part must be faster thanreal time so that the steady-state part has enough time forits solution. Since the operator usually observes the powersystem through SCADA, which updates every few seconds, atime step of one second for the simulation provides enoughaccuracy. Today, however, with the increasing availability ofphasor measurements which update at 30 Hz or more, theOTS simulation needs to be done at much smaller time steps.New OTS implementations are appearing that use the transientstability formulation with around 10 ms time steps to be ableto generate phasor measurements in real-time.G. Reliability Modeling and Simulation

Reliability modeling refers to the failure and repair cycleof power system components. These are typically modeled byalternating renewal processes. One common method is the se-quential Monte-Carlo simulation with stochastic point processmodeling [55]. The application of probability models to theevaluation of generation reliability allows for the integrationof different unit sizes and types, the effects of maintenance,the capacity of interconnections and other factors such aseconomic aspects. The commonly employed metrics for thispurpose are called “loss of load probability” (LOLP), “lossof energy probability” (LOEP), and “frequency and duration(FAD). For detailed definitions of these metrics please see[56].H. Simulation of Power Markets

Electricity markets are one of the major drivers and enablersfor power system operation [57], [58]. Many commercial soft-ware such as Powerworld, GridLAB-D, and Power SystemsOptimizer (PSO) with mixed-integer programming solverssuch as GUROBI currently exist for simulating power markets.Several research institutions have also recently developed effi-cient market simulators. Examples include the AMES (Agent-based Modeling of Electricity Systems) wholesale powermarket testbed at Iowa State University, JASA (Java AuctionSimulator API) developed by Ripple Software Ltd., a Python-based software called MiniPower developed in University ofWashington, and several other market simulators created byArgonne National Lab. Detailed descriptions of these packagescan be found at [59]. The main applications of a power marketsimulator include valuation of assets including both physicaland financial contracts, transmission planning, policy analysisand market design, market surveillance, generation schedul-ing and trading support, risk analysis, modeling of demandresponse participation in markets and ancillary services, andreliability assessments [60]. The key components of a marketsimulator are as follows:

1) Inputs: demand forecasts, generation and transmissionexpansion, fuel prices, emission allowance prices

2) Models: loads, demand response, generation and trans-mission models including storage and variable genera-tions, market rules

3) Algorithms: maintenance scheduling, contingency anal-ysis, co-optimization of energy and ancillary services,topology control, mixed-integer programming


4) Physical outputs: generation and reserve schedules,power flow, fuel usage, emissions, load curtailments

5) Financial outputs: electricity prices, revenues, costs.In a typical power market simulator the user represents

power generation companies, submitting hourly generationbids into a virtual market. Simulated agents submit these bids.Bids are submitted for 24-hour periods (day-ahead market) andentered into the model. The total of all energy bids submittedby a player may not exceed the operational capability ofon-line generating units that she owns. Bid block sizes aredetermined by the player as part of her bidding strategy. Bidprices for each unit must increase as a function of blocknumber. The software is then run to determine market clearingprices, generation levels for all units, and profits and losses foreach market participant. Before starting the simulation, eachparticipant is provided with a set of “market rules” that haveto be followed by each player. The objective of the simulationis for each player to maximize her profit by selling electricityinto the market. The player with the highest profit wins thegame. If no one makes money, the player with the lowestloss is declared the winner. Net profit is calculated based onenergy sales in each hour, real-time market clearing prices,incremental production costs for each block of each generationunit, energy production of each block, operating costs, andfixed capital charges. Appropriate test systems such as theeight-zone ISO-New England system has also been developedto showcase the efficiency of these simulation methods [59].Multi-objective optimization is applied to ensure the securityof the market operations [61].

I. Simulation of Smart Grid Communications

Incorporating communication models in smart grid simula-tions is becoming imperative in the current state of art. Real-time digital simulators (RTDS), for example, can seamlesslyemploy communication protocols such as IEC 61850 forsubstation automation applications, DNP3 and IEC 60870-5-104 for SCADA system applications, and IEEE C37.118for Synchrophasor applications. Comprehensive surveys ofvarious such communication protocols for smart grids canbe found in [62]-[63]. In this section we highlight threeof these communication models. The first one is used foremulating wide-area communication of Synchrophasor data intransmission-level systems, the second one for smart meteringin distribution-level systems, and the third for home automa-tion.

1) NASPI-net: Wide-area monitoring and control of trans-mission grids require communication of large volumes ofSynchrophasor measurements from substations spread acrosshundreds of miles. The time-scale associated with taking thesecontrol actions can be in fractions of seconds. Therefore,controlling latencies and data quality, and maintaining highreliability of communication are extremely important for theseapplications. The media used for WAN communications typi-cally use longer-range, high-power radios or Ethernet IP-basedsolutions. Common options include microwave and 900 MHzradio solutions, as well as T1 lines, digital subscriber lines(DSL), broadband connections, fiber networks, and Ethernetradio. The North American Synchrophasor Initiative (NASPI)

[?], which is a collaborative effort between the U.S. Depart-ment of Energy (DOE), the North American Electric Reliabil-ity Corporation (NERC), and various electric utilities, vendors,consultants, federal and private researchers and academics iscurrently developing an industrial grade, secure, standardized,distributed, and scalable data communications infrastructurecalled NASPI-net to support Synchrophasor applications inNorth America. NASPI-net is based on an IP Multicastsubscription-based model. In this model the network routingelements are responsible for handling the subscription requestsfrom potential PMU data receivers as well as the actualoptimal path computation, optimization, and re-computationand rerouting when network failures happen. An excellentsurvey of NASPInet can be found in [64].

2) AMI Communication: Advanced metering infrastruc-ture (AMI) communications is a central topic for distributiongrids today. AMI communication is typically executed via lo-cal area networks (LAN) that provide two-way communicationpaths between the consumer’s and the utility’s smart meters.The two predominant technologies involved are power linecarrier (PLC) signals transmitted over the utility’s distributionlines, and radio frequency (RF). PLCs use frequencies andmodulation techniques that translate into a range of availablespeed and bandwidth options, including the 60 Hz power waveas well as superimposing high-bandwidth carrier signals ofother frequencies onto the power line. They are more cost-effective in low-density areas such as rural and suburbanservice territories where feeders serving the smart meters tendto be lengthy. Similarly, RF solutions include technologies thatuse an assortment of frequencies, covering a range of distancesover a variety of network topologies. These types of LANcommunications can also be used for demand response, load-control, and distribution automation [65].

3) Home Area Networks: Home area networks (HAN)are used for communication between devices behind a smartmeter inside a consumer’s home. This includes messages topresent information via in-home displays, as well as messagesthat demand response devices can use for controlling keyappliances to manage energy usage and capacity constraints.HAN is being conceptualized in response to mandates acrossthe nation to improve energy efficiency, and educate andempower consumers. ZigBee is a common protocol that is usedfor HAN. ZigBee messages are transported via RF messagingprotocols in unlicensed RF spectrum shared by Blacktooth andWi-Fi for communication between devices inside a house [62].

Despite the wide usage of these models, a long list ofchallenges still exist for modeling and simulation of today’srapidly evolving grid. These challenges need to be surpassedover the next decade through research and development tocreate robust, reliable, and cheap simulation platforms. Thefollowing sections list five of those primary challenges.

IV. CHALLENGE 1: ADVANCED MODELING METHODS

With millions of new devices penetrating the grid withevery passing day, simulation models for tomorrow’s gridare bound to become much more complex than just a setof DAEs as in (25)-(26). All of these new devices, thatcome with their own share of nonlinearities, will impact


the dynamics, stability, and most importantly the secondsto tens-of-seconds-time-scale performance of the grid whencritical disturbances hit. Developing simulation models thatcan capture such impacts in a scalable way is a seriouschallenge. Heat flow models in buildings, for example, areextremely high-dimensional, complex, and nonlinear, oftenmodeled by partial differential equation models, for whichstrong knowledge of fluid dynamics is required in developingreliable simulations. The increasing use of modern powerelectronic devices in the distribution grid, for example, ismaking the grid models nonlinear, and often non-smooth dueto switching actions. A study recently reported in [66], in fact,shows that power electronic converters can pose critical limitson the line currents and voltages beyond which the systemexperiences a Hopf bifurcation leading to sudden vanishing offeasible equilibria. Taking all of these factors into account, aset of relevant questions that naturally arise are as follows.

1) With significant integration of wind, solar, storage, andelectric vehicles, how can we develop holistic modelsthat capture the dynamic impact of all of these newenergy sources and sinks on the conventional grid? Howcan we identify important coupling parameters that canreveal the possibility of weak operating points that maylead to transient instability and voltage collapse?

2) How can we develop robust numerical methods that willenable us to simulate these models in a scalable andmodular fashion so that if tomorrow one more wind plantor solar plant or load is installed in the system then itwould not take an enormous amount of computationaleffort to re-simulate the new model.

3) Most importantly, as highlighted in sections II-E andIII-I, with many of the modern-day grid applicationsbecoming increasingly dependent on communication ofinformation across large geographical spans of the grid,how can we accurately simulate the cyber-physical in-teraction between the physical model of a grid and themodels of its computation and communication layers?The behavior of a power system is driven by physicallaws while the behavior of communication networksdepend largely on random stochastic processes, logicfunctions, if-then rules, and human decision-making.Hybrid system simulations will become imperative tocapture the continuous-time state transitions of the phys-ical models, and their coupling with the discrete-eventstate transitions in the cyber layer. Although combiningdiscrete-time and continuous-time behavior is not new,and has been around in the power system literature sincethe inception of digital controls [67], the challenge liesin accurate modeling of stochastic channels, variablerouting delays, protocols for managing queuing delays,and so on, all of which are essential ingredients formodeling a cyber-physical system (CPS).

4) Similarly, how can we model and co-simulate complexinteractions between power system models and modelsof other engineered infrastructures such as transportationand social networks?

The key factor for these types of scientific experimentation

is modeling. To properly answer these questions one requiresa holistic approach to modeling by, for example, developingsimulation models that are not only easier to build, reuse,and validate, but also those that fundamentally characterizethe environment in which they operate. We next list thechallenges that currently stand in way of these advancedmodeling methods.

A. Structural Properties

One of the desired properties of simulation models isstructural dynamicity. By definition, structural dynamicity isthe notion that one is not able to predetermine the numberof structural configurations that a system may take on. Theconcept is not new to power systems. Contingencies, renew-able integration, and interactions between prosumers are fewobvious examples. Upon such events, the structural orientationof the system along with the underlying mathematics thatmodel the physics of the system change in quantity, direction,etc. Another desired property is to capture the hybrid natureof the grid operating as a CPS, consisting of a combinationof continuous-time (physical domain) and discrete-time (cy-ber domain) behaviors. Component interactions is the thirdrequired property. Models that inherently and implicitly cap-ture the interaction between different components are criticalfor holistic simulation approaches, especially when there ismore than one explicit representation for a given component.Another necessary property is component reflectivity [68].Models that are closely reflective of its components helpin promoting modularity, and lessen the burden of derivingcausality when the consequences of the interactions are notknown a-priori.

B. Scalability

One big challenge is to enable a real-time simulation capa-bility for power system models such as (25)-(26) at extremescales. Foundational work on taxonomy theory for modelingand analysis of DAE power models has been done in theory[69], but its translation to simulation models is still missing.Software such as Modelica and Hydra, for example, need tobe exploited for modeling scalability through modularity, com-position, correctness, implicit representations and structuraldynamics. All of these abstractions then need to be broughtunder the umbrella of a common modeling language and thefront end of a compiler, followed by a library and language-level abstractions that support the needs of experimentation.

Scalability is also an issue for controller designs. Conven-tional controllers such as Linear Quadratic Regulators (LQR)involve the computation of large matrix decompositions thatcan result in detrimental numerical inaccuracies without anyguarantee of robustness. They also demand every node inthe network to share its state information with every othernode, resulting in an impractically large number of communi-cation links. Traditionally, control theorists have addressed theproblem of controlling large-dimensional systems by imposingstructure on controllers. The trade-off, however, is that theresulting controllers are often agnostic of the natural cou-pling between the states, especially the coupling between theclosed-loop states, as many of these couplings were forcibly


eliminated to facilitate the design itself. Ideas on aggregatecontrol [70], glocal control [71], and hierarchical control [72]have recently been proposed to address this challenge. Whatdesigners are still lacking, however, is a tractable approachfor constructing controllers that can be simulated and imple-mented at the scale of hundreds of thousand of buses. Modelaggregation has been proposed as one possible solution forthis problem [73], [74].

C. Model Validation

Another significant challenge is the validation of the mod-els used in these simulations. This is particularly true forthe dynamic models of the generators and their associatedcontrols. After the large blackout in the US west coast in1996, it became clear that the generator models used in thestudies were not accurate enough. Since then the WesternElectricity Coordinating Council (WECC) standard requiresupdating of the model parameters through standardized testing.Power systems in other parts of the world where stability isan issue have followed a similar approach. However, off-linetesting is expensive and as PMUs have proliferated over thepast decade, using PMU data during disturbances to updatemodel parameters on-line have become more common [75].Several challenges still stand on the way. For example, newtechnologies such as renewable generation sources and storagethat require power-electronic grid interfaces have completelynew types of dynamic models, as will be highlighted in SectionV. Developing accurate modeling methods and devising pro-cedures to validate these models are currently lagging behind.

D. Model Aggregation

Given the large size and extraordinary complexity of anyrealistic power system, deriving and simulating the dynamicmodel for an entire network becomes extremely challenging.Constructing approximate, aggregated, reduced-order modelsusing simplifying assumptions, therefore, becomes almostimperative in practice. Papers such as [76] have definedaggregation methods for simulation time, for generation units,and for load demand units. The performance of the aggregatedmodels is checked against detailed models including binaryeffects such as minimum down-time, minimum generation ordemand side contracts. This is especially important for controldesigns such as model predictive control, where very largeoptimization problems need to be solved online. The opti-mization in these cases usually has to be simplified by usingapproximated power plant models, aggregating several assetsto single units, and limiting controller foresight. The mainquestion is whether fast generation units, curtailable renew-able generation, demand-side management on the consumerside, and storage capacities are still correctly represented inthe approximate models so that one can guarantee optimaloperation of the grid.

Not just for reducing complexity of simulations, model ag-gregation may also be necessary if one is purposely interestedin simulating only a certain part of the grid. Or, perhaps a cer-tain phenomenon that happens only over a certain time-scale.For example, one may be interested in simulating only the low-frequency inter-area oscillations of a group of synchronous

generators instead of the entire spectrum of their frequencyresponse. Identifying coherent sub-groups among this groupof generators, aggregating the sub-groups into an equivalenthypothetical generator, and analyzing the oscillation patternsof these equivalents become necessary in that situation. Thesedays we often hear power system operators mentioning ‘North-ern Washington’ oscillating against ‘Southern California’ inresponse to various disturbance events. The main questionhere is whether we can analytically construct dynamic modelsfor these conceptual, aggregated generators representing Wash-ington and California, which in reality are some hypotheticalcombinations of thousands of actual generators. One examplefor this motivating problem is the Pacific AC Intertie system inthe US west coast, a five-machine dynamic equivalent mass-spring-damper model for which has been widely used in theliterature [75], [77]. This model is shown in Figure 3. The mainquestion is - how can we construct an explicit dynamic modelfor this conceptual figure, and that too preferably in real-time,using voltage, current or power flow signal measurements inorder to establish a prototype for the inter-area dynamics ofthe entire interconnection?

Recently, several papers such as [78], [79] have addressedthis question, and derived a series of results on PMUmeasurement-based model reduction that combines aggrega-tion theory with system identification. Several open questionsstill exist, however. For example, once a baseline model isconstructed, one must study how it can be updated at regularintervals using newer PMU data. Ideas from adaptive learningand decomposition theory [80] can be useful for that. How thisupdated model can be used to predict the slow frequencies andcorresponding damping factors also needs to be formalizedand validated via realistic simulations. Questions also existon how the reduced-order model can predict the sensitivityof the power flow oscillations inside any area with respectto faults and wind power penetration in any other area. Ifanswered correctly, utilities can exploit this information fromsimulations of the aggregated model, and evaluate their dy-namic coupling with neighboring companies, leading to moreefficient resource planning. Significant amount of work stillneeds to be done in formalizing how different failure scenariosin the actual full-order grid model can be translated to theaggregated model, what kind of advanced signal processingand filtering need to be applied to PMU data for accurateidentification of the aggregated model parameters [81], andhow controllers designed based on the aggregated model canbe mapped back to the original system for implementation.

E. Challenges in Power Market Simulations

Three main challenges facing power market simulatorstoday are

1) the curse of one-scenario-at-a-time, meaning that sim-ulators today are not set up to generate, execute, andpost-process multiple scenarios taken together,

2) lack of parallel processing, meaning that many of the op-timization problems used for market emulation are easilyparallelizable but the software are not well-designed totake advantage of that, and


V

V

jx

V

V

r

jx

E

E

E

E

E

PMU

PMU

PMU

PMU

PMU

Fig. 3. Model Aggregation for simulation of the WECC power system [75].

3) turn-around times are unacceptably long due to lack ofscalable algorithms.

Many simulators today rely on heuristic optimization, whichcan be replaced by more sophisticated non-convex methods forsolving mixed-integer programming problems. For example,as pointed out in [82], simulation times can be significantlyimproved if traditional game-theoretic market models suchas Nash-Cournot models and supply function equilibrium(SFE) models are replaced by faster models such as strategicproduction cost models (SPCM). This is especially useful forsimulating real-time markets. Examples of these cost modelsfor capital investments in transmission networks were recentlyhighlighted in [83]. Market operators are currently in need forsoftware that can handle significant increase in computationalrequirements, workflow management, real-time data process-ing, and data storage. The use of cloud computing by whichusers can create their own virtual hardware environments formanaging a variety of market processes has recently beenproposed in [60] as a solution to this problem. Market emula-tors such as pCloudAnalytics using Amazon Web Services arecurrently being developed. Cloud-based simulators, however,come with their own share of research challenges such as cybersecurity, data privacy, and usage tracking and accounting.

F. Load Forecasting

As pointed out in Section IIIH, accurate load forecasting isa key requirement for power market simulations. Current fore-casting models are based on autoregressive models, Kalmanfilters, nonlinear least squares, and non-parametric regression[84]. But the increasing use of demand-side participation inpower system operations is going to make load forecast-ing much more challenging. We need data-driven functionalmodels that relate the value of load demands with predictorvariables such as setpoints for all the generation resources,setpoints for the duty cycles of power electronic convertersthat connect renewable generators to the grid, weather datasuch as temperature and humidity levels in a hot versus cold

day, amount of PV generation and battery storage units thatmay be owned by the customers, and so on. Recent results inthe machine learning literature provide us with some positivehope of creating accurate models by using nonlinear regressionmethods and reinforcement learning [85]. As pointed outcorrectly in [85], it is not easy for a human operator to judgeif the decision of changing the load setpoint is correct ornot. Operators are more used to handling these types of loadchanges with fixed logic and rules, and prefer models thatare more transparent, where it is clear exactly which factorswere used to make a particular prediction. Deep reinforcementlearning can be a perfect choice for that. This is particularlyimportant for microgrid simulations as their control algorithmsare strongly dependent on the forecasted loads.

G. Smart Loads

One challenge from the perspective of load modeling is theemerging concept of smart loads. This is becoming especiallyimportant for simulation of microgrid models. For example,recently a new technology called “electric springs” (ES) wasproposed in [86] to enable a demand-side control scheme atlow cost, and in a way less intrusive to the consumer comparedto other demand-side response technologies such as energystorage and on-off control of loads. ES is a class of powerelectronic devices, and can be ideally regarded as a current-controlled AC voltage source. Installed in series with othernon-critical loads, ES and the non-critical load together act asa smart load whose power consumption can be boosted andreduced in a plug-and-play fashion, whenever needed. Thissmart load can then easily participate in system regulation.However, a comprehensive microgrid model showing howthese smart loads and their associated control systems willinteract with the various DER models developed in SectionII, for example, is still missing. Some preliminary resultshave been presented in [87], triggering potential research ideasalong this direction.

H. Handling Model Uncertainties

Models are always an approximation. For example, asmentioned in Section IIA, the phasor-based model (1)-(4) forsynchronous generators is well-suited for transient stabilityanalysis, but it ignores electro-magnetic transient phenomena.Even the load model (16)-(17) can become a gross approxi-mation depending on the desired fidelity of a simulation. Itis referred to as a ZIP (constant impedance-current-power)model. It does not capture the delayed voltage recoveryassociated with induction motor loads in distribution systems.Thus, it may be completely invalid if one intends to simulatethose types of loads. This is a daunting challenge for themodern grid, where load models are further complicated bynew protocols such as demand response, direct load control,vehicle-to-grid integration, and various other factors by whichload variations are shaped through customer participation.Using approximate models may not provide a very goodrepresentation of the reality in these scenarios. Modeling andaccounting for uncertainties thus becomes imperative.


A number of approaches are currently used for assessing andreducing the impact of uncertainty in power system simula-tions - for example, trajectory sensitivity method, probabilisticcollocation method (PCM), grazing, etc. Two excellent surveysfor these three approaches can be found in [88], [89]. The statetrajectory is expressed as a flow function x(t) = φ(t, θ), whereθ is a vector of the model parameters. The trajectories arisingfrom a perturbation ∆θ is approximated as [89]

x(t) = φ(t, θ + ∆θ) ≈ φ(t, θ) + g(t, θ)∆θ (28)

where g(·) is the trajectory sensitivity function. Since (28) isaffine in ∆θ, parametric uncertainties can be easily mappedthrough to bounds around the nominal trajectory. If thesensitivity g(·) is large, then one may try to minimize theuncertainty by estimating the parameter values directly frommeasurements using nonlinear least squares, with the solutionobtained via a Gauss-Newton process. PCM, on the otherhand, uses Gaussian quadrature concepts to select appropriatepoints from the set of uncertain parameters to approximatethe mapping between parameters and outputs of a simulation.Grazing is used to determine the smallest changes in param-eters that would cause a discrete event such as a protectionoperation. These information can help in assessing whetherthe simulation model is robust to parametric uncertainties.

V. CHALLENGE 2: ROLE OF POWER ELECTRONICS

In Section II we introduced dynamic models of powerelectronic converter circuits needed for DER integration. Tra-ditionally, the simulations of such converters are carried outindependently of those for power systems. But with renew-able penetration and significant intrusion of power electronicdevices in distribution grids, these simulations are graduallybecoming more inter-dependent, and must be combined at theright scale [90]. Two important examples of power electronicsare:

1) FACTS (Flexible AC Transmission System), and2) HVDC (High Voltage Direct Current).

Both can assist power system operators during a contingency isrescheduling power flows to avoid overloading of AC transmis-sion lines [91], [92]. Depending on applications, both detailedswitching models and simplified average models may be usedfor simulating FACTS and HVDC converters. For transientstability studies, for example, it is adequate to represent theconverters by their average models in which the valve switch-ings are ignored using duty-cycle based switching functionsor approximated phase-locked loop models, thereby ignoringthe AC and DC harmonics [93]. For other applications suchapproximations may not be valid, and detailed switchingmodels may have to be used. Simulation of switching models,however, can sometimes become problematic for a number ofreasons. The converters operate by using switches to changethe configuration of a network at frequencies up to severalmegahertz, and the time constants of the network are usuallyone or more orders or magnitude larger than the switchingperiod. Each switching event is a discontinuity. Standardsimulators can run for hundreds of switching cycles, creatingproblem for the simulation software that may not have an

explicit mechanism for handling such discontinuities. This canresult in long simulation times due to reduced integration time-steps in the neighborhood of the discontinuity, lack of con-vergence, and erroneous outputs [94]. With the advancementin programming languages in the late 1990s, modeling lan-guages such as Dymola were proposed to resolve this problemby using object-oriented programming. Today, software suchas PLECS, PSIM, PSpice, OpenModelica, PSCAD, VissiM,SABER, COMSOL, Matlab, and LTSpice are all widely usedfor large-scale converter simulations.

Another factor motivating the interest for accommodatingpower electronics in power system simulations is the envi-sioned replacement of conventional low-frequency magnetictransformers in near future by power-electronic transformersmade out of the solid-state technology. These transformersconsist of a front-end rectifier stage, which converts highvoltage AC to high voltage DC, a dual-active bridge (DAB)stage, which converts high DC voltage to a low DC voltageto be used for DC distribution segment, and a voltage-sourceinverter, which converts low DC voltage to low single-phaseAC voltage to be used for AC distribution segments. Sincea typical microgrid may contain several hundreds of theseconverters, a common practice is to use averaged modelsinstead of detailed models, especially for transient stabilitystudies. These models are most often described by nonlineardynamics that are valid only in specified ‘safe zones of oper-ation’ [66]. This is similar to the concept of reachable zonesin DFIG-based wind turbine models, which are often usedfor testing their voltage ride-through capabilities [95]. High-fidelity simulations are required for revealing these safe zonesfor different operating conditions and network topologies.

A solid-state transformer (SST) contains both AC andDC terminals that may be connected to wind, solar, batterystorage, and loads. Fig. 4(a) shows the interface circuit forAC components. A boost converter is coupled with the rectifiercircuit to convert the lower DC voltage of solar generators tolower AC voltage. Fig. 4(b) shows the interface circuit for DCcomponents. PV panels are directly connected with a boostconverter to amplify the panel voltage to the DC bus voltage.Based on voltage level of the panels, other DC-DC converterscan also be used for this interface. A similar interface withbi-directional energy flow can be used to control the batterycharging and discharging for power and energy managementof the system. The charging voltage of a battery is typicallylower than the DC bus voltage. Thus, a good choice for thisinterface circuit is a buck converter for the DC energy cell,and an inverter-coupled buck converter for the AC energy cell.However, to allow flexibility in choosing the voltage rating ofthe battery, a bidirectional buck-boost converter is used for DCbatteries, and an inverter coupled with a buck-boost converteris used for AC batteries. The circuits for AC and DC energystorage are shown in Fig. 5(a) and Fig. 5(b), respectively.

Recent examples in [66] show that when these circuits areconnected to a radial distribution grid, the small-signal modelof the system exhibits a wide variation in time-scales rangingfrom micro-second dynamics arising from switching to tensof seconds of dynamics arising from the inertial effects ofthe DFIGs. Thus, multi-time-scale numerical methods that can


successfully capture such large spectra of eigenvalues need tobe developed as more of these power electronic devices getintegrated to the grid.

+

-+

-

+

-IMPP

(a) Boost converter and rectifier for solar AC interface

+

-

+

-

+

-IMPP

(b) Boost converter for solar DC interface

Fig. 4. Power electronic converter models for renewable generators

+

-+- +-

Ibat

+

-Voc

+

-

(a) Rectifier circuit for AC battery average model

+

-+- +-

Ibat

+

-Voc

+

-

D3

+

-

+

-

(b) Buck-Boost circuit for DC battery average model

Fig. 5. Power electronic converter models for energy storage

Yet another important factor for simulating converter modelsis harmonics. For example, it was recently shown in [66] thatthe effective duty cycle ratio that enters the dynamics of therectifier stage model of a converter can be written as

dα(t) = d1(t) cos(θ(t)) + d2(t) sin(θ(t)), (29)θ(t) = ωs(t) (30)

where d1 and d2 are the d and q-axis components of the dutycycle, and θ represents the dynamics of the phase-locked loop(PLL) controlling the grid frequency. The time constant ofthe PLL is often assumed to be faster compared to that ofthe rectifier, and thereby ωs is approximated to be constantat the synchronous value of 377 rad/sec. Accordingly, θ(t) isapproximated as

θ(t) = θ(t = 0) + ωst. (31)

Depending on more complex dynamic models of the PLL,θ(t) may be simulated by more complicated functions thanjust a simple straight line. The important point to note is that

although θ(t) by itself is an unbounded function, it enters therectifier dynamics through bounded trigonometric functions.The implications of these bounded time-varying terms forjudging the stability of the converter models must be validatedusing simulations.

VI. CHALLENGE 3: CO-SIMULATION

A. Co-simulation of T&D Models

Traditionally, transmission and distribution grid models havealways been treated as two individual, decoupled infrastruc-tures. However, given the recent nationwide initiatives inpromoting demand response (DR), many researchers are nowproposing to bring these two infrastructures together, andco-simulate their models. Recent papers such as [85] haveproposed so-called DR-advisors that act as a recommender forpower consumption prediction and control strategies for theloads on both transmission and distribution sides of the grid.Necessary aggregation techniques may need to be applied hereto lump the transmission grid as a single source of information,as shown in Fig. 6. Once a load is predicted and the DRstrategy is evaluated, co-simulation models can guide operatorson how to update the load setpoints. Highly reliable load fore-casting, as mentioned earlier in Section IVF, will be critical forthis purpose. Online estimation algorithms need to be appliedfor gathering adequate information about weather, possibilitiesof natural disasters, approximate generation schedules over thenext hour, information about transmission line faults that mayseverely disrupt the currents flowing on the distribution side,and so on. The control algorithms for regulating the loads willthereafter need to be tuned accordingly, depending on whetherthe distribution grid can depend on the transmission grid forany deficit power feed.

B. Co-Simulation of Infrastructures

Not just for T&D models, co-simulation is being envisionedas a necessary tool for verification and validation of variousother infrastructures whose functionalities are interconnectedwith power systems [63]. For example, power systems dependon the delivery of fuels to the generation stations throughtransportation services. The production of those fuels dependsin turn on the use of electrical power. The fuels are also neededby the transportation services. Evaluating these types of inter-dependencies is critical when big natural calamities hit thegrid, causing loss of power for extensive periods of time. Fig.7, reproduced from [96], shows a logical diagram on how thepower grid is dependent on seven other major infrastructures,and vice versa. This inter-dependence is only going to increaseover time, thereby necessitating the development of modelingand simulation tools that can assess the risks and possibleimpacts of critical events on quality of life by identifyingfactors that trigger the domino effect between these eightinfrastructures. Co-simulation can be a potential solution forthis problem. Co-simulation refers to a combination of theoryand techniques to enable global simulation of a coupled systemvia the composition of smaller simulators. Each simulator isa black-box mock-up of a constituent infrastructure, whosemodel is developed by its respective operator, allowing for


ZLine1

Z1

ZLine2

Z2

SST 3

DC Load

Generation

Storage

AC Load

Generation

Storage

ZLine3

Z3

SST 2

DC Load

Generation

Storage

AC Load

Generation

Storage

SST 1

DC Load

Generation

Storage

AC Load

Generation

Storage

Microgrid 1 Microgrid 2 Microgrid 312 KV

AC bus

120/240 V

AC bus380 V DC

bus

380 V DC

bus

380 V DC

bus120/240 V

AC bus

120/240 V

AC bus

AC Load

Generation

Storage

Radial Distribution Feeder

Weather

Forecasting

Electricity

Markets

Aggregated

Model of

Transmission

Grid

Fig. 6. Model aggregation for co-simulation of transmission and distribution grids

Oil and

Gas

Communication

Systems

Water

Banking &

Finance

Government

Services

Emergency

Services

Transportation

Power &

Energy

Fig. 7. Interdependence of power systems with other infrastructures

each operator to work on his part of the problem with his owndomain-specific tools. An alternative to co-simulation is co-modelling, where models of the different infrastructures mustbe described in a unified language, and then simulated. Thereare advantages to this approach, but each domain has its ownparticularities when it comes to simulation as a result of whichit is impractical to find a unified language that fits all. Anexcellent survey on this topic was recently presented in [97].

A recent study was done by Sandia National Laboratory in[98] about shock propagation and physical-to-market feedbackin the US natural gas infrastructure using agent-based model-ing and state-space dynamic models. A related study on co-simulation was done in [99] using the idea of co-optimization.Co-optimization models are computer-aided decision-supporttools that search among possible combinations of differentinfrastructure-level investments to identify integrated solutionsthat are best in terms of cost or other objectives whilesatisfying all physical, economic, environmental, and policyconstraints. Recent works in [99], [100] have reviewed the

state-of-the-art in power system expansion planning toolsincluding existing co-optimization models, and also summa-rized data and computational requirements for simulating co-optimization models. The conclusion was that co-optimizationbased modeling and simulations can be excellent tools forsystem expansion planning, and especially important for powersystems given the large transmission investments that are an-ticipated to promote inter-regional power trades and renewableintegration.

A typical co-simulation may consist of one or more of thefollowing topics [97]:

1) Numerical analysis where conditioning, accuracy andstability of the coupled system model must be addressed.

2) Differential-algebraic system simulation: the composi-tions of co-simulation units are made through the al-gebraic constraints, similar to the power system model(25)-(26).

3) Hybrid systems: Co-simulation scenarios, generallyspeaking, involve hybrid systems where bothcontinuous-time state transitions and discrete-eventswitchings are involved.

4) Hierarchy: Interconnected models of infrastructures areoften hierarchical, and thus their co-simulation scenariosmust be hierarchical too. Compositionality properties ofco-simulations are, therefore, important research chal-lenges.

5) Formal verification: Co-simulation orchestrators mustbe equipped with proper formal verification methodsto coordinate the information flows between differentcomputers running models of different infrastructures.

6) Structure and time-scales: Different infrastructures mayhave different structural properties in their dynamics,and different time constants, all of which are reflectedin their co-simulations.

7) Time-varying models: Many subsystems, includingpower systems, can also have different models at dif-ferent levels of abstractions while the same simulationis running. The relationships between these models mustbe known so that correct switching between the levels


of abstraction can be made at the correct time instants.Co-simulations can facilitate fast automated decision-

making for grid operators during emergency conditions. Sig-nificant amount of research still needs to be done, for example,on developing efficient algorithms for discrete-event based co-simulation models, their compositional convergence, hybridco-simulations, identification and handling of discontinuities,real-time constraints, and the necessary simulation standards[97].

VII. CHALLENGE 4: CYBER-PHYSICAL MODELING ANDSIMULATIONS

As mentioned in Section IIIE, power system operations to-day are becoming increasingly dependent on communication.New types of generation, loads and control devices are beingadded to the grid, adding more complexity to its dynamics, andtherefore necessitating the traditional practice of local monitor-ing and control to be replaced by global or system-wide actions[31]. These actions require a robust and resilient communica-tion backbone over which large volumes of grid data can betransported in real-time. Developing simulation packages fortesting and validation of these communication networks, andthe way they interact with the physical operation of the grid iscurrently a big challenge. Challenges in modeling and simula-tion on the communication plane, for example, can arise frommodeling of delays, queuing, packet loss, routing, bandwidthsharing, and load balancing. Other important challenges canarise from inter-operability standards, interfacing middleware,security and privacy constraints. Typically, the Internet cannotprovide the required latency and packet loss performance forgrid operation under high data-rates. Moreover, the networkperformance is highly random, and therefore, difficult to modelaccurately. Very little studies have been conducted to leverageemerging IT technologies such as cloud computing, software-defined networking (SDN), and network function virtualization(NFV), to accelerate this development [101], [102]. Withthe recent revolution in networking technology, these newcommunication mechanisms can open up several degrees offreedom in programmability and virtualization for simulationplatforms. However, customized SDN control and protocols,and sufficient experimental validation using realistic testbedsare still missing in almost all power system applications andin their hardware-in-loop simulations.

Fig. 8 shows a cyber-physical architecture that can bridgethis gap. This architecture, proposed in [32], reflects the threepillars or the three C’s of any typical cyber-physical system,namely, communication, computing, and control. One may useSDN to model communication, cloud computing for compu-tation, and distributed nonlinear control designs for control.The figure shows the example of a transmission networkrepresenting the physical layer, but the same architecture canvery well be applied to distribution systems as well. The powersystem model, which may be running in RTDS or Opal-RT, isdivided into multiple non-overlapping areas that represent bal-ancing regions belonging to different utility companies. Time-synchronized PMU measurements from local buses inside eacharea are communicated to phasor data concentrators (PDCs),

`

Area 1

Area 2 Area 3

Area 4

Power

Transmission

Network

Local

PDC 1

)(3 tx

)(2 tx

)(1 tx

)(4 tx

)(5 tx

)(6 tx

)(7 tx

)(9 tx

Local

PDC 2

Local

PDC 3

)(8 tx

)(10 tx

Internet of Clouds

VM

VM

VM

Local cloud in Area 1

VM

VM

VM

Local cloud in Area 4

Local cloud in

Area 2

)(1 tx

)(2 tx

)(3 tx

VM VM

)(4 tx )(5 tx

Local cloud in

Area 3

VM VM

)(6 tx)(7 tx

Local

PDC 4

)(8 tx

)(9 tx

)(10 tx

Fig. 8. Cloud-in-the loop architecture for wide-area control of power systems[32]

and then to virtual computers residing in a local cloud. Thevirtual computers are denoted as VM (virtual machine) in Fig.8. The geographical location of the VMs can be close to thatof the generators in that area so that the latency from PDC-to-cloud communication is small. The local clouds themselvesconstitute an Internet of clouds that connects the VMs throughan advanced, secure, third-party wide-area communicationnetwork such as SDN. Virtual machines in each local cloudcommunicate with their neighboring machines inside the cloudas well as to those across other clouds exchanging PMU data,and computing control signals via pre-embedded feedbackcontrol laws. The control signals are, thereafter, transmittedback from the local cloud to the actuators of the correspondinggenerator models in RTDS or Opal-RT.

Cloud-in-the-loop CPS architectures can also be useful foremulating SCADA. Recent research trends indicate that in nearfuture control centers will require hybrid software-hardwaresimulation platforms for SCADA and contingency analysis.One example is the Graphic Contingency Analysis (GCA) Tooldeveloped in Pacific Northwest National Laboratory (PNNL)to provide effective decision making support to operators[103]. A SCADA-based state estimation (SE) simulator hasbeen developed recently at Washington State University us-ing the GridSim simulation platform [104]. More advancedSCADA emulators based on Synchrophasor data need to be


developed in the future. These emulators will require a veryrobust CPS platform such as the one shown in Fig. 8 forprocessing data at the substation level, fast communicationof the results to the control center, and synchronization of thedata at the control center in order to generate state estimatesfor an entire power system model. In Section II we highlightedthe mathematical equations for such physical models. In thefollowing subsections we discuss the modeling challenges forthe cyber layer, which in this case is comprised of multitudesof virtual machines dispersed across the internet of clouds.

A. Simulation of Cyber Layer

Data delivery latency and loss rate are important factorsin the performance of wide-area control and protection appli-cations in transmission networks. Software such as RTDMS(real-time dynamics monitoring system), PGDA (phasor griddata analyzer), and GridSim are used for online oscillationmonitoring using Synchrophasors. A list of related open-source software can be found in [105]. These simulationengines need to be integrated into executable actions so thatresults from the monitoring algorithms can be exported toa custom SQL database that can be set to trigger alerts oralarms whenever damping levels of oscillatory modes fallbelow pre-specified thresholds. These alarm signals need tobe communicated to the operator through a reliable commu-nication network so that the operator can take manual actionsto bring the damping back to acceptable levels [106]. Inrecent years, simulation platforms such as ExoGENI-WAMS[107] have been developed to emulate such communicationplatforms. The computation and communication planes areentirely shifted away from the physical infrastructure, asshown in Fig. 8. The idea is similar to the NASPInet describedin Section III, but with an additional layer of cloud computing.Service-based third-party private clouds are used to create on-the-fly virtual machines, which continuously receive real-timestreaming data from specific sets of PMUs, and exchangethem with both intra-cloud and inter-cloud VMs through awide-area network such as Internet2. The measurements aredigitally represented as a periodic stream of data points.The data delivery infrastructure for supporting these typesof applications is still evolving. Another example of a CPSsimulator is GridSim [108]. The data delivery componentin this simulation platform, also referred to as GridStat, isa publish-subscribe middleware, which allows for encryptedmulticast delivery of data. GridStat is designed to meet therequirements of emerging control and protection applicationsthat require data delivery latencies on the order of 10 to 20ms over hundreds of miles with extremely high availability.

Similar to GridStat, the VMs in any cloud-in-the-loop CPSsimulator may consist of two communication planes, namelya data plane and a management plane. The data plane is acollection of forwarding engines (FEs) designed to quicklyroute received messages on to the next VMs. The FEs areentirely dedicated to delivering messages from publishers tosubscribers. Routing configuration information is delivered tothe FEs from the management plane. The forwarding latencythrough an FE implemented in software is generally on theorder of 100 micro-seconds, and with network processor

hardware it is less than 10 micro-seconds. The managementplane, on the other hand, is a set of controllers, called QoSbrokers, that manage the FEs of the data plane for everyVM. The Quality-of-Service (QoS) brokers can be organizedin a hierarchy to reflect the natural hierarchy in the physicalinfrastructure of the grid model. When a subscriber wishes toreceive data from a publisher, it communicates with a QoSbroker that designs a route for the data, and delivers therouting information to the relevant FEs and VMs, creatingthe subscription.

Simulation platforms such as GridStat are just startingpoints for research. Much more advanced cyber emulatorsneed to be developed for the future grid. One outstandingchallenge is to develop a reliable communication softwarethat will enable timely delivery, high reliability and securenetworking for these emulators. This is needed for all appli-cations, whether that be on the transmission side or distributionside. Timeliness of message requires guaranteed upper boundson end-to-end latencies of packets. Legacy networking devicesdo not provide such guarantees, neither for commodity Internetconnections nor for contemporary proprietary IP-based net-works that power providers may operate on. Moreover, directcommunication lacks re-routing capabilities under real-timeconstraints, and resorts to historic data when communicationlinks fail. Both timeliness and link-failures can be handledusing the idea of Distributed Hash Tables (DHT), which wasrecently introduced for wide-area control applications in [109].

B. Simulation of Cyber Vulnerabilities

Another driving motivation for developing CPS-centric sim-ulation tools for power systems is cyber-security [110], [111].The integration of cyber components with the physical gridintroduces new entry points for malicious attackers. Thesepoints are remotely accessible at relatively low risks to attack-ers compared to physical intrusions or attacks on substations.They can be used to mount coordinated attacks to cause severedamages to the grid. Attacks can be originated on the cyberlayer as well to trigger cascading events leading to damages onphysical facilities, leading to major outages. While simulationmodels have been developed to model electrical faults anddevice failures, we still do not have any reliable simulationpackage that can guide us on how the modern-day gridcan be equipped with necessary protections against differenttypes of cyber vulnerabilities. Several universities and nationallaboratories have started developing simulation testbeds toemulate these vulnerability scenarios. Research demonstrationevents such as Cyber Security Awareness Month, which is astudent-run cyber security event in the US, has been introducedby the Department of Homeland Security [112]. The goalis for power system operators to work with these standardsorganizations to develop simulation software that can model,detect, localize, and mitigate cyber vulnerabilities in the gridas quickly as possible.

We conclude this section by showing a simulation of adenial-of-service (DoS) cyber-attack on a wide-area controllerfor the IEEE 39-bus power system. The details of this con-troller were recently reported in [113]. First, the power systemmodel was linearized about the solution of its load flow. Then,


Fig. 9. Simulation of a cyber-attack: time-response of generator speeds before, during, and after a denial-of-service attack

a linear quadratic regulator (LQR)-type optimal controller wasdesigned, and sparsified using matrix sparsification methods.The control problem was stated as:

minK

J =1

2

∫ ∞0

(xTQx+ uTRu)dt (32)

s.t., x = Ax+Bu (33)u = Kx, K = K(G), Q > 0, R ≥ 0 (34)

where, G is a designated sparse graph topology imparted onthe wide-area communication network. The nonlinear modelof the power system was simulated with this sparse state-feedback controller using Power System Toolbox (PST) inMatlab [114]. A fault was induced at t = 0, and the small-signal speed deviations of the synchronous generators wererecorded, as shown in Fig. 9. The closed-loop system behavioris observed to be stable. At t = 10 seconds, a DoS attackinduced on the communication link connecting generators 1and 8, which means that these two generators are no longercapable of exchanging state information for their controlactions. Instability is noticed immediately, with the frequencyswings diverging with increasing amplitude at a frequencyof roughly 0.05 Hz. This can be seen in Fig. 9 onwardsfrom t = 10 sec. At t = 60 sec, the communication linksconnecting generators (2, 6) and (3, 6) are added, and thecorresponding control gains are recomputed. The system isseen to regain stability, indicating that the attack has beensuccessfully mitigated. The frequency deviations are all seento converge to zero over time.

The above example shows the importance of simulationsfor investigating different types of attack scenarios in the grid.Simulations can reveal the most important pairs of generatorsthat must communicate to maintain stable operating conditionbefore and after an attack, and also the lesser important pairsthat, either due to large geographical separation or weakdynamic coupling, do not necessarily add any significantcontribution to stability. Simulation packages also need tobe developed for illustrating various other types of attack

scenarios such as data manipulation attacks, jamming, eaves-dropping, and GPS spoofing [115].

VIII. CHALLENGE 5: TECHNOLOGIES OF SIMULATIONS

A. Parallel Computing

An obvious approach to speed up simulations is to utilizeparallel or distributed computing. Although specially writtenprograms for particular parallel architectures can provide highspeed-ups, the rapidly changing hardware and software makesit impossible to keep modifying the simulation programs tokeep up. The trend today is to use multiprocessor computerswith compilers that can distribute the computation optimallyto multiple processors. For power system simulations, someapplications are much more amenable to parallelization thanothers. For example, any simulation that requires runningmany contingencies (such as security constrained OPF) canrun these hundreds of contingency cases in separate proces-sors. The dynamics of individual generators can be run inparallel but the network that connects the generators has to besolved simultaneously. It turns out, however, that the algebraicequations representing the network cannot be parallelized veryefficiently, and becomes the main bottleneck for speeding uppower system simulations. Parallel computing can also be usedfor gain-scheduling of robust controllers.

B. Hybrid Simulations

In order to implement fast control strategies in grid models,it is highly desirable to have a faster than real-time emulationor simulation of that model in hand. One promising solution ishybrid mixed-signal hardware emulation. In these emulationshardware-based digital simulation and analog simulation canbe used in an integrative way to achieve a massively paralleland scale-insensitive emulation architecture. There have beenseveral attempts at building hardware accelerators in the past[116], [117]. For example, the DAEs (25)-(26) can be emulatedin hardware by a coupled set of oscillators, resistors, capacitorsand active inductors. A higher frequency can be used so asto suit the scale of on-chip elements and permit faster than


real-time operation. These elements then need to be builton chip, and connected with a customizable switch matrix,allowing a large portion of the grid to be modeled in real-time.Researchers have proposed the use of Verilog-AMS model,which is a hardware modeling language that includes featuresfor Analog and Mixed Signal (AMS) elements [118]. It canincorporate equations to model analog sub-components. TheAMS model can be designed to emulate open-loop models ofvery large-scale power systems with tens of thousands of buseswith built-in equations for AC power flow, electro-magneticand electro-mechanical dynamics of synchronous generatorsand induction generators, AC load models, voltage control,droop control, automatic generation control (AGC), and powersystem stabilizers. Several challenges that still stand on theway for developing at-scale faster than real-time simulationsare:

1) How to synthesize the transmission network withoutunnecessary and unrealistic assumptions, and approx-imations using state-of-the-art microelectronics designtechnology

2) How to develop a scalable mixed-signal emulation ar-chitecture capable of large power systems with tens ofthousands nodes

3) How to design configurable units of emulation on achip so that any large power system with realistictransmission connections can be realized via software-based configuration.

Research activities are underway on building VLSI chipsthat can scale to large grid models for faster than real-timeemulators, especially directed towards transient AC emula-tions. In [117], for example, predictive simulations were shownto be very useful for real-time path rating. The concept wasverified using a 39-bus power system model, shown in Fig.10(a). To calculate the total transfer capability of Path 1, thegeneration level in Area I was increased while the generationin Area II was decreased progressively while screening con-tingencies. Comprehensive path rating studies considering N-1 contingencies were conducted using DSA Tools, developedby Powertech Labs. Simulation results, shown in Fig. 10(b),reflect that the total transfer capability can vary significantlyin real time. The path rating estimated with the worst-casescenario would be the smallest of all values for 24 operatinghours. Any capacity above the smallest number is an additionalgain enabled by real-time path rating studies. If the additionaltransfer capability can be fully utilized, a total amount of25.74% more energy can be transferred without compromisingthe current reliability level. This could generate multi-milliondollar revenues.

IX. SIMULATION TESTBEDS

Gaining access to realistic grid models and data owned byutility companies can be difficult due to privacy and non-disclosure issues. More importantly, in many circumstanceseven if real data are obtained they may not be sufficient forstudying the detailed operation of the entire system becauseof their limited coverage. To resolve this problem severalCPS smart grid simulation testbeds have recently been de-veloped to facilitate hardware-in-loop simulation of different

(a) One-line diagram of the 39-bus test system

(b) Transfer rating calculation

Fig. 10. Predictive dynamic simulations for real-time path rating [117]

grid applications without the need for gaining access to realdata. Selected examples of such testbeds in the United Statesinclude CPS testbeds at Washington State University usingGridStat [108], cloud-assisted wide-area control testbed atNorth Carolina State University using ExoGENI [107], cyber-security testbeds at Iowa State [119], a microgrid testbed calledDyMonDS (Dynamic Monitoring and Decision Systems) atCarnegie-Mellon [120], TCIPG in University of Illinois [121],DETER-lab testbed at University of Southern California [122],CPS testbeds at Idaho National Lab, Cornell University, andPacific Northwest National Labs, and a big data hub at TexasA&M [123]. A comprehensive list of many other smart gridtestbeds and their CPS capabilities was recently presentedin the survey paper [124]. Two key questions that most ofthese testbeds are trying to answer are - (1) is it possibleto design sufficiently general CPS standards and protocolsto support a mass plug-and-play deployment of a smart gridwithout sacrificing reliability, data privacy and cyber-security,and (2) if so, then what standards and protocols are requiredto transform today’s grid into an end-to-end enabler of electricenergy services.


A. Hardware Components

Generally speaking, the physical component of thesetestbeds are comprised of Real-Time Digital Simulators(RTDS) and Opal-RT. These are power system simulatortools that allow real-time simulation of both transmission anddistribution models with a time-step of 50 microseconds. TheRTDS comes with its own proprietary software known asRSCAD, which allows the user to develop detailed dynamicmodels of various components in prototype power systems.The RTDS also comes with digital cards that allow externalhardware to interface with the simulation. For example, theGigabit Transceiver Analog Output (GTAO) card allows theuser to view low-level signals proportional to voltages andcurrents at different buses of the system in real-time. TheGTAO card generates voltage and current waveforms, andcommunicates them to sensors such as relays, circuit breakers,and PMUs. The PMUs measure these signals, and send theresulting digitized phasor data calculations to the PDCs. ThePDC time-stamps and collects the data from all the PMUs,and sends them to the server for display and archival, whenrequested. The hardware and the software layers of thesetestbeds are integrated with each other to create a substation-like environment within the confines of research laboratories.The two layers symbiotically capture power system dynamicsimulations as if these measurements were made by real sen-sors installed at the high-voltage buses of a real transmissionsubstation.

B. Cyber and Software Components

The cyber-layer, on the other hand, is generally emulatedby either a local-area network or a local cloud service. TheExoGENI-WAMS testbed at NC State [107], for example,is connected to a state-funded, metro-scale, multi-layeredadvanced dynamic optical network testbed called BreakableExperimental Network (BEN) that connects distributed cloudresources in local universities [125]. It allows one to set updynamic multi-layer connections of up to 10 Gbps betweendifferent sites. One may simulate different types of disturbanceevents in power system models in RTDS, collect the emulatedgrid responses via PMUs and other sensors, communicatethese data streams via BEN, and run virtual computing nodesat various sites in the ExoGENI cloud overlaid on top of BENto execute distributed estimation and control algorithms. Someopen questions for these types of set-ups in the future are, forexample - where to deploy the computing facilities to betterfacilitate data collection and processing, and how to designbetter communication topologies.

C. A Network of Remote Testbeds

One pertinent question is whether these different simulationtestbeds at different locations should be conjoined with eachother to create a much bigger nationwide network of CPStestbeds. And if yes, then what are the most common chal-lenges for such remote testbed federation? Developing proto-cols for usability by different users, and potential safety haz-ards are two important challenges, for example. Researchersare also contemplating making their testbeds open to publicfor accelerating research in the power and CPS community,

but a robust economic and ethics model for sharing access toprivate resources still needs to be developed. Should there bea common centralized simulation testbed for accessing powersystem model and data, one must also resolve standardizationissues, communication issues, maintenance costs, and strate-gies for sustainability.

D. Interoperability: Databases and User Interfaces

One of the major challenges for the users of the hundreds ofexisting simulations is that none of them are compatible witheach other. Using two different simulations require keeping uptwo different databases of input data, and being familiar withtwo different sets of graphical outputs. This is not only true fordifferent types of applications but also of the same applicationmarketed by different vendors. Thus, in the current state ofart it is impossible to integrate these different simulationprograms. The easiest way to encourage interoperability is tostandardize the databases. The data that go into these databasesare proprietary to the utilities, and if the utilities can agree onusing a standard database the simulation vendors will have toadopt it. In the USA, the National Institute for Standards andTechnology (NIST) has been tasked to develop such standards.An earlier standard called the Common Information Model(CIM) is now an IEC standard, and is slowly being adoptedat different rates in different countries. A similar effort shouldbe made to standardize user interfaces. If the database anduser interface for a simulation is standardized, the abilityto integrate different simulators as mentioned above wouldbecome much simpler.

X. CONCLUSIONS

In this paper, we covered the state of the art of power systemsimulations, which has evolved since the 1960s by utilizingthe rapidly increasing power of digital computation. Increasedcomputational capabilities have resulted in more accuratemodeling and more powerful algorithms that allow engineersmany more analytical tools at their fingertips to obtain results.The challenge, however, is arising from a different direction.The grid itself is changing because of the proliferation ofnew technologies. New sources of renewable generation thatare intermittent and have no rotating inertia, are being addedrapidly. New types of loads such as electric vehicles and smartbuildings are also proliferating. Power electronic convertersand controllers are being introduced to connect these newgeneration sources, load and storage devices to the grid. Manyof these sources are being added to the distribution system andon the customer side of the meter, thus making the operationof the grid more complicated.

Added to this, the tremendous improvement in computer andcommunication technologies over the past decade, in additionto improving simulation capabilities, has also improved theoperation and control of the power grid. The modern grid isbeing overlaid with more sensors, communications, computers,information processors, and controllers (ICT) so that it can bebetter monitored and controlled. The combination of all ofthese modern technology makes the grid the largest cyber-physical system (CPS) in the world. The challenge is that allthese new technologies including the ICT must be modeled


and simulated together. We highlighted these challenges inmodeling, power electronics, co-simulation, cyber-physicalsystems and simulation technologies. We are optimistic thatthese challenges will be met by even more judicious use ofcomputational technology. Growth in computing power showsno signs of slowing down, so we do not foresee any limitationson model size or algorithm speed. However, the ability toutilize such powerful simulations depends on how easy theirhandling can be made to the engineers. Moreover, powersystems today are gradually becoming an integral part of otherinterconnected infrastructures such as gas networks, trans-portation networks, communication networks, water networks,economics, and food chain networks. Thus, inter-operabilityof simulation programs will become a key to minimizing themanual effort needed to set up and run co-simulations of theselarge interacting systems, and interpret the results.

XI. ACKNOWLEDGEMENTS

The first author would like to thank Tomonori Sadamoto(Tokyo Institute of Technology), Ryan Goodfellow (Informa-tion Sciences Institute), Yufeng Xin (Renaissance Comput-ing Institute), Pramod Khargonekar (University of CaliforniaIrvine), and Abhishek Jain (NC State University) for helpfuldiscussions on different parts of this paper.

REFERENCES

[1] A. Bose, “Smart Transmission Grid Applications and Their SupportingInfrastructure,” IEEE Transactions on Smart Grid, vol. 1(1), 2010.

[2] A. Chakrabortty and P. Khargonekar, “Introduction to Wide-Area Con-trol of Power Systems,” Proceedings of American Control Conference,DC, 2013.

[3] K. Tomsovic, D. E. Bakken, V. Venkatasubramanian, and A. Bose,“Designing the Next Generation of Real-Time Control, Communication,and Computations for Large Power Systems,” Proceedings of the IEEE,vol. 93(5), pp. 965-979, 2005.

[4] M. D. Ilic, “Dynamic Monitoring and Decision Systems for EnablingSustainable Energy Services,” Proceedings of the IEEE, vol. 99(1), pp.58-79, Jan. 2011.

[5] D. Bakken, A. Bose, K. M. Chandy, P. P. Khargonekar, A. Kuh, S.Low, A. von Meier, K. Poolla, P. P. Varaiya, and F. Wu, “GRIP - Gridswith Intelligent Periphery: Control Architectures for Grid 2050,” IEEEInternational Conference on Smart Grid Communications, 2011.

[6] IEEE Vision for Smart Grid Controls: 2030 and Beyond, Availableonline: http://ieeexplore.ieee.org/document/6577608/

[7] M. D. Ilic, J. Joo, L. Xie, M. Prica, and N. Rotering, “A Decision-Making Framework and Simulator for Sustainable Electric EnergySystems,” IEEE Transactions on Sustainable Energy, vol. 2(1), pp. 37 -49, 2011.

[8] L. Xie, Y. Zhang, and M. D. Ilic, “Multi-scale Integration of Physics-Based and Data-Driven Models in Power Systems,” IEEE/ACM ThirdInternational Conference on Cyber-Physical Systems, pp. 129 - 137,2012.

[9] J. D. Glover, T. Overbye, and M. S. Sarma, Power System Analysis andDesign, 6th edition, 2016.

[10] A. M Gole, S. Filizadeh, R. W Menzies, and P. L Wilson, “Optimization-enabled Electromagnetic Transient Simulation,” IEEE Transactions onPower Delivery, vol. 20(1), pp. 512 - 518, 2005.

[11] P. Kundur, Power System Stability and Control, McGraw-hill, 1994.[12] P. Sauer and M. A. Pai, Power System Dynamics and Stability, Prentice

Hall, 1998.[13] P. F. Odgaard, J. Stoustrup, and M. Kinnaert, “Fault-Tolerant Control of

Wind Turbines: A Benchmark Model,” IEEE Transactions on ControlSystems Technology, 2013.

[14] T. Sadamoto, A. Chakrabortty, T. Ishizaki, and J. Imura, “RetrofitControl of Wind Power Systems,” submitted to IEEE Transactions onPower Systems, 2017.

[15] E. Muljadi, M. Singh, and V. Gevorgian, “User Guide for PV DynamicModel Simulation Written on PSCAD Platform,” Technical Report,NREL/TP-5D00-62053, National Renewable Energy Lab., 2014.

[16] R. G. de Almeida and J. A. Pecas Lopes, “Participation of DoublyFed Induction Wind Generators in System Frequency Regulation,” IEEETransactions on Power Systems, vol. 22(3), Aug. 2007.

[17] N. Ullah, T. Thiringer, and D. Karlsson, “Temporary Primary FrequencyControl Support by Variable Speed Wind Turbines Potential andApplications,” IEEE Transactions on Power Systems, vol. 23(2), 2008.

[18] J. Ingleson and E. Allen, “Tracking the Eastern Interconnection Fre-quency Governing Characteristic,” IEEE PES General Meeting, July2010.

[19] Z. Miao, L. Fan, D. Osborn, and S. Yuvarajan, “Control of DFIG-based Wind Generation to Improve Interarea Oscillation Damping,”IEEE Transactions on Energy Conversion, vol. 24(2), pp. 415 - 422,2009.

[20] A. Ortega and F. Milano, “Generalized Model of VSC-Based EnergyStorage Systems for Transient Stability Analysis,” https://arxiv.org/abs/1509.05290v1.

[21] N. Diorio, A. Dobos, S. Janzou, A. Nelson, and B. Lundstrom, “Techno-economic Modeling of Battery Energy Storage in SAM,” TechnicalReport, NREL/TP-6A20-64641, National Renewable Energy Laboratory,2015.

[22] R. Hu, State of Charge Balancing Droop Control, MS Thesis, MichiganTech University, 2015.

[23] IEEE Task Force, “Standard Load Models for Power Flow and DynamicPerformance Simulation,” IEEE Transactions on Power Systems, vol.10(3), 1995.

[24] I. A. Hiskens and J. V. Milanovic, “Load Modeling in Studies of PowerSystem Damping,” IEEE Transactions on Power Systems, vol. 10(4), pp.1781 - 1788, 1995.

[25] D. Hill, “Nonlinear Dynamic Load Models with Recovery for VoltageStability Studies,” IEEE Transactions on Power Systems, vol. 8(1), pp.166 - 176, 1993.

[26] W. Zhang, K. Kalsi, J. Fuller, M. Elizondo, and D. Chassin, “Aggre-gate Model for Heterogeneous Thermostatically Controlled Loads withDemand Response,” IEEE PES Gen. Meeting, San Diego, CA, 2012.

[27] J. L. Mathieu, P. N. Price, S. Kiliccote, and M. A. Piette, “QuantifyingChanges in Building Electricity Use, with Application to DemandResponse,” IEEE Transactions on Smart Grid, vol. 2(3), pp. 507-518,Sep. 2011.

[28] T. V. Cutsem, Voltage Stability of Electric Power Systems, Springer,1998.

[29] J. Zhang, S. Nabavi, A. Chakrabortty, and Y. Xin, “ADMM OptimizationStrategies for Wide-Area Oscillation Monitoring in Power Systems underAsynchronous Communication Delays,” IEEE Transactions on SmartGrid, vol. 7(4), pp. 2123 - 2133, 2016.

[30] J. Zhang, S. Nabavi, A. Chakrabortty, and Y. Xin, “ConvergenceAnalysis of ADMM Based Power System Mode Estimation UnderAsynchronous Wide-Area Communication Delays,” IEEE PES GeneralMeeting, Denver, CO, July 2015.

[31] S. Nabavi, J. Zhang, and A. Chakrabortty, “Distributed OptimizationAlgorithms for Wide-Area Oscillation Monitoring in Power SystemsUsing an Inter-Regional PMU-PDC Architectures,” IEEE Transactionson Smart Grid, vol. 6(5), pp. 2551 - 2559, Sep. 2015.

[32] D. Soudbakhsh, A. Chakrabortty, and A. M. Annaswamy, “A Delay-Aware Cyber-Physical Architecture for Wide-Area Control of PowerSystems,” IFAC Control Engineering Practice, vol. 60, pp. 171-182,2017.

[33] J. G. Deshpande, K. C. Budka, and M. Thottan, Communication Net-works for Smart Grids, Springer, 2014.

[34] D. Anderson, W. S. Cleveland, B. Xi, “Multifractal and Gaussianfractional sumdifference models for Internet traffic,” Performance Eval-uation, vol. 107, Jan. 2017.

[35] RTDS Technologies Inc.: https://www.rtds.com/[36] OPAL-RT: www.opal-rt.com[37] PowerWorld: https://www.powerworld.com[38] F. Milano, “A Python-Based Software Tool For Power System Analysis,”

IEE Power and Energy Society General Meeting, 2013.[39] F. Milano, L. Vanfretti, and J. C Morataya, “An Open Source Power

System Virtual Laboratory: The PSAT Case and Experience,” IEEETransactions on Education, vol. 51(1), pp. 17-23, 2008.

[40] R. C. Dugan, “The Open Distribution System Simulator (OpenDSS),”EPRI OpenDSS Manual, Nov. 2012.

[41] C. Ptolemaeus (Ed), System Design, Modeling, and Simulation UsingPtolemy II, 2014.

[42] P. Fritzson, Introduction to Modeling and Simulation of Technical andPhysical Systems with Modelica, Wiley, IEEE Press, Sep. 2011.


[43] G. Giorgidze and H. Nilsson, “Embedding a Functional Hybrid Mod-elling Language in Haskell,” Implementation and Application of Func-tional Languages, pp. 138-155, Springer, 2011.

[44] D. Zimmer, “A New Framework for the Simulation of Equation-BasedModels with Variable Structure,” Simulation, vol. 89(8), pp. 935963,2013.

[45] D. K. Molzahn and I. A Hiskens, “Sparsity-Exploiting Moment-BasedRelaxations of the Optimal Power Flow Problem,” IEEE Transactionson Power Systems, vol. 30(6), pp. 3168 - 3180, 2015.

[46] J. Lavaei and S. Low, “Zero Duality Gap in Optimal Power FlowProblem,” IEEE Transactions on Power Systems, vol. 27(1), Feb. 2012.

[47] D. K Molzahn, J. T Holzer, B. C Lesieutre, and C. L DeMarco,“Implementation of a Large-Scale Optimal Power Flow Solver BasedOn Semidefinite Programming,” IEEE Transactions on Power Systems,vol. 28(4), pp. 3987 - 3998, 2013.

[48] D. K. Molzahn, B. C. Lesieutre, and C. L DeMarco, “A SufficientCondition for Global Optimality of Solutions to the Optimal Power FlowProblem,” IEEE Transactions on Power Systems, vol. 29(2), pp. 978 -979, 2014.

[49] R. Madani, S. Sojoudi, and J. Lavaei, “Convex Relaxation for OptimalPower Flow Problem: Mesh Networks,” IEEE Transactions on PowerSystems, vol. 30, no.1, pp. 199-211, 2015.

[50] J. Blackburn and T. Domin, Protective Relaying: Principles and Appli-cations, CRC Press, 2014.

[51] N. Zhang and M. Kezunovic, “Implementing an Advanced SimulationTool for Comprehensive Fault Analysis,” IEEE PES Transmission andDistribution Conference and Exhibition: Asia and Pacific, 2005.

[52] S. Takayama and R. Matsuhashi, “Development of Model for LoadFrequency Control in Power System with Large-Scale Integration ofRenewable Energy,” IEEE Power and Energy Conf. at Illinois, 2016.

[53] M. Prais, C. Johnson, A. Bose, and D. Curtice, “Operator TrainingSimulator: Component Models,” IEEE Transactions on Power Systems,vol. 4(3), Aug 1989.

[54] J. M. Bucciero, J. A. Dodge, R. N. Gillespie, R. O. Hinkel, S. L.Mann, S. M. Martin, R. P. Schulte, D. Hayward, “Dispatcher TrainingSimulators: Lessons Learned,” IEEE Transactions on Power Systems,vol. 6(2), May 1991.

[55] H. Kim and C. Singh, “Reliability Modeling and Simulation in PowerSystems With Aging Characteristics.” IEEE Transactions On PowerSystems, vol. 25(1), 2010.

[56] J. F. Prada, “The Value of Reliability in Power Systems - PricingOperating Reserves,” Technical Report EL99-005 WP, MIT, 1999.

[57] M. Shahidehpour, H. Yamin, and Z. Li, Market Operations in ElectricPower Systems: Forecasting, Scheduling, and Risk Management, JohnWiley and Sons, 2003.

[58] M. D. Ilic, L. Xie, and J. Joo, “Efficient Coordination of Wind Powerand Price-Responsive DemandPart I: Theoretical Foundations,” IEEETransactions on Power Systems, vol. 26(4), pp. 1875 - 1884, 2011.

[59] Open Source Software (OSS) for Electricity Market Research, Teaching,and Training, Available online: http://www2.econ.iastate.edu/tesfatsi/ElectricOSS.htm\#Markets

[60] J. Goldis, A. Rudkevich, L. Omondi, and R. Tabors, “Use of CloudComputing in Power Market Simulations,” presented at the FERCTechnical Conference on Increasing Real-Time and Day-Ahead MarketEfficiency through Improved Software, Washington, DC June, 2014.

[61] K. Kalsi, S. Elbert, M. Vlachopoulou, N. Zhou, and Z. Huang,“Advanced Computational Methods for Security Constrained FinancialTransmission Rights,” IEEE PES General Meeting, 2012.

[62] Y. Yan, Y. Qian, H. Sharif, and D. Tipper, “A Survey on SmartGrid Communication Infrastructures: Motivations, Requirements andChallenges,” IEEE Communications Surveys and Tutorials, vol. 15(1),pp. 5 - 20, 2013.

[63] A. Annaswamy, A. Hussain, A. Chakrabortty, and M. Cvetkovic, “Foun-dations of Infrastructure-CPS,” American Control Conference, pp. 2649-2664, Boston, MA, 2016.

[64] P. T. Myrda, J. Taft, and P. Donner, “Recommended Approach toa NASPInet Architecture,” 45th Hawaii International Conference onSystem Science, (HICSS), 2012.

[65] http://www.elp.com/articles/powergrid international/print/volume-14/issue-8/features/ami-communications-technology.html

[66] A. Milani, M. T. Khan, A. Chakrabortty, and I. Husain, “EquilibriumPoint Analysis and Power Sharing Methods for Distribution SystemsDriven by Solid-State Transformers,” IEEE Transactions on PowerSystems, 2017.

[67] K. Sadek, G. Wild, L. Hugelschafer, A. Gole, X. Jiang, and D. Brandt,“Modelling of Digital HVDC Control Systems using a Graphical Elec-

tromagnetic Simulation Program,” International Conference on PowerSystems Transients, Lisbon, Portugal, 1995.

[68] L.G. Perez, A.J. Flechsig, and V. Venkatasubramanian, “Modeling theProtective System for Power System Dynamic Analysis,” IEEE Trans-actions on Power Systems, vol. 9(4), pp. 19631973, Nov. 1994.

[69] V. Venkatasubramanian, H. Schattler, and J. Zaborsky, “Dynamics ofLarge Constrained Nonlinear Systems: A Taxonomy Theory,” Proceed-ings of the IEEE, vol. 83(11), pp. 1530-1561, Nov. 1995.

[70] A. M. Boker, T. R. Nudell, and A. Chakrabortty, “On Aggregate Controlof Clustered Consensus Networks,” American Control Conference, pp.5527-5532, Chicago, IL, USA, 2015.

[71] S. Hara, J. I. Imura, K. Tsumura, T. Ishizaki, and T. Sadamoto, “Glocal(Global/Local) Control Synthesis for Hierarchical Networked Systems,”IEEE Conference on Control Applications (CCA), pp. 107-112, 2015.

[72] T. Sadamoto, T. Ishizaki, and J. I. Imura, “Hierarchical DistributedControl for Networked Linear Systems,” 53rd IEEE Conference onDecision and Control, pp. 2447-2452, 2014.

[73] A. Chakrabortty, “Wide-Area Damping Control of Power Systems UsingDynamic Clustering and TCSC-Based Redesigns,” IEEE Transactionson Smart Grid, vol. 2(3), Nov. 2012.

[74] N. Xue and A. Chakrabortty, “H2-Clustering of Closed-loop ConsensusNetworks under a Class of LQR Design,” American Control Conference,2016.

[75] G. Chavan, M. Weiss, A. Chakrabortty, S. Bhattacharya, A. Salazar,and F. Ashrafi, “Real-Time Identification and Predictive Analysis of aMulti-Area WECC Power System Model Using Synchrophasors,” IEEETransactions on Smart Grid, vol. 8(4), pp. 1977 - 1986, 2017.

[76] S. Deml, A. Ulbig, T. Borsche, and G. Andersson, “The Role ofAggregation in Power System Simulation,” IEEE PowerTech, Eindhoven,2015.

[77] A. Chakrabortty and A. Salazar, “Building a Dynamic Electro-mechanical Model for the Pacific AC Intertie using Distributed Syn-chrophasor Measurements,” European Transactions on Electric Power,Special Issue on PMU Applications, (Editors: C. C. Liu, M. Crow, J.H. Chow), vol. 21(4), pp. 1657-1672, 2011.

[78] A. Chakrabortty, J. H. Chow, and A. Salazar, “A Measurement-basedFramework for Dynamic Equivalencing of Power Systems using Wide-Area Phasor Measurements,” IEEE Transactions on Smart Grid, vol.1(2), pp. 68-81, 2011.

[79] S. Nabavi and A. Chakrabortty, “Topology Identification for DynamicEquivalent Models of Large Power System Networks,” American ControlConference, DC, 2013.

[80] G. B. Dantzig and P. Wolfe, “Decomposition Principle for LinearPrograms,” Operations Research, vol. 8(1), pp. 101 - 111, 1960.

[81] J. H Chow, A. Chakrabortty, M. Arcak, B. Bhargava, and A. Salazar,“Synchronized phasor data based energy function analysis of dominantpower transfer paths in large power systems,” IEEE Transactions onPower Systems, vol. 22(2), pp. 727-734, 2007.

[82] R. Weron, “Electricty Price Forecasting: A Review of the State-of-the-art with a Look Into the Future,” International Journal of Forecasting,vol. 30(4), pp. 1030-1081, 2014.

[83] F. Lian, A. Chakrabortty, and A. Duel-Hallen, “Game-Theoretic Multi-Agent Control and Network Cost Allocation Under CommunicationConstraints,” IEEE Journal on Selected Areas in Communications, vol.35(2), pp. 330 - 340, 2017.

[84] M. Negnevitsky, P. Mandal, and A. K. Srivastava, “An Overview ofForecasting Problems and Techniques in Power Systems,” IEEE PESGeneral Meeting, 2009.

[85] M. Behl, A. Jain, and R. Mangharam, “Data-Driven Modeling, Controland Tools for Cyber-Physical Energy Systems,” ACM/IEEE 7th Inter-national Conference on Cyber-Physical Systems (ICCPS), Apr. 2016.

[86] S. Hui, C. Lee, and F. Wu, “Electric Springs - A New Smart GridTechnology,” IEEE Transactions on Smart Grid, vol. 3(3), pp. 1552 -1561, 2012.

[87] L. Zhu and D. J. Hill, “Modeling and Stability of Microgrids with SmartLoads,” 20th IFAC World Congress, pp. 10438 - 10443, France, 2017.

[88] I. A. Hiskens and J. Alseddiqui, “Sensitivity, Approximation and Un-certainty in Power System Dynamic Simulation,” IEEE Trans. on PowerSystems, vol. 21(4), 2006.

[89] I. A. Hiskens, B. C. Lesiutre, and S. Roy, “Uncertainty Evaluation andMitigation in Electrical Power Systems,” XI Symposium Of Specialists inElectric Operational And Expansion Planning (SEPOPE), Brazil, 2009.

[90] A. M Gole et al., “Guidelines for Modeling Power Electronics inElectric Power Engineering Applications,” IEEE Transactions on PowerDelivery, vol. 12(1), pp. 505 - 514, 1997.

[91] N. G. Hingorani and L. Gyugyi, Understanding FACTS: Concepts andTechnology of Flexible AC Transmission Systems, Wiley, 1999.


[92] U. P. Mhaskar and A. M. Kulkarni, “Power Oscillation Damping usingFACTS Devices: Modal Controllability, Observability in Local Signalsand Location of Transfer Function Zeros,” IEEE Transactions on PowerSystems, vol.21(1), 2006.

[93] K. R. Padiyar, Structure Preserving Energy Functions in Power Systems:Theory and Applications, CRC Press, 2013.

[94] J. S. Glaser, F. E. Cellier, and A. F. Witulski, “Object-Oriented PowerSystem Modeling Using the Dymola Modeling Language,” 26th AnnualIEEE Power Electronics Specialists Conference, Atlanta, GA, 1995.

[95] H. N. Pico and D. C. Aliprantis, “Voltage Ride-Through Capability Ver-ification of DFIG-Based Wind Turbines Using Reachability Analysis,”IEEE Transactions on Energy Conversion, vol. 31(4), pp. 1387 - 1398,2016.

[96] http://www.cis.uniroma1.it/node/5614[97] C. Gomes, C. Thule, D. Broman, and P. G. Larsen, “Co-simulation:

State of the art,” Available online: arxiv.org/pdf/1702.00686.pdf[98] P. H. Kobos et al., “Natural Gas Value-Chain and Network Assessments,”

Technical Report, SAND2015-7915, Sandia National Labs, 2015.[99] A. L. Liu, B. H. Hobbs, J. Ho, J. D. McCalley, V. Krishnan, M.

Shahidehpour, and Q. P. Zheng, “White Paper: Co-optimization ofTransmission and Other Supply Resources,” Prepared for Eastern In-terconnection States’ Planning Council and National Association ofRegulatory Utility Commissioners, 2013.

[100] M. Caramanis, E. Ntakou, W. W. Hogan, A. Chakrabortty, and J.Schoene, “Co-optimization of power and reserves in dynamic T&Dpower markets with nondispatchable renewable generation and dis-tributed energy resources,” Proceedings of the IEEE, vol. 104(4), pp.807-836, 2016.

[101] M. Jarschel, S. Oechsner, D. Schlosser, R. Pries, S. Goll and P. Tran-Gia, “Modeling and Performance Evaluation of an Openflow Architec-ture,” Proceedings of the 23rd International Teletraffic Congress, 2011.

[102] Y. Xin, I. Baldine, J. Chase, T. Beyene, B. Parkhurst, and A.Chakrabortty, “Virtual smart grid architecture and control framework,”IEEE International Conference on Smart Grid Communications, 2011.

[103] Graphical Contingency Analysis Tool at PNNL: http://eioc.pnnl.gov/research/contingencyanalysis.stm

[104] D. Anderson, C. Zhao, C. H. Hauser, V. Venkatasubramanian, D. E.Bakken, and A. Bose, “A Virtual Smart Grid,” IEEE Power and EnergyMagazine, Jan/Feb 2012.

[105] SmarTS Lab, Royal Institute of Technology (KTH), Sweden: https://github.com/SmarTS-Lab

[106] M. Chenine, L. Vanfretti, S. Bengtsson, and L. Nordstrom, “Im-plementation of an Experimental Wide-Area Monitoring Platform forDevelopment of Synchronized Phasor Measurement Applications,” IEEEPower and Energy Society General Meeting, 2011.

[107] A. Chakrabortty and Y. Xin, “Hardware-in-the-Loop Simulations andVerifications of Smart Power Systems Over an Exo-GENI Testbed,” pro-ceedings of 2nd GENI Research and Educational Experiment Workshop,GREE-2013, Utah, Mar. 2013.

[108] GridStat: www.gridstat.net[109] T. Qian, A. Chakrabortty, F. Mueller, and Y. Xin, “A Distributed

Storage System for Multi-Resolution Virtual Synchrophasors,” IEEEPES General Meeting, Washington DC, July 2014.

[110] S. Sridhar, A. Hahn, and M. Govindarasu, “CyberPhysical SystemSecurity for the Electric Power Grid,” Proceedings of the IEEE, vol.100(1), pp. 210-224, 2012.

[111] J. Zhang, P. Jaipuria, A. Hussain, and A. Chakrabortty, “Attack-Resilient Estimation of Power System Oscillation Modes using Dis-tributed and Parallel Optimization: Theoretical and Experimental Meth-ods,” Conference on Decision and Game Theory for Security (GameSec),Los Angeles, CA, 2014.

[112] National Cyber Security Awareness Month: https://www.it.iastate.edu/blog/view/36

[113] A. Jain, A. Chakrabortty, and E. Biyik, “An Online StructurallyConstrained LQR Design for Damping Oscillations in Power SystemNetworks,” American Control Conference, Seattle, WA, 2017.

[114] K. Cheung and J. H. Chow, “A Toolbox for Power System Dynamicsand Control Engineering Education and Research,” IEEE transactionson Power Systems, vol. 7(4), pp. 1559 - 1564, 1992.

[115] G. Dan, H. Sandberg, M. Ekstedt, and G. Bjrkman, “Challenges inPower System Information Security,” IEEE Security & Privacy, vol.10(4), pp. 62-70, 2012.

[116] I. Nagel, “Analog Microelectronic Emulation for Dynamic PowerSystem Computation,” PhD Thesis, EPFL, 2013.

[117] S. Jin, Z. Huang, R. Diao, D. Wu, and Y. Chen, “Parallel Imple-mentation of Power System Dynamic Simulation,” IEEE PES GeneralMeeting, 2013.

[118] E. J. Wyers, M. Steer, C. Kelley, and P. Franzon, “A Bounded andDiscretized Nelder-Mead Algorithm Suitable for RFIC Calibration,”IEEE Transactions on Circuits and Systems I: Regular Papers, vol.60(7), pp. 1787-1799, Jul. 2013.

[119] Power Infrastructure Cybersecurity Laboratory: http://powercyber.ece.iastate.edu/penetintro.html

[120] M. D. Ilic, L. Xie, and Q. Liu, Engineering IT-Enabled SustainableElectricity Services: The Tale of Two Low-Cost Green Azores Islands,vol. 30, Springer, 2013.

[121] TCIPG: Trustworthy Cyber Infrastructure for the Power Grid, https://tcipg.org

[122] The DETER Project: https://deter-project.org[123] Global Network for Synchrophasor Solutions: http://gnssconsortium.

org[124] M. H. Cintuglu, O. A. Mohammed, K. Akkaya, and A. S. Uluagac,

“A Survey on Smart Grid Cyber-Physical System Testbeds,” IEEECommunications Surveys & Tutorials, vol. 19(1), First Quarter 2017.

[125] GENI: www.geni.net

Aranya Chakrabortty received his PhD degree inElectrical Engineering from Rensselaer PolytechnicInstitute, Troy, NY in 2008. He is currently anAssociate Professor in the Electrical and ComputerEngineering department of North Carolina StateUniversity, Raleigh, NC. His research interests arein all branches of control theory with applications topower systems, especially in wide-area monitoring,stability, and control of large power systems us-ing Synchrophasors. He is particularly interested ininvestigating different cyber-physical modeling and

control challenges for the next-generation power grid, at both transmission anddistribution levels. He currently serves as an Associate Editor for the IEEEControl Systems Society Conference Editorial Board (2012-present) and alsofor the IEEE Transactions on Control Systems Technology (2015-present). Hereceived the NSF CAREER award in 2011.

Anjan Bose has over forty years of experience inindustry and academia, as an engineer, educator,and administrator. He is well known as a tech-nical leader in the power grid control industry, aresearcher in electric power engineering, an educa-tor in engineering, and an administrator in highereducation. He is a Regents Professor at WashingtonState University (WSU), where he also served asthe Dean of Engineering and Architecture (1998-2005) and in 2012-13 served as a Senior Advisor tothe US Department of Energy (DOE) in the Obama

administration. Dr. Bose is a Member of the US National Academy ofEngineering (2003) and has served on many National Academy Committees.He is a founding Member of the Washington State Academy of Sciences andhas been elected as its President. He is also a Foreign Fellow of the IndianNational Academy of Engineering. He is a Fellow of the IEEE and is activein several international professional societies. He was the recipient of theOutstanding Power Engineering Educator Award (1994), the Third MilleniumMedal (2000) and the Herman Halperin Electric Transmission & DistributionAward (2006), from the IEEE. He has been recognized as a distinguishedalumnus of the Indian Institute of Technology, Kharagpur (2005) and theCollege of Engineering at Iowa State University (1993).

proceedings of the ieee, vol. x, no. xx, pp. 1 -23, 2017 1 … · 2017-08-07 · proceedings of the...

Documents