ERCOT Wide Area Network Overview & EDS Communication Standard

John FisherERCOTNetwork Manager

2

ERCOT WAN

• The ERCOT WAN consists of 2 separate Wide Area Networks– IP WAN– Voice and RTU WAN (aka “Point-to-Point network)

• IP WAN (MPLS)– MPLS on Sprint network– Each MP is allocated 1.544Mbps bandwidth– ERCOT “core” is allocated 9Mbps per site (Austin and Taylor)– Majority of traffic is XML, HTTPS, and ICCP

• Voice and RTU WAN (Point-to-Point)– Dedicated T1s through AT&T DACS network– Provides connectivity for voice systems (turret, hotline, OPX)– Provides connectivity for RTUs– Provides redundant data connection for IP WAN

3

IP Data WAN Overview

ERCOT AUSTIN

ERCOT TAYLOR

Cisco 7200 Series VXR

3

1

0

2

4

Cisco 7200 Series VXR

3

1

0

2

4

Cisco 7200 Series VXR

3

1

0

2

4

SPRINT MPLSNETWORK

T11.544Mbps

Fractional T1128Kbps à 768Kbps

MARKET PARTICIPANT NETWORK

POWER RPS ACTIVITY

Cisco 2600 SERIES

R

POWER RPS ACTIVITY

Cisco 2600 SERIES

R

UTILSTAT

DUPLEXSPEED

SYSTEMRPS

CATALYST 35503

4

5

6

7

8

9

10

11

12

1

2

15

16

17

18

19

20

21

22

23

24

13

14

1 2

UTILSTAT

DUPLEXSPEED

SYSTEMRPS

CATALYST 35503

4

5

6

7

8

9

10

11

12

1

2

15

16

17

18

19

20

21

22

23

24

13

14

1 2

DS39Mbps

AT&T DACSNETWORK

Multiple T1Connections

4

Voice & RTU (Point-to-Point) WAN Overview

Austin

Taylor

POWER RPS ACTIVITY

Cisco 2600 SER IES

R

Rtr-B

Rtr-B

SWX-BSWX-

Poi

nt-t

o-P

oint

Channel BankDS0 13-24

DS0 1-12

Hotline Phones

Taylor RTU

AT&T Network

AustinRTU

Austin

Taylor

Channel Bank

DACS

4 T1s

TaylorVoice

Only (OPX &

Hotline)

Greenwood Tennyson

18 T1sDACS

8 T1s

Voice & RTU Redundancy

Modem Bank

A

B

OPX Phones

Channel Bank Spectel

Hotline

Modem Bank

A

B

Visual NetworksCSU/DSU

Data & RTU

Voi

ce &

RT

U

Nortel PBX (4-digit)

IPCPBX

(Trader)

Spectel Hotline

Nortel PBX (4-digit)

IPCPBX

(Trader)

ERCOT Austin ERCOT Taylor

5

ERCOT Internet Presence

Highly secure

– Encrypted data communications

– Identity validation via digital certificates

– Layered security architecture

– Protected DMZ

Highly reliable & highly redundant

– Multiple layers of hardware redundancy

– Adequate bandwidth

– Multiple DNS servers

– Redundant, diverse, carrier connections

– Private IP space

6

Internet Design Overview

ERCOT AUSTIN

ERCOT TAYLOR

Cisco 7200 Series VXR

3

1

0

2

4

Cisco 7200 Series VXR

3

1

0

2

4

Cisco 7200 Series VXR

3

1

0

2

4

Cisco 7200 Series VXR

3

1

0

2

4

OC-3

DWDM

ISP-A ISP-B

Internet

Ethernet Ethernet

DMZ

7

ICCP

ICCP Stats (An example of a Market Participant)• Currently runs at approx 128Kbps• Nodal testing will increase their ICCP load by 100%• Anticipate roughly 256Kbps of ICCP load. • This is 25% of circuit capacity

Capacity Calculation• Total reliable* capacity is 1024Kbps

– Equation as follows• Point-to-Point T1 capacity = 24DS0 x 64Kbps each = 1,536Kbps• Must reserve 8 DS0 (512Kbps) channels for voice• 1,536 – 512 = 1024

* Reliable capacity is defined by amount of available bandwidth on Point-to-Point WAN. The MPLS network is capable of 1.544Mbps; however, we can only guarantee reliability of up to 1024Kbps due to requirement of dynamic failover to Point-to-Point network in the event of an MPLS network failure.

8

Approach to EDS

• We do not want to impact Zonal production traffic with Nodal test traffic.

• All non-ICCP EDS traffic will transit the Internet.

• Once trending and analysis are complete, a determination will be made regarding availability of web services over the WAN. We hope to have statistical analysis complete by March 2008.

• The expected outcome of the analysis is that most, if not all, non-grid critical traffic will be moved to the Internet.