Retail SLA Proposed Changes

RMS/TDTWG August 2008

Trey FeltonIT Account Manager

Weekend Transaction Volume vs. Current/Proposed Outage Windows

12

36

12 12

18 18

12

36

0

5

10

15

20

25

30

35

40

1 2 3 4

Week of Month

Ho

urs

0

100,000

200,000

300,000

400,000

500,000

600,000

700,000

800,000

To

tal

Tra

nsa

ctio

ns

Current Outage Window

Proposed Outage Window

Transactions: Sat - Sun (48-hr)

Transactions: 8am Sat - 8pm Sun(36-hr)

Transactions: 12pm Sat - 12amMon (36-hr)

Apr – Jun 2008

Sample Data

TRANS NAME COUNT(*)

TRANS NAME COUNT(*)

TRANS NAME COUNT(*)

TRANS NAME COUNT(*)

810_02 - 810_02 - 810_02 - 810_02 - 814_01 861 814_01 1383 814_01 1,683 814_01 822 814_02 148 814_02 156 814_02 212 814_02 64 814_03 2,737 814_03 3072 814_03 4,067 814_03 1,330 814_04 1,677 814_04 1165 814_04 3,879 814_04 2,202 814_05 1,775 814_05 964 814_05 3,244 814_05 1,469 814_06 210 814_06 72 814_06 379 814_06 169 814_07 599 814_07 338 814_07 642 814_07 43 814_08 295 814_08 259 814_08 505 814_08 210 814_09 248 814_09 163 814_09 491 814_09 243 814_11 - 814_11 - 814_11 - 814_11 - 814_12 54 814_12 59 814_12 68 814_12 14 814_13 34 814_13 28 814_13 70 814_13 36 814_14 - 814_14 - 814_14 - 814_14 - 814_15 - 814_15 - 814_15 - 814_15 - 814_16 1,500 814_16 1296 814_16 1,890 814_16 390 814_17 136 814_17 134 814_17 186 814_17 50 814_18 4 814_18 4 814_18 4 814_18 - 814_19 4 814_19 4 814_19 4 814_19 - 814_20 6,271 814_20 4786 814_20 9,038 814_20 2,767 814_21 7,297 814_21 5576 814_21 9,024 814_21 1,727 814_22 18 814_22 6 814_22 25 814_22 7 814_23 25 814_23 6 814_23 25 814_23 - 814_24 2,105 814_24 1997 814_24 2,589 814_24 484 814_25 1,209 814_25 792 814_25 2,606 814_25 1,397 814_26 - 814_26 - 814_26 - 814_26 - 814_27 - 814_27 - 814_27 - 814_27 - 814_28 268 814_28 167 814_28 459 814_28 191 814_29 400 814_29 173 814_29 451 814_29 51 824_02 833 824_02 63 824_02 905 824_02 72 867_02 4,242 867_02 3092 867_02 5,814 867_02 1,572 867_03 281,855 867_03 15053 867_03 558,249 867_03 276,394 867_04 3,835 867_04 830 867_04 8,062 867_04 4,227

318,640 41,638 614,571 295,931

May 3 and 4, 2008

Saturday 8 a.m. to Sunday 8 p.m.

Saturday 12:01 a.m. to 8:00 a.m.and Sunday 8:00 p.m.

to midnight

In Period 48 Hour Period Outside Period

Saturday 12 a.m. to Sunday 12 p.m.

In Period

For complete data set see: Retail SLA_Transaction Stats.xls

Weekend Transaction Volume during Outage Window Timeframes (1-year average vs 3-month average)

0

100,000

200,000

300,000

400,000

500,000

1 2 3 4

Week

# o

f T

ra

ns

ac

tio

ns

1-year Average (8 to 8)

3-month Avg (8 to 8)

3-month Avg (12 to 12)

Current Release Window

Proposed Release Window

Summary

• SLA Proposed Changes– Add 6 hours to Week 1

• Currently 12 hours

– Move 36-hr release window to Week 4 • Currently in Week 2—busiest week

– Add a 12-hour outage window in Week 5s (Mar/May/Aug/Nov)• Not in current SLA

• November will not be used for a release or week 5 outage

• Rationale– 5-week Months become general outline for major releases

(Mar/May/Aug/Dec)• Extra weekend gives additional outage window opportunity

– Remaining months will be reserved for minor releases

Additional Considerations

• For 2009– Move November Week 4 Outage to December Week 2

• To accommodate for Thanksgiving Holiday

– Move May Release up 1 week (to Week 3)• To accommodate for Memorial Day

– Analyze Quarterly Release concept

• For 2010– Reduce monthly minor release windows if possible

TDTWG Recommendations

• Limit Maintenance Window request to 15 hrs– Yes

• Reduce Release Outage to 30 hrs– Cannot Support

• Marketrak available 8am – Noon on Saturday– Researching, along with other data for MT Task Force

• Have 1 weekend a month with zero outages– With our current systems, ERCOT cannot support– Targeted for 2010

Proposed Release Calendar (2009)

Month Release DateJanuary 24/25February 21/22March 21/22April 25/26May 16/17June 27/28July 25/26August 22/23September 26/27October 24/25November N/ADecember 12/13

Proposed Maintenance/Release Windows for 2009

Week 1

Week 2

0

1200

0600

1800

2400

Mon ThursWedTue Fri Sat Sun

Availability – 99% 1

Maintenance

Week 3

Week 5

0

1200

0600

1800

2400

Mon ThursWedTue Fri Sat Sun

Availability – 99% 1

Maintenance

2

2

MAINTENANCE1st and 2nd Sunday– 6:00am until 9:00pm Sunday (15 hours)

Option 2: Sunday– 9:00am until 12:00am Monday (15 hours)3rd and 5th Sunday—8:00am until 8:00pm (12 hrs)

Option 2: Sunday—10:00am until 10:00pm (12 hrs)

RELEASE: 4th weekend– 12:00pm Saturday until 12:00am Monday (36 hours)Option 2: 10:00am Saturday until 10:00pm Sunday (36 hours)

Week 4

0

1200

0600

1800

2400

Mon ThursWedTue Fri Sat Sun

Availability – 99%

1

1

Release

2

2

Month Release DateJanuary 24/25February 21/22March 21/22April 25/26May 16/17June 27/28July 25/26August 22/23September 26/27October 24/25November N/ADecember 12/13

Actual Maintenance Outages (2008)

0

5

10

15

20

25

Jan (1) Jan (3) Feb (1) Mar (1) Mar (4) Apr (1) May(1)

May(3)

Jun (1) Jun (3) Jun (4)

Month (Wk)

Ho

urs

Hrs

Maintenance Outages (since Jan 1, 2008)

ERCOT Proposed SLA (18 hours)

Current SLA (12 hours)

Average = 9.8 hrs

40SQL Server DBs

70Linux servers

32Oracle DBs

449Windows servers

40SQL Server DBs

70Linux servers

32Oracle DBs

449Windows servers

TDTWG Proposed (15 hours)

0

5

10

15

20

25

30

35

40

45

50

Feb Aug Sep_2 Nov Jan Mar Jun

Current / Proposed SLA (36 hours)

TDTWG Proposed (30 hours)

Releases (2007-2008)

Average = 26.5 hrs

37% took more than 36 hrs

50% took more than 30 hrs

Marketrak

Marketrak Saturday Availability (2008)

0

20

40

60

80

100

120

2/16 3/8 3/29 4/19 5/10 5/31 6/21 7/12 8/2

Saturday Availability

Overall Availability

Backup Slides

Release Outage (apportioned time)

Start Time Sat 12:00pm

Shut down apps and clear queues 1.5 hrs

New Code Migration 4 - 6 hrs

Bring Systems Back Up / Tech Verification 1 hr

Production Verification 4 hrs

Rollback Contingency 1 - 4 hrs

Minimum Time Needed  11.5 – 16.5 hrs

Variations

• June Release: Planned 11 hrs, Actual 24.8 hrs (Prod verification failure; rollback)

• July Release: Planned 9 hrs, Actual 9 hrs

• Average Release has been 24 hrs (since Jan 1, 2008)

2008 Net Service Availability

99.00%

98.00%

97.00%

94.00%

95.00%

96.00%

93.00%

92.00% and below

100.0%

2008 Net Service Availability Through June 30, 2008

Transaction Processing TML MarkeTrak Frequency ControlRealtime Balancing Market

99.03% 98.79%99.14% 99.999%99.868%

Wholesale Total

94.730%

Transaction Processing SLA Target 99.9%

TML Availability Target99%

MarkeTrak Availability Target – 98%

Retail & Wholesale Operations System Operations

15 July, 2008 ERCOT Confidential

Releases (2008)

Actual Release Outages (2008)

0

5

10

15

20

25

30

35

40

Jan Feb Mar Apr May Jun Jul

Hrs Outage (Hrs)

Current / Proposed SLA (36 hours)

Average = 24 hrs

TDTWG Proposed (30 hours)