integrated systems market trials : market participant test

Integra ted Sys tems Market Tr ia l s :

Marke t Par t i c ipant Tes t Approach

April 6, 2015

IS Integrated Market Trials

Southwest Power Pool, Inc.

Integrated Systems Market Trials Market Participant Test Approach 1

Revision History

Date or Version Number

Author Change Description Comments

0.1 Chris Baldwin First Draft

0.2 Chris Baldwin Revisions throughout



0.5 Chris Baldwin Small revision to the Market Trials Test Preparation section



Table of Contents

Revision History ...........................................................................................................................................1

1.0 Overview of Integrated Systems Integration Market Trials..............................................................4

1.1 Objectives ........................................................................................................................................... 4

1.2 Approach ............................................................................................................................................ 5

2.0 Scope of Market Trials ..........................................................................................................................5

2.1 Components in Scope ......................................................................................................................... 5

2.2 Out of Scope ....................................................................................................................................... 8

3.0 IS Integration Testing Timeline ............................................................................................................8

4.0 Market Trials Test Preparation ...........................................................................................................9

4.1 Market Trials Test Preparation Details ......................................................................................9

4.1.1 Market Trials Test Documents ........................................................................................................ 9

4.1.2 Testing Liaisons ............................................................................................................................ 10

4.1.3 Market Trials Communication Meetings ...................................................................................... 10

5.0 MTE Market Trials Test Execution .....................................................................................................11

5.1 IS Integration Market Trials Test Execution Entry Criteria ....................................................11

5.1.1 SPP Entry Criteria ......................................................................................................................... 11

5.1.2 Market Participant Entry Criteria .................................................................................................. 11

5.2 MTE Market Trials Test Execution Details .............................................................................11

5.3 Market Trials Test Stages ................................................................................................................. 14

5.4 SPP Activities ................................................................................................................................... 15

5.5 Market Participant Activities ........................................................................................................... 16

5.5.1 Determine Appropriate Tests to Execute ...................................................................................... 16

5.5.2 Market Trials Calendar .................................................................................................................. 17

5.5.3 Request Troubleshooting Assistance for Failed Tests .................................................................. 19

6.0 Communication Plan .............................................................................................................................19

6.1 General Information and Inquiries ...........................................................................................19

6.2 Weekly Conference Calls ........................................................................................................19

6.3 Coordination with the SPP Change Working Group ...............................................................19

7.0 Readiness Metrics ..................................................................................................................................19

7.1 Introduction ..............................................................................................................................19

7.2 Readiness Metrics Detail .........................................................................................................20



7.3 Exit Criteria ..............................................................................................................................20

8.0 Assumptions............................................................................................................................................21

9.0 Risks ........................................................................................................................................................21

10.0 Supporting Documents ........................................................................................................................21

11.0 Acronyms ..............................................................................................................................................22



1.0 Overview of Integrated Systems Integration Market Trials

The purpose of this document is to outline the Integrated Systems (IS) Integration Market Trials approach for validating the technical and business readiness for the IS entities transitioning to SPP. This section of the document describes the objectives of Market Trials testing and describes the planned approach for meeting testing objectives.This document will outline the testing objectives along with the approach to accomplish those objectives. The overall management approach is included along with the expected resource involvement. All Transmission Customers (“TCs”), Market Participants (“MPs”) and Metering Agents (“MAs”), as well as Transmission Owners (“TOs”) and Transmission Operators (“TOPs”), participating in the transition, must take part in the testing. MP and SPP readiness will be assessed in the “Testing Scorecard” that will be maintained by SPP. The Testing Scorecard will track and monitor SPP system readiness, MP participation and progress, and outstanding issues identified during testing. This Market Trials effort will be conducted with the new Integrated System Market Participants only. SPP is not asking any existing Market Participants to participate. Therefore, any reference to Market Participants in the document refers only to the Integrated System Market Participants that will have load and/or generation balanced by SPP BA per 10/1/2015. Existing Market Participants may participate in Market Trials to the extent that participation does not disrupt testing for the new Integrated System Market Participants. Any results reflecting updated bids/offers, etc. will be made available to all Market Participants, existing and new. 1.1 Objectives The overall objective of Market Trials testing is to determine Market Participant and SPP readiness to cutover for balancing , interchange, tariff, market operations, and the associated settlements.

Market Trials will validate that SPP and Market Participant systems, processes, and staff are able to operate in alignment with Integrated Marketplace Protocols, through testing of specific business processes and system functions in a very controlled manner between SPP and Market Participants. Market Trials is conducted in SPP’s Member Test Environment (MTE). Furthermore, Market Trials is intended to provide Market Participants with an opportunity to test their end-to-end systems.

The specific objectives of Market Trials are as follows: Integrated Marketplace and Reliability:

1. Demonstrate that all key operational information is exchanged between parties. 2. Verify that power flows within the Integrated Marketplace are appropriately accounted for. 3. Ensure that SPP’s situational awareness tools are adequate to reflect operations of larger

reliability and market footprints. 4. Demonstrate that the IS entities can appropriately create transaction schedules subject to SPP

tariff, business practice and market protocol requirements. 5. Verify that scheduling components calculated by SPP are accurately reflected in the ACE

calculation. 6. Facilitate a market simulation for new MPs through which they can verify the processes and

procedures intended for use in production. Integrated Marketplace Settlements:

1. IS Meter Agents and Market Participants demonstrate the ability to submit meter data, Bilateral Settlements Schedules, and other settlements-related information to SPP according to prescribed timeframes.



2. Market Participants demonstrate the ability to download settlement statements, verify billing determinants, and verify invoices related to Integrated Marketplace settlements.

3. Market Participants verify that SPP has modeled their assets and commercial relationships consistently with what was registered

1.2 Approach

Market Trials testing will consist of the following:

• Base Case Runs • Supported Days • Unsupported Days • Deployment Tests • Regulation Qualification

The entire Market Trials period from May 18 thru July 17 will consist of running a Day Ahead Market and Reliability Unit Commitment introducing Operating Reserves, Virtual bids and offers (Virtual applies to Day-Ahead-only) and constraints with controlled input. The Real Time Balancing Market will be running and generating dispatch basepoints that will be communicated to the Automated Generation Control (AGC) applications which will then create the setpoints to be sent to the Market Participants. These setpoints will be informational only and the Market Participants will not be following these setpoints. Beginning May 11 to July 17, 2015 Meter Agents/Market Participants will be responsible for creating and submitting their own meter and bilateral settlement schedules.

Because the data that is input into the settlements system will improve throughout testing, the expectation is that the shadow settlement systems will not be able to be fully validated until May 25.

Beginning with May 25th, settlement results for each Operating Day cleared by the Market Clearing Engine (MCE) will be posted in order to test the MPs ability to process and download statements and invoices. The settlement results will be accurate, but may be based on unrealistic input data. During the month of June, SPP will test various scenarios with the Market Participants on the Supported Days. As much as possible, the scenarios will be constructed to impact both the Day-Ahead and Real-Time Balancing Markets to simulate production. Also in June, SPP will schedule a series of Deployment Tests to demonstrate Market Participants can follow SPP setpoints send out by SPP through ICCP. In addition, in June, SPP will schedule a series of Regulation Qualification Tests for Market Participants who wish to certify a resource as a Regulation Qualified Resource, Regulation-Up Qualified Resource or Regulation-Down Qualified Resource for regulation that has not previously provided regulation.

2.0 Scope of Market Trials 2.1 Components in Scope

• Market Trials will be conducted in SPP’s Member Test Environment (MTE).

• Centralized Modeling Tool (CMT) o The network and commercial models will be able to handle all items described in the Integrated

Marketplace Protocols. However, only registered Resources will be added to the models. For example, Block Demand Resources (BDR) are described in the Protocols but will not be included in the models unless a MP registers a BDR for use in the Integrated Marketplace.



• Energy Management System (EMS)

o In the Consolidated Balancing Authority (CBA) system, the EMS performs two main roles. The first role is to facilitate the transfer of ICCP data between the Market Systems and the member systems. The second role is to perform many of the normal requirements that AGC is expected to perform for a NERC BA. These include the following:

Process redundant SCADA telemetry data for use by AGC and RTBM Calculate Area Control Error (ACE) Calculate and deploy required Regulation to regulating units Deployment of required Contingency Reserve to available online units Support adjustment of dispatch targets to follow system generation requirements

when RTBM dispatch data is not available for a prolonged period Calculate and maintain data for NERC CPS/DCS/BAAL reporting Calculate and issue setpoint instructions Support monitoring of resource outputs vs. desired output MW requests Support monitoring of tieline data Support monitoring of reserves based on market cleared values as well as physical

unit capability

• Marketplace Portal o The Marketplace Portal is the interface through which Market Participants will access the various

Integrated Marketplace user interfaces. o The Marketplace Portal is required to be online for Market Trials in order to provide the Market

Participants a venue to access the Integrated Marketplace user interfaces. o Market Participants will be responsible for setting up their users through their Local Security

Administrator (LSA). o Market Participants will utilize the Marketplace Portal to access the Marketplace user interfaces

(including the Markets UI and TCR MUI) with the use of a digital certificate.

• Market Systems o Day-Before-Day-Ahead Reliability Unit Commitment (DBDA-RUC)

Day-Before Day-Ahead Reliability Unit Commitment runs prior to the Day-Ahead Market for the Operating Day. The commitment is dispatched by running a Security Constrained Economic Dispatch (SCED) for the Operating Day and Operating Day hourly injections are created for input into the Reserve Zone requirements calculation.

o Reserve Requirement Calculator The Reserve Requirements Calculator will calculate minimum and maximum reserve

requirements for each Reserve Zone (RZ) within the SPP BA and will also calculate the Reserve Obligations for each Market Participant. The calculated hourly Regulation & daily Contingency Reserve requirements are provided as an input into the Energy and Operating Reserve markets.

o Day-Ahead Market The Day-Ahead Market is a forward market in which energy and operating reserves are

cleared on a simultaneously co-optimized basis for each hour in the next Operating Day using a Security Constrained Unit Commitment (SCUC) followed by a Security Constrained Economic Dispatch (SCED). Energy and Operating Reserves are cleared to satisfy the energy demand bids and operating reserve requirements of the Day-Ahead Market subject to transmission and system constraints. The results of the Day-Ahead Market include:

• Hourly Locational Marginal Price (LMP) values for Energy demand and supply • Hourly Market Clearing Price (MCP) values for Regulation Up, Regulation Down,

Spinning reserve, and Supplemental reserve • Hourly energy demand schedules • Hourly import, export, & through transaction schedules • Hourly Virtual supply and demand schedules • Hourly energy supply schedules for each resource



• Hourly Regulation Up, Regulation Down, Spinning reserve, and Supplemental reserve schedules for each resource

o Day-Ahead Reliability Unit Commitment (DA-RUC) Day-Ahead Reliability Unit Commitment is run after the Day-Ahead Market to ensure that

there are sufficient resources committed to supply forecasted demand and Operating Reserve requirements.

o Intra-Day Reliability Unit Commitment (ID-RUC) Intra-Day Reliability Unit Commitment is run for the balance of the day to determine what

additional commitment requirements are necessitated by intra-day changes. Intra-Day Reliability Unit Commitment is run automatically every 4 hours starting at market

time 00:00. o Real-Time Balancing Market

The Real-time Balancing Market (RTBM) runs on a continuous five-minute basis. SPP clears the RTBM by determining the Security-Constrained Economic Dispatch (SCED) that co-optimizes the balancing of generation and load and meets Operating Reserve (OR) requirements within the SPP Balancing Authority Area. The RTBM is cleared based on actual conditions, forecasted conditions and submitted Offers. The network state information used in RTBM comes from the State Estimator (SE).

RTBM produces resource MW dispatch instructions, reserve assignments, and real-time Ex-Ante LMPs for Energy and MCPs for Reserve products. Resource dispatch instructions and reserve assignments are made available to the Automatic Generation Control (AGC) on the EMS side as resource base points and reserve obligations, respectively. The ExAnte prices, dispatch instructions and reserve clearing amounts are also used in the Settlements system.

• All Day-Ahead and Real-Time Balancing Market notifications

• CROW –Outage Scheduler o CROW is the system used for outage coordination. o We will be using the outages from the CROW production Environment. o Non-production (test) data should not be submitted to the Production CROW at any time. o Data: Any submitted, approved, or implemented outages will be used in current Production

systems as well as Integrated Marketplace systems during Market Trials.

• OATI/RTOSS o The SPP Integrated Marketplace systems will be connected to the OATI ITE environment during

Market Trials. SPP will work with Market Participants to create tags to test the interface and functionality of the new Integrated Marketplace tags. This will be accomplished by allowing Market Participants to submit through the OATI ITE environment during Market Trials.

• Reserve Sharing System (RSS)

o RSS will only be used during Market Trials for specific scenario testing.

• Ramp Reservation System (RRS) o The Ramp Reservation System Tool will be available for Market Trials. Market Participants (PSEs)

will access the system to 1) check capability available at any given time and 2) to create a ramp reservation.

• Credit Management System (CMS) – Not an MP Facing System

o CMS will be online for Market Trials in order to perform realtime credit management. o This CMS is a SPP internal system; MPs do not interact directly with the system. Credit positions

information is accessible via the Marketplace Portal. o Credit limit availability is assessed for each Virtual and TCR transaction (bid/offer) submitted for

clearing. If enough credit is available for a proposed transaction, that transaction will be considered credit approved and available for clearing. If there is not enough credit available to cover the transaction, that transaction will be credit disapproved and not considered for clearing. CMS does



not assess credit limit availability for day-ahead, real-time, or transmission service transactions prior to potential clearing.

o Data: A set of test data (Virtual Reference Prices and credit limits) will be needed for CMS to perform credit management.

Virtual Reference Prices will be calculated by SPP and distributed via the Marketplace Portal for all MP use.

• MPs are requested to submit a credit limit amount to SPP which will allow them to interact appropriately during Market Trials

• SPP may increase the value of both secured and unsecured credit in order to ensure successful completion of Market Trials and unstructured testing.

• MPs will not be expected to submit secured credit for Market Trials • MPs will be expected to submit secured collateral (if required) prior to any

production trading activities. • MPs should reference the SPP Cedit Policy, Attachment X, for additional

information concerning minimum participation criteria, forms of secured collateral, or other credit related policy questions.

• Settlements :

o Settlements is needed as a downstream system from the Market Systems. o Market settlements will be available for each operating day throughout Market Trials.

Market Participants/Meter Agents are requested to submit meter data and Bilateral Settlement Schedules daily that matches daily offers and normal production processing.

o Transmission settlements will test that each entity can receive information from the transmission settlements portal. Dummy files will be placed in each entities folder on the portal. The following Dummy files will be posted.

Daily XML Monthly XML Excel Summary files

• Market Monitoring Data Depository (MMDD):

The MMDD Mitigated Offer Data Submission application provides the Market Participant and Asset Owner the ability to transmit the productions costs associated with providing energy and operating reserves as well as their policies surrounding how they calculate their production costs. Production costs are the costs incurred to operate a generating resource for a specific period of time. Several different cost components are needed to determine a generating resource’s total production cost. These costs include:

• Start-Up Costs • No-load Costs • Incremental Costs (energy cost per segment of output range) • Operating Reseve Costs

2.2 Out of Scope Market Trials does not include any testing of Transmission Congestion Rights (TCR) and Auction Revenue Rights (ARRs), daily Transmission Settlements, Great Plains, Legacy Systems, and GIS.

3.0 IS Integration Testing Timeline The chart below displays the testing timeline for the IS Integration Project.



Figure 1

4.0 Market Trials Test Preparation This section provides details about preparation for Market Trials testing.

4.1 Market Trials Test Preparation Details Market Trials Test Preparation is from February 16 – May 11 and exercises the following activities. Details about each activity are provided in subsequent sections of this document.

Deadline Activity 4/6 SPP publishes IS Integration Market Trials Test Approach 4/17 SPP will conduct a Market Trials Kick-off Meeting 4/17 SPP schedules the weekly testing call to start on 5/1 and last until July 17 4/30 Connectivity Testing was successfully completed 4/30 Deadline for questions in response to the published Approach 5/1 1st IS Integration Market Trials weekly testing status Call

4.1.1 Market Trials Test Documents SPP will place the Integrated Systems Market Trials Test Approach and IS Integrated Deployment Test MP Guide out in a folder on SPP.org for your review. Market Participants should carefully review this documentation and provide any questions about either document via the IS Integration Test Approach RMS quick pick prior to May 8, 2015. These RMS tickets will remain open through May 8, when tickets that remain open on that date will close.

Integrated Systems Market Trials Test Approach The Integrated Systems Market Trials Test Approach (current document) provides the logistical plan and details to assist Market Participants in understanding their expectations and processes involved in MTE Market Trials.



IS Integrated Deployment Test MP Guide The IS Integrated Deployment Test MP Guide provides the logistical plan and details to assist Market Participants in understanding their expectations and processes involved in the IS Integrated Deployment Tests.

Regulation Pre-Certification How to Guide/SPP Pre-Cert Test Data Certified Doc The Regulation Pre-Certification How to Guide provides detail to Participants who want to qualify a resounce for Regulation in SPP’s market. The guide explains how Market Participants gain qualification by providing historical data of their Regulation. The Market Participant will collect and provide, via the SPP Pre-Cert Test Data_Certified template, Rate of Compliance and Regulation Mismatch data for every Registered Regulation Resource they wish to certify to Regulate in SPP’s Integrated MarketPlace. 4.1.2 Testing Liaisons A Market Participant Testing Liaison must be identified to serve as a point of contact for the Member organization responsible for performing the following functions. • Review Testing documentation and provide feedback • Assess milestones and deliverables for feasibility • Serve as the main point of contact between SPP and Member organization • Provide Network Market Trials resource • Review project status and provide your organization’s status • Coordinate Member Market Trials, Deployment, and Regulation Qualification testing efforts • Represents the Member’s interest on the Testing team • Attend Readiness/Market Trials Member Testing Meetings

o SPP Testing Team will schedule directly via Outlook with the registered liaisons o Agenda will be published in advance o Minutes will not be required after the meeting o Action items identified will be documented internally on the project internal site o As necessary, action item updates will be communicated back to the liaisons via e-mail or

subsequent meeting • MPs work with SPP to provide test data for SPP’s testing as needed

4.1.3 Market Trials Communication Meetings Weekly communication meetings provide a mechanism to collaboratively address Market Participant questions regarding Market Trials testing. SPP expects to schedule these meetings by April 17. The meetings will begin approximately May 1 and extending through July 17. Market Trials Communication meetings will be held weekly each Friday morning. The intent of these meetings is to review testing results of the week and inform Market Participants of upcoming action items. The meetings will be used as an open forum for Market Participants to follow up on any open questions. Testing Liaisons are expected to attend these meetings. During every deployment test SPP will open a call for the BA Operator, TOP, SPP RC, MP and SPP testing Staff. This call will be opened at least 1 hour prior to the start of deployment test and will stay open during the deployment test and up to 1 hour after ending test. The purpose of the open call is to allow for immediate communication of any issues experienced during the test and to initiate an early termination of the deployment test by any party on the Call.



5.0 MTE Market Trials Test Execution

Market Trials Test Execution is from May 18 through July 17 and exercises the following activities. Details about each activity are provided in subsequent sections of this document.

Deadline Activity 5/1 – 7/17 IS Integration Market Trials weekly testing status Call

5/11 – 7/17 IS Integration Market Participants to submit Production like data into SPPs MTE environment on a daily basis.

5/18 – 7/17 Market Participant MTE Market Trials Window

7/17 Market Trials Test Window Closes

5.1 IS Integration Market Trials Test Execution Entry Criteria This section provides details associated with entry criteria for the Market Trials Test Execution Test Phase. Criteria details are found in previous sections of this document.

5.1.1 SPP Entry Criteria • The full IS Model has been uploaded in MTE • Data for non-IS MPs has been seeded into MTE • Connectivity Testing in MTE has been successfully completed by each Market Participant • Market Trials Test Approach published • IS Integration Deployment Test Market Participant Guide published • Regulation Pre-Certification How to Guide published • Weekly testing meetings scheduled

5.1.2 Market Participant Entry Criteria • MPs have completed the Connectivity testing and have confirmed their ability to use Web

Services or the Markets User Interface to submit transactions necessary to support Market Trials including the ability to submit daily transactions to support both Day-Ahead and Real-Time Balancing Market activities.

• Market Participants involved in Market Trials should have completed all training CBTs. • Market Participants have also confirmed the setup of their Markets and Settlements XML

Listeners. • Market Participant Testing Liaisons identified and communicated to SPP. • Market Trials Approach received and receipt acknowledged. • SPP, Market Participants with Resources and TOPs have substantially completed the creation

and testing of new and critical ICCP points (including new resource data points and tie-line data points)

• Market Participants have reviewed Pre-requisite Checklist to prepare for Market Trials

5.2 MTE Market Trials Test Execution Details Market Trials Test Execution testing details describes the SPP and Market Participant activities for Market Trials related to Market Systems including Day-Ahead Market, Reliability Unit Commitment (Day-Ahead and Intra-Day), Real-Time Balancing Market (RTBM) and Generation Dispatch (RTBM). The objective of Market Trials Execution is to test the ability of the Market Participant and SPP to submit and receive a full set of transactions in support of the day-ahead and real-time activities, for SPP to execute the Market,



SPP to communicate the results to the market and confirm that ability of the Market Participants to respond to dispatch instructions. Protocol timelines will be used whenever possible when this functionality is executed. In addition, operational scenarios may be exercised to simulate events. Day-Ahead and Reliability Unit Commitment Day-Ahead Market and Reliability Unit Commitment will be executed daily in the MTE to generate awards and notifications. To support DA Market/RUC executions, Market Participants will provide the following market submission items when required and in accordance with the limits outlined in this document if applicable:

Day- Ahead Market: MPs with Load

• Market Participants should submit Demand Bids between OD-6 and 1100 CPT OD-1.

o Each Market Participant must offer sufficient Resources and Import Transactions to the Day-Ahead Market to cover their load and Firm exports plus Operating Reserve obligation to the extent the Resources are available (e.g. not on forced outage, planned outage or Reserve Shutdown). A Market Participant’s load for purposes of this section shall be equal to that Market Participant’s expected daily peak Resident Load.

o A Market Participant’s daily Operating Reserve obligation shall be equal to the sum of that Market Participant’s maximum daily Regulation-Up, Regulation-Down and Contingency Reserve obligation as calculated by SPP.

Day-Ahead Market: MPs with Resources

• Submit Resource Offers for each Resource for use in the DA Market starting seven days prior to the Operating Day. For the Day-Ahead Market, Resource Offers must include: (i) a Start-Up Offer, a No-Load Offer and an Energy Offer Curve for Resources qualified to provide Energy, (ii) a Regulation-Up Offer for Regulation-Up Qualified Resources, (iii) a Regulation-Down Offer for Regulation-Down Qualified Resources, (iv) a Spinning Reserve Offer for Spin Qualified Resources and (v) a Supplemental Reserve Offer for Supplemental Qualified Resources.

• Submit Mitigation Offers for all Resources. • Review cleared MW for consistency between energy offers and cleared LMP values at each

settlement location. • Make sure market-committed resources are within submitted notification times • Ensure all market-committed resources do not exceed maximum energy requirements and

meet minimum energy requirements or run times

Day-Ahead RUC: MPs with Resources

• Submit RTBM Resource Offers for each Resource for use in the RUC process. RTBM Resource Offers must include: (i) a Start-Up Offer, a No-Load Offer and an Energy Offer Curve for Resources qualified to provide Energy, (ii) a Regulation-Up Offer for Regulation-Up Qualified Resources, (iii) a Regulation-Down Offer for Regulation-Down Qualified Resources, (iv) a Spinning Reserve Offer for Spin Qualified Resources and (v) a Supplemental Reserve Offer for Supplemental Qualified Resources.

• Monitor for Start-up and Shut down notifications at 2000. (May or may not receive any) Intra-Day RUC: MPs with Resources

• Continue to make sure RTBM offers for current Operating Day are up-to-date. • Continue to Monitor for Start-up and Shut down notifications

All MPs



• Submit Dispatchable or Up to Transmission Usage Charge Schedules to the ITE RTOSS for testing.

o Dispatchable and Up to TUC transactions should be 10 MW or less and no more than 2 per hour.

DA Market notifications will be active per Protocols, when possible. SPP will make every effort to execute the DA Market on the Protocol timeline; however, there may be days wherein the DA Market results will be delayed. These delays will be reported to Market Participants via notifications as would be expected during Production. Real-Time Balancing Market SPP plans to execute RTBM persistently, except during communicated maintenance windows, to generate dispatch instructions and LMPs using market ICCP inputs from the SPP EMS and market submission inputs from the Markets UI/API. To support RTBM execution and reasonability analysis, Market Participants will provide all ICCP data feeds on a continuous basis and submit Resource Offers through the Markets UI/API (7 days a week). Real-Time Balancing Market Notifications will be active per SPP Market Protocols. It is expected that SPP will provide IT and operational support Monday-Friday from 8 AM – 5 PM unless it is a communicated systems upgrade period or SPP holiday. Market Participants are expected to review their dispatch instructions and cleared reserves for consistency with the reported LMPs, MCPs, resource parameters, control status and RTBM offers. Settlements Settlements will also be integrated during Market Trials. Whenever possible, specific scenarios will be tied to Settlements testing in order to provide a full view of the impacts of scenarios on settlements/billing. SPP plans to settle each operating day cleared by the Market Clearing Engine (MCE) beginning with May 18. An accelerated posting schedule will be used during Market Trials: Initial (OD+7), F (OD+14), R1 (OD+21), R2 (OD+28), etc. There will be a settlements calendar posted to the MTE portal that reflects the accelerated schedule and it will be updated throughout market trials when resettlements are scheduled. Settlement results for each Operating Day cleared by the Market Clearing Engine (MCE) will be posted in order to test the MPs ability to process and download statements and invoices. It is expected that the input data to the settlements process could be intermittent. The settlement results will be accurate, but may be based on unrealistic input data. Meter Agents/Market Participants will be responsible for creating and submitting their own meter and bilateral settlement schedules. Because the data that is input into the settlements system will improve throughout testing the expectation is that the shadow settlement systems will not be able to be fully validated.

Control Mode Status Accuracy

Resource Control Mode is a signal sent to SPP via ICCP for each Resource Asset. Resource Control Mode is also referred to as ICCP System Control Mode (CMOD). It is critical the Resource submit correct Telemetered Control Mode.

• All Resources on outage for off-line should submit a Telemetered Control Mode equal of “0”. • All Resources capable of providing contingency reserves, online and are dispatchable should

provide a telemetered Control Mode of “1” if they are not qualified to provide regulation. • All Resources capable of providing regulation, online and are dispatchable should provide a

telemetered Control Mode of “2”.



• All Resources that are coming on-line, going off-line or otherwise not able to follow dispatch instructions should provide a telemetered Control Mode of “3”.

Operational Scenarios SPP will run various operational scenarios on the Supported Days (Tuesday and Wednesday) beginning in June. These scenarios are defined below for Market Participants to review. Please note that SPP may need to adjust MP submissions to effectuate these scenarios.

5.3 Market Trials Test Stages

Market Trials Market Trials will consist of running a Day Ahead Market determining Day-ahead clearing of Energy, Operating Reserves, Virtual bids and offers (Virtual applies to Day-Ahead-only) and binding constraints with controlled input from May 18, 2015 to July 17, 2015. The DA Reliability Unit Commitment process and Intra-day RUC processes will also be executed to further commit resources for Real-time. The Real Time Balancing Market will be running and generating dispatch basepoints that will be communicated to the Automated Generation Control (AGC) applications which will then create the setpoints to be sent to the Market Participants. These setpoints will be informational only and the Market Participants will not be following these setpoints. Supported Operating Days Supported days are days on which SPP will answer questions regarding data submittal and official Operating Day results. Generally, supported hours are Wednesday and Thursday, 8:00 – 5:00. Support includes, but is not limited to the following: • Run Test Scenarios • Official operating days results through RMS • Other Market questions through RMS Unsupported Operating Days

Scenario Function/s SPP activity MP activity required OOME RTBM Manually OOME a unit Observe changes to

setpoint instruction and OOME value

Offer Curve subject to mitigation

DA Market Ensure offer fails mitigation tests

Submit offer that fails mitigation tests


RUC Ensure offer fails mitigation tests



RTBM Ensure offer mitigation fails tests


CRD Event RTBM Enable units from each MP with resource to have contingency reserves cleared for at least one resource and run a CRD event

Observe contingency deployment on resources with cleared contingency reserves

Commitment Notification

DA Market/RUC

Schedule Training Observe commitment notifications in Markets UI/API

Fall DST All Schedule a table top exercise



MP’s may continue to submit data on Unsupported Operating Days and Market solution results will be made available. On any day that is not an official Operating day, SPP may not be available to answer questions regarding solutions. On non-official Operating Days, SPP may overwrite the offer data, run test scenarios, or not approve solutions. Deployment Tests The IS Integration Deployment Tests are designed to demonstrate Real-Time Market Operations that result in SPP control of Registered Resources across the Market Footprint. Market Participants will provide the applicable Real-time Resource Offers (consisting of Start-Up Offer, No-Load Offer, Energy Offer Curve, Regulation-Up Offer, Regulation-Down Offer, Spinning Reserve Offer and Supplemental Reserve Offer) that will shape the dispatchable range of Resources during testing. The goal of each Deployment Test is for IS Market Participants to demonstrate to SPP that they can follow set points send out through ICCP. Regulation Qualification Test There are specific testing requirements for a Resource to become a Regulation Qualified Resource, Regulation-Up Qualified Resource or Regulation-Down Qualified Resource within SPP’s Integrated Marketplace.

If a Market Participant wants to bring a resounce into SPP’s market that is currently providing regulation outside SPP’s Market, they gain qualification by providing historical data of their regulation. The Market Participant will collect and provide, via the SPP Pre-Cert Test Data_Certified template, Rate of Compliance and Regulation Mismatch data for every Registered Regulation Resource they wish to certify to Regulate in SPP’s Integrated MarketPlace. For more details refer to the Regulation Pre-Certification How to Guide.

If a Market Participant wishes to certify a resource as a Regulation Qualified Resource, Regulation-Up Qualified Resource or Regulation-Down Qualified Resource for Regulation that has not previously provided Regulation than the Market Participant resource will need to pass a Regulation Test with a Score of 75% or above. This will need to be completed for each resource the Market Participant wishes to certify for Regulation.

5.4 SPP Activities

During Market Trials, SPP will:

• Execute DA Market, DA-RUC daily • Execute ID-RUC periodically • Execute and monitor Real-Time system components, including RTBM/AGC, State Estimator and all

Real-Time security applications • Execute RTBM every 5 minutes using Market Participant data from ICCP and Portal submissions • Execute all EMS 2.5 Production studies, RTNET, RTRFCALC, RTCA, Losses, etc. • Execute all rolling Forecasting applications, Wind, Parallel Loop Flow, Mid-Term, & Short-Term Load

Forecasts. • Verify MP submission activities • Work with Market Participants to resolve issues • Generate setpoints, LMPs, MCPs and Shadow Prices • Execute AGC every 4 seconds • Communicate Real-Time reports and outputs • Update Counterparty credit limits with self-selected credit limits submitted by MPs



• Generate Awards, Day-Ahead LMPs/MCPs, and other reports • Communicate DA Market and RUC reports and outputs to the market • Organize and direct scenarios as defined by SPP and MPs • Produce Meter Data reports • Produce Settlement Statements, Invoices • Execute Deployment Tests to deploy generation from the Integrated Marketplace systems.

5.5 Market Participant Activities During Market Trials Test Execution, Market Participants will be performing the following activities:

• Request troubleshooting assistance for any failed tests via RMS • Submit completed results once all applicable tests are passed. • Attend weekly MTE Communication Meetings • Use the Markets User Interface (UI) and Web Services (APIs) to submit, update, and cancel DA

Market submission items, while keeping within the limits noted in this document. o Resource Offers (including Start up and No Load Costs) o OR Offer o Demand Bids o Virtual Offers and Bids

• Verify ability to receive Resource Setpoints via ICCP & XML • Submit Interchange Transactions • Report non-conforming Loads. • Create and submit meter data for each day • Download and review pre-DA Market and post-DA Market Report postings • Download and review RUC postings, as applicable • Download and review Real-Time postings • Submit self-selected credit limit to SPP, if desired, as often as weekly • Participate in scenarios as defined by SPP and MPs • Create and Submit Bilateral Settlement Schedules • Download and review Settlement Statements and invoices • Download and view Meter Agent reports from Portal. • Federal Service Exemptions (FSE) WAUE Only • During Market Trials RTBM submissions should strive to be in closer tolerance.

o Update a unit’s min/max capacity if it changes by >10% or 5 MW, whichever is greater. o Update a unit’s offer curve if it changes by >10% or $2 per 5 MWs, whichever is greater.

The timeframe for Market Trials Test Execution begins May 18 and extends through July 17.

5.5.1 Determine Appropriate Tests to Execute The Market Trials Test Checklist is separated into several sections based on MP functionalities and resource types. Market Participants need to determine which of the following sections are applicable for their company:

• Markets UI/API

o Market Participant Financial Only (Virtuals) o Market Participant with Load Assets o Market Participants with Resources

• Settlements UI/API o General Settlements Market Participants (All MPs) o Meter Agents

• RTOSS



o Submit Dispatchable or Up to (TUC)Transmission Usage Charge Schedules to the ITE RTOSS. Dispatchable and Up to TUC transactions should be 10 MW or less and no more than 2 per hour.

• XML Notification Listeners o Market Energy Notifications

Market Virtual Notifications Market Energy Notifications Market Reserve Notifications

o Settlements Notifications Settlements Notify Event

Tests contained within the sections above are deemed critical operations for each particular type of functionality.

5.5.2 Market Trials Calendar SPP will produce a Market Trials Calendar for Market Participants and SPP to follow during the Market Trials timeframe. The calendar which will be sererated out by month (May, June, & July). The calendar will be a living document and will be updated periodically as we proceed through the testing timeframe. The calendar will track the different testing stages and events that will make up Market trials as a whole. Market Trials is made up of several different test stages;

• Market Trials which is the daily submission of data including Markets Settlement data including Meter Data and Bilateral Schedules

• Supported Operating Days • Deployment Tests • Regulation Qualification Test

The Market Trials calendar also will be used to communicate specific events that are important to Market Trials. The events listed on the calendar will include:

• SPP Company Holidays • MP Testing Status Conference Call • Go/No Go check points • Preparation for the different tests



Below is a draft of what the calendar will look like.



5.5.3 Request Troubleshooting Assistance for Failed Tests For any errors or unexpected results obtained during execution of the tests, Market Participants should request assistance via RMS. Using the IS Market Trials Test Execution Questions quick pick, Market Participants should provide all error messages and execution steps leading up to the obtained error. IT Applications staff will work with the Market Participant via RMS to assist in resolving the specified error. During test execution, RMS issues/requests will follow standard RMS processes and SLAs.

6.0 Communication Plan This section describes communication methods in which Market Participants and SPP will interact throughout IS Integration Market Trials.

6.1 General Information and Inquiries All Inquiries should be submitted via the Request Management System (RMS) using the following quick picks. IS Integration Test Approach: Use for questions regarding testing logistics and documentation. IS Integration Market Trials Test Execution: Use for questions regarding specific applications during test execution and/or issues that may arise during test execution that require SPP’s assistance for triage/resolution. For mass or immediate communications that need to be sent from SPP to Market Participant, outreach will be via e-mail to IS Integration Testing Liaisons. All other SPP communications to specific Market Participants will be sent to IS Integration Testing Liaisons via RMS.

6.2 Weekly Conference Calls Communication of status, updates, and general information will occur on weekly IS Integration Market Trials Test Communication Calls that occur every Friday from May 1 through July 24 as needed. Attendance to these calls should include IS Integration Testing Liaisons or their proxy. Information on how and when to attend these sessions will be distributed via e-mail to IS Integration Testing Liaisons.

6.3 Coordination with the SPP Change Working Group The SPP Change Working Group (“CWG”) will be apprised of the status of testing efforts related to the IS Integration Project. They will be given an opportunity to raise concerns and issues with the risks that this transition has on their normal operations, and SPP staff will coordinate the necessary actions to address their issues.

7.0 Readiness Metrics 7.1 Introduction The IS Integration Readiness Metrics will measure, monitor, and report on the overall readiness of the IS Integration project. The IS Integration Readiness Metrics help determine, monitor, and measure the milestones and activities (both internally and externally) needed to ensure a successful on-boarding of Market Participants to the Integrated Marketplace. Readiness Metrics will be reported both internally and externally to ensure organizational and program readiness prior to the October 1st Go Live date. If an identified readiness area is not achieving desired results



or is falling behind schedule, the Readiness Metrics will help identify what mitigating actions are required to return the project back on track. The goals of the Readiness Metrics are to:

• Provide clear and consistent reporting to both internal (SPP) and external (Market Participants)

audiences • Monitor readiness areas over the lifecycle of the project through measureable criteria • Indicate if an identified readiness area is not achieving desired results

The Readiness Metrics are intended to be a subset of the many factors considered in the go/no-go criteria for IS Integration.

7.2 Readiness Metrics Detail 1. Verify Market Participant participation in Market Trials.

• This metric measures MP participation in Market Trials. Participation will be measured based on execution in the market. The following measurement criteria will be used to determine MP participation in Market Trials.

• Resource Owning MPs: o Maintain offers for each operating day o Resource-Owning MPs need to submit an offer in the DA Market and RTBM

• Load Owning MPs: o Submit market energy bids based on load forecasts for each operating day o Load-Owning MPs need to submit a bid in the DA Market

• Federal Service Exemptions (FSE): o

2. Market Participants with Regulation resources collaborate with SPP to complete Regulation pre-

certification process. • This metric measures the completion of the Regulation pre-certification for MPs with Regulation

resources. The following milestones will be used to track this metric: o MPs notify SPP of resources that need to be deployment tested o MPs submit data for Regulation pre-certification o Conduct Re-testing for identified resources o All MP identified Regulation resources have completed the Regulation Pre-Certification

3. ICCP Models are up-to-date and validated

• This metric will measure whether member ICCP Models are up to date, and the data exchange and values have been validated for all applicable entities and relationships

o 100% of resource-owning Market Participants ICCP models are up-to-date and validated o 100% of the tie-line measurements have been verified for applicable Transmission

Operators of Settlement Areas being integrated into the SPP Balancing Authority 4. Market Participants acknowledge their readiness as of 7/17/2015 for the Integrated Marketplace by

returning a completed/signed Certification Requirements form. • This metric measures MP acknowledgment of Integrated Marketplace readiness by tracking the

MPs that have signed the Certification Requirements form as of 7/17/2015. The following milestones will be used to track this metric:

o SPP provides MPs via RMS the Certification Requirements Form o MP returns completed/signed Certification Requirements Form to SPP via RMS o By signing the Readiness Certification form, MPs acknowledge that as of 7/17/2015, they

anticipate to be ready for the IS Integration go-live on October1, 2015

7.3 Exit Criteria Prior to exiting Market Trials, the following criteria should be met:



Exit Criteria for testing will be developed. Some of these criteria may include: acceptable level of market readiness as a whole, number of MPs meeting readiness criteria, and acceptable numbers of open RMS tickets. These criteria will be developed jointly by SPP and stakeholders in the testing process through the IS Integration transition team.

8.0 Assumptions The following assumptions have been made in relation to Market Trials. ID Description Impact to the Test 1 Resources will be made available to support

test prep, execution, and issue resolution Unavailable Resources for test prep may prevent SPP from completing the entry criteria and delay the start of testing.

2 All components will be available for Market Trials

Unavailability of any critical components would prevent the ability to begin Market Trials

3 All non-critical components will be available for Market Trials

Unavailability of any other components could prevent SPP from completing the entry criteria; however depending on the component, Market Trials may be able to move forward.

4 Market Participants planning to execute Web Services have completed applicable internal testing and Connectivity Test. SPP is expecting Market Participants to confirm that their internal systems that are planned to be utilized for Market Trials are functional and ready for Market Trials prior to beginning testing with SPP.

Untested Market Participant systems may lead to high numbers of defects on the Market Participant side. This may increase the amount of retest required and extend the duration of testing.

5 SPP may substitute any missing upstream data that is required in order to post settlement statements

Substituting data may result in unrealistic, but accurate, charge amount outputs. RNU, in particular, may produce unrealistic amounts.

9.0 Risks The following risks apply to Market Trials. ID Description Severity Impact to the Test 1 Functional Integration

Testing/Delivery Schedule High Unavailability of critical components would

prevent SPP from completing the entry criteria and delay the start of testing.

2 Market Participant readiness High Unready Market Participants may delay the start of testing or extend the duration.

3 Mid-Year ARR Allocation High Mid-Year ARR Allocation may need the model to migrate to Production prior to Market Trials completeing.

4

10.0 Supporting Documents The following table describes the key Supporting Documents specific to the Program. Document Name Document Location Request Management System Training Tool

Request Management System

Integrated Marketplace Connectivity Specification

Integrated Marketplace Environments

Portal Application Role Marketplace User Interface Guides

http://www.spp.org/publications/RMS%20Training%20Tool.pdf

http://www.spp.org/section.asp?group=3070&pageID=27




Reference Guide LSA User Guide Marketplace User Interface Guides LSA Portal Job Aid Marketplace User Interface Guides Settlements UI Guide Marketplace User Interface Guides Settlements Reference Tools

Settlements Reference Tools

Data Exchange Guides: Output Reports, Bilateral Schedules, Determinant Report, Meter Data

Marketplace Technical Specifications

Notification DEG and WSDL

- SPP External Notification API Manifest

- Notification – Data Exchange Document

- WSDLs

XSDs


Sample Settlement Reports Marketplace Technical Specifications LMS Information Learning Management System Integrated Marketplace Markets User Interface Guide

Marketplace User Interface Guides

Integrated Marketplace Market User Interface Business Validations

Marketplace User Interface Guides

Marketplace Markets Web Services

- Markets API XSD Manifest

- Markets API Data Exchange Guide

- WSDLs

XSDs


TCR API Specifications - Nexant API

Documentation Manifest

- API Definitions for ARR Nomination

- API Definitions for Download

- SPP TCR API XML Reference

- WSDLs

XSDs


SPP TCR MUI User Manual Marketplace User Interface Guides

11.0 Acronyms The following table describes the key acronyms specific to the Program








https://spp.csod.com/client/spp/default2.aspx








Acronym Term ACE Area Control Error AGC Automatic Generation Control BDR Block Demand Resource BA Balancing Authority CBA Consolidated Balancing Authority CC Combined Cycle Resource CMS Credit Management System CMT Centralized Modeling Tool COP Current Operating Plan CRD Contingency Reserve Deployment CROW Control Room Operating Window DA-RUC Day-Ahead RUC DBDA-RUC Day Before Day-Ahead RUC DDR Dispatchable Demand Resource EAD Enterprise Analytic Data Store EIS Energy Imbalance Service EMS Energy Management System IDC Interchange Distribution Calculator ID-RUC Intra-Day RUC IPP Independent Power Producer JOU Jointly Owned Unit LIP Locational Imbalance Price LMP Locational Marginal Price LSA Local System Administrator MCE Market Clearing Engine MCP Market Clearing Price MMU Market Monitoring Unit MOI Market Operator Interface NSI Net Scheduled Interchange OD Operating Day OH Operating Hour OR Operating Reserve PFC Parallel Flow Calculator PSE Purchasing and Selling Entity RCIS Reliability Coordinator Information System RMS Request Management System RRS Ramp Reservation System RSS Reserve Sharing System RTBM Real-Time Balancing Market RUC Reliability Unit Commitment RZ Reserve Zone SE State Estimator SCED Security Constrained Economic Dispatch SCUC Security Constrained Unit Commitment TCR Transmission Congestion Rights UI User Interface VER Variable Energy Resource VRL Violation Relaxation Limit