Test Approach

Smart Metering Implementation Programme (SMIP)Interface Testing Approach Document

Version 1.0

03/10/2014

Document LocationThe source of this printed document can be found in Smart DCCs Programme Management Office, or with the author if in an unapproved status.Associated DocumentsThis document is associated with the following other documents:RefTitle & Originators ReferenceSourceRelease DateVersion

[1]Glossary of Testing TermsISTQBOct 2012v2.2

[2]Joint DSP/CSP Test StrategyDSPFeb 2014v2.3

[3]Smart Energy Code Stage 3DECC

[4]DSP ContractsDCCv .1

[5]CSP Contracts (North, Central and South) DCCv .1

[6]Device Selection MethodologyDCCConsultation publication

[7]CTSD/SREPTSD (Common Test Scenarios Document/SMKI and Repository Entry Process Test Scenarios Document)DCCConsultation publication

[8]Guide for Testing ParticipantsDCCNot published

[9]SEC Definitions (Section A1)DECCSEC3

[10]Code of Connection for DCC User Gateway InterfaceDSPAug 2014v0.9

[11]Code of Connection for Self Service InterfaceDSPJan 2014v0.13

[12]Testing Issue Resolution and Disputes ProcessDCCNot published

[13]DCC User Gateway Interface Specification (DUGIS)DSPSept 2014V0.8

ContentsTable of Figures71Introduction81.1Context81.2Change Forecast101.3Approvals and Appeals Process101.4Terminology112Scope142.1Overview142.2Out of Scope143Objectives154Test Stage Description164.1Introduction164.2DSP-led testing164.2.1DCC User Message Gateway174.2.2DCC Self Service Interface174.2.3Additional Functionality Tests174.2.4CSP Regions174.2.5Non-Functional Testing174.3UEPT185Deliverables195.1Responsibilities for Deliverables195.2Notes on Deliverables196Test Procedure216.1Requirements traceability216.2Test Stage overlap216.2.1SP UAT Test Stage with Interface Test Stage216.2.2Solution Test Stage with Interface Test Stage216.2.3Interface Test Stage with End to End Test Stage216.3Technical Readiness216.3.1Service Providers216.3.2SEC Parties226.3.3RDPs226.4Testing Principles226.4.1Non-discrimination226.4.2Use of Devices/Device stubs226.4.3Testing with one CSP236.4.4Testing by Region236.5Release management236.6Configuration management236.7Timetable246.7.1IT Test Initiation246.7.2IT Test Execution276.7.3IT Test Completion286.8Dependencies306.9Stage Entry and Exit316.9.1Entry Criteria316.9.2Exit Criteria317Environments, Networks and Test Labs337.1Introduction337.2DCC Enterprise environments337.3DSP environment and network337.4CSP environments and networks337.5TSP environment337.6Parse & Correlate337.7SEC Parties337.8CSP Test Labs (for devices)347.9De-selection and substitution of Devices357.10Summary of test facilities provided368Test Data378.1Test Data Plan378.2Test data principles378.3Responsibilities for data provision399Testing Issue Management409.1Principles409.2Testing Issue Management tool419.3Issue Review Board419.4Testing Issue Severities and Priorities429.5Testing Issues and exiting IT439.6Testing Issue reporting4310Test Tracking & Reporting4510.1Introduction4510.2Test Preparation Reporting4510.2.1General4510.2.2Test Preparation Progress Report4510.2.3Test Readiness Report4610.3Test Execution Reporting4611Roles & Responsibilities4811.1General4811.2Roles and Responsibilities4811.2.1Roles of Test Participants4811.2.2Reporting structure5111.2.3Roles of Test Management5211.3Governance5211.4Communication5311.4.1Progress meetings5311.4.2Testing Issues and their resolution5411.5Test Assurance5411.5.1Introduction5411.5.2Quality Gate Reviews5411.5.3Test Witnessing5511.5.4Test Quality Audits5611.5.5Product inspections5611.5.6Documentation review5612Operational Acceptance Testing57Appendices58A.Test Scenarios59i.Connectivity Test Scenarios59ii.Additional Functionality Test Scenarios61Change of Supplier (On-Demand) Test Scenario for EIS SEC Party Role62Change of Supplier (On-Demand) Test Scenario for GIS SEC Party Role64Self Service Interface Test Scenarios66Billing Test Scenario69Reporting Test Scenario70Volume Test Scenarios71B.Documentation RTM73C.Functional Coverage RTM74Table of FiguresFigure 1 DCC Solution8Figure 2 Programme Test Documentation Hierarchy9Figure 3 Interface Test Documentation Hierarchy10Figure 4 - Abbreviations13Figure 5 DCC system16Figure 6 RACI Matrix showing Deliverables19Figure 7 Timetable for test initiation27Figure 8 Timetable for test execution28Figure 9 Timetable for test completion29Figure 10 Milestones and their dependencies31Figure 11 Number of device sets34Figure 12 Summary of test facilities provided36Figure 13 High-level data types and responsibilities for provision39Figure 14 - Testing Issue Priorities42Figure 15 - Target testing issue response times43Figure 16 Outstanding testing issues and exiting IT43Figure 17 Test Preparation Reporting45Figure 18 Test Execution Reporting46Figure 19 Roles of participating Test Participants50Figure 20 IT main roles and structure51Figure 21 Roles of Test Management52Figure 22 - IT Governance53Figure 23 Test Stage Certificates55Figure 24 Document RTM73Figure 25 Functional coverage RTM74IntroductionContextThis document sets out the manner in which Interface Testing (IT) will be conducted for the Smart Metering eco-system, which is depicted in the following diagram. Readers are expected to be familiar with the Joint DSP/CSP Test Strategy[footnoteRef:2] (Reference [2]). [2: This document will be update in accordance with the requirements that are set out in SEC3. ]

Figure 1 DCC SolutionThis Interface Testing Approach is based on:the Smart Energy Code (SEC) (Reference [3])the Joint DSP/CSP Test Strategy (Reference [2])the DSP and CSP Service Provider Contracts (References [4], [5])a series of workshops held at the DCC during July September, attended by Service Users, Service Providers, DECC, SECAS and Energy UK.Evidence of this documents compliance with the Smart Energy Code can be found in Appendix A.The diagram below demonstrates how this Test Approach (enclosed within the red oval) fits in with the overall hierarchy of test documents that will be produced for the programme.

Figure 2 Programme Test Documentation Hierarchy

The following diagram illustrates the documentation set for Interface Testing. Note that the documents in the diagram also apply to E2E Testing and Enduring Testing.

Figure 3 Interface Test Documentation HierarchyChange ForecastThis document will be assessed and, where applicable, updated by the DCC when approved versions of the following documents are available:Device Selection Methodology [6] (due mid-November);Common Test Scenarios Document/SMKI and Repository Test Scenarios Document (CTSD/SREPTSD) [7] (due end November);Testing Issue Resolution and Disputes Process [12]; andfurther versions of the Smart Energy Code, specifically SEC4.

Approvals and Appeals ProcessFollowing consultation, the DCC will submit the document to the TAG for review prior to submission to the SEC Panel for approval. This document will be presented to the SEC Panel in timescales such that it can be published at least 6 months prior to the start of the Interface Testing stage.If the SEC Panel decides not to approve the IT Approach and the DCC is required to do re-work, it will follow the process laid out in SEC T3.11.If the SEC Panel decides to approve the IT Approach, the DCC will follow the process laid out in SEC T3.12.TerminologyIn this document the term Service Provider includes all of the following:the DSP;both CSPs;the Trusted Service Provider (TSP), supplier of the SMKI solution element; andthe DCC Enterprise Service Provider (DCC Enterprise), i.e. the DCC in its role as supplier of Enterprise systems such as Billing and BI/MI.The term User Integration Testing (UIT) refers to the test phase that comprises Interface Testing and End to End Testing.The term Test Stub means systems and actions which simulate the behaviour of Devices and User systems. The term Testing Issue means in respect of any tests (a) anything that is preventing the execution of the test or (b) once commenced or executed, the test has an unexpected or unexplained outcome or response.The term Registration Data Provider (RDP) means (a) in respect of each Electricity Distributor, the person nominated in writing to the DCC from time to time by that Electricity Distributor; or(b) in respect of each Gas Transporter, the person nominated in writing to the DCC from time to time by that Gas Transporter,on the basis that more than one Party may specify the same Registration Data Provider, and that the Electricity Distributor or the Gas Transporter shall be deemed to have so nominated itself in the absence of any other nomination.The term RDP Systems means any Systems: a) which are operated by or on behalf of an Electricity Distributor or Gas Transporter responsible for providing (or procuring the provision of) Registration Data in respect of a particular MPAN or MPRN; andb) which are used wholly or partly for the collection, storage, Back-Up, processing or communication of that Registration Data prior to, or for the purposes of, its provision to the DCC over the Registration Data Interface.This document uses standard testing terminology, a glossary (Reference [1]) of which can be found on the International Software Testing Qualification Board website, www.istqb.org.Abbreviations used in this document are listed in the following table.

AbbreviationsFull Title

BIBusiness Intelligence

CIConfiguration Item

CoSChange of Supplier

CPACommercial Product Assurance

CSPCommunication Service Provider

CTSD/SREPTSDCommon Test Scenarios Document/SMKI and Repository Test Scenarios Document

DABDesign Assurance Board

DCCData Communications Company

DCC KIDCC Key Infrastructure

DECCDepartment of Energy and Climate Change

DLMSDevice Language Message Specification

DSPData Service Provider

DSMDevice Selection Methodology

DUGISDCC User Gateway Interface Specification

E2ETEnd to End Testing

GBCSGreat Britain Companion Specification

HANHome Area Network

IHDIn Home Display

iGTIndependent Gas Transporter

IRBIssue Review Board (for Testing Issues)

ITInterface Testing

MIManagement Information

MPRSMeter Point Registration System

OATOperational Acceptance Testing

PITPre-Integration Testing

RDPRegistration Data Provider

RTMRequirements Traceability Matrix

SECSmart Energy Code

SITSystems Integration Testing

SMKISmart Meter Key Infrastructure

SPService Provider

SREPTSMKI & Repository Entry Process Test

SSISelf Service Interface

SSMISelf Service Management Interface

S/WSoftware

TAGTest Advisory Group

TBDGTechnical Business Design Group

TSPTrusted Service Provider

UATUser Acceptance Testing

UEPTUser Entry Process Test

UITUser Integration Testing

Figure 4 - Abbreviations

ScopeOverviewIn its role as Systems Integrator, the DSP will manage Interface Testing (IT) with support from SEC Parties, the Registration Data Providers (RDPs) and Service Providers (SPs).IT will verify that SEC Party, SP and RDP system elements integrate together to form a working system that meets the agreed functional and non-functional requirements defined in the SEC and the SP contracts. The system consists of the whole Smart Metering eco-system.IT will be undertaken on a CSP Region by CSP Region basis, with CSP Regions being tested in parallel where practicable. It will use the Devices the DCC successfully used as the basis of testing in SIT. Should Devices not be available, then Device Stubs will be used.Large Supplier Parties (and Network Parties if so required by the Secretary of State) are obliged to be ready to begin their User Entry Process Tests (UEPT) during IT. Other SEC Parties can also conduct their UEPT during IT.Out of ScopeThe following are outside the scope of IT:certification of Meter Device Models (energy Suppliers are responsible for ensuring that any meters they install at consumers premises are SMETS compliant, including a requirement that they are protocol certified);certification of Communications Hubs (the CSPs, in conjunction with their Comms Hub manufacturers, are responsible for this activity);testing of Meter Device Models, other than the interaction of those Devices that are installed in CSP Test Labs, for the purpose of conducting User Entry Process Tests, with the DCC solution (energy Suppliers are responsible for ensuring that any Meter Device Models that they install are interoperable with the DCC and are SMETS compliant);testing of the Home Area Network (HAN) other than its interaction with the DCC solution (it is the responsibility of energy Suppliers to ensure that a SMETS compliant HAN is established in each consumer premise);testing of Hand Held Terminals other than their interaction with the DCC solution (energy suppliers are responsible for ensuring that any Devices that they use are compliant with the relevant technical specifications);testing the inter-changeability[footnoteRef:3] of Devices connected to the Home Area Network (this is not a requirement under the provisions of the SEC); [3: The ability to exchange one device with another without affecting the original functionality]

testing of Service User Business Processes (Service Users can test their back office systems against the DCC during the End to End Test stage on a voluntary basis).ObjectivesThe objective of Interface Testing is to demonstrate that the DCC and the DCC Systems together with the Communications Hubs interoperate with User Systems in order that the DCC is capable of complying with its obligations under Sections E (Registration Data), G (Security) and H (DCC Services) (in each case) at levels of activity commensurate with the relevant Volume Scenarios. This Test Approach document sets out the manner in which the IT objective will be achieved and includes the requirements that are set out in Section T3 of the SEC. Test Stage DescriptionIntroductionThere are two elements to Interface Testing:DSP-led testing of the interoperability of the DCC solution with SEC Parties systems; andSEC Parties undertaking their own User Entry Process Testing (UEPT).DSP-led testingThe primary focus of IT is on the dynamic interaction between the Service Provider and Service User systems. The key elements of the testing that will be undertaken are:the DCC User Message Gateway (item 1 on the diagram below), which was stubbed in PIT and SIT: integral to this is Parse & Correlate; the DCC Self Service Interface (item 2), which was tested in PIT and SIT by imitating the behaviour of Service Users over a local connection.The other External Interfaces that are shown in the following diagram will already have been verified during PIT and SIT. Regression testing of these interfaces will be conducted to the extent necessary as part of the Additional Functionality tests.

Figure 5 DCC systemTesting will be undertaken with Devices if Test Stubs have been used in SIT and if such Equipment is available in time for Interface Testing.DCC User Message GatewayThe conduct of UEPT by Parties will extensively test the DCC User Message Gateway as set out in the CTSD/SREPTSD.DCC Self Service InterfaceThe testing undertaken in PIT and SIT will be extended in IT by requiring SEC Parties to verify that the Self Service Interface works correctly for SEC Parties Identity Providers (used for authentication and access control).Additional Functionality TestsSEC Parties will be required to undertake tests as listed below:Change of SupplierBilling;Reporting;Self Service Interface; andVolume test.Further details are contained in Appendix A - Test Scenarios. The volume test requires the participation of all SEC Parties. CSP RegionsThe DSP-led testing will be performed on a CSP Region by CSP Region basis, so that achievement of the IT Objective can be demonstrated for each CSP Region separately. The technical solution for CSP Regions Central and South is identical[footnoteRef:4], and therefore these will be treated as one Region for purpose of testing, notwithstanding the fact that MPxNs from both geographical Regions will be included in the testing. [4: CSP Central/South (Telefonica) will be required to provide evidence to the DCC that the technical solution is the same in each region]

Non-Functional TestingPerformance TestingPerformance testing will be undertaken to the extent possible taking into account the constraints of the IT test environment. The system will be subject to load (using a stub) to a proportionate volume to the load used in Systems Integration Testing. A co-ordinated performance test will then be undertaken involving all SEC Parties that are participating in Interface Testing (anticipated to be a minimum of 20) during which they will concurrently submit a Service Request. The metrics from this test will be compared with the metrics for the equivalent test using the Service User Simulator during SIT to confirm that the introduction of SEC Party systems has not adversely affected the performance of the DCC solution.Full Performance Testing will take place on the production environments connected together with the production communications links as part of Operational Acceptance Testing (OAT). Where practical to do so, this OAT testing will be conducted in the same timescales as IT and the results provided in the IT Stage Completion Report to the SEC Panel. Otherwise the results will be reported separately to the SEC Panel for information.Resilience TestingAs with Performance Testing, full Resilience Testing is not feasible in IT and will instead take place on the production environments connected together with the production communications links as part of OAT. SEC Parties are not expected to participate in Resilience Testing. Where practical to do so, OAT testing will be conducted in the same timescales as IT and the results provided in the IT Stage Completion Report to the SEC Panel. Otherwise the results will be reported separately to the SEC Panel for information.UEPTSEC Parties will undertake User Entry Process Testing as described in the CTSD/SREPTSD [7]. The DCC will facilitate UEPT as described in the CTSD/SREPTSD.Prior to starting UEPT, the DCC requires SEC Parties to undertake a Connectivity Test in order to verify that the Service Users system can connect to the DCC. This comprises the sending of a DCC-only Service Request and the receipt of a Response.

DeliverablesResponsibilities for DeliverablesThe responsibilities of each Test Participant with respect to each IT deliverable are set out in Figure 6 RACI Matrix showing Deliverables.Key:RResponsibleAAccountableCConsultedIInformed

Test ParticipantDeliverableDCCDSPCSPDCC EnterpriseTSPRDPSEC Party

Interface Test Approach documentARCCCCC

Test Infrastructure (hardware & Communication)ARRRR-R

Test Environments (software & configuration)ARRRR-R

Test Labs Premises + Communication HubsACR---C

Test Labs - DevicesA/RCC---C

Interface Test PlanARCCCC-

Test Data Plan documentARCCCC-

Test Data ProvisionARRRRRR

Test ToolsARR---R

Executed TestsARIII-R

Support for Test ExecutionARRRR--

IT Preparation Progress ReportsARRRRRR

IT Execution Progress ReportsARIII-R

IT Stage Completion ReportARIII-R

Figure 6 RACI Matrix showing DeliverablesNotes on DeliverablesThe DSP, in its role as systems integrator, is responsible for the overall planning, management and delivery of Interface Testing. The DCC, in its overarching assurance role, is accountable.The DCC (together with Device manufacturers) is responsible for provision of the Devices for use in CSP Test Labs and any related technical support which is necessary. These Devices will be selected in accordance with the Device Selection Methodology [6]. The CSPs are responsible for provision of the physical environment of each Test Lab, the communications hubs and related technical and testing support. The Device manufacturers will install Devices in the CSP Test Labs, and the CSPs (together with the Device manufacturers) will verify that they function in connection with the communications hubs.The Test Tools row in the above table refers to any tools which need to be used for Interface Testing. The CSPs will provide the stubs used in SIT, if Devices are not available for IT. These stubs will provide a platform for the receipt and transmission of commands. IT does not test the functionality of the Device and the use of stubs should therefore provide an adequate level of assurance that a DCC user can communicate with a Device (noting the criteria that are set out in the SIT Approach Document for verifying that a test stub is suitable for use). However, Energy Suppliers will be invited to re-perform an install and commission test when actual Devices are available. The DSP may need to provide a stub to simulate the activity of a SEC Party, in order to carry out some of the test preparation and specific test execution activities (e.g. Change of Supplier). Test ProcedureRequirements traceabilityThe Requirements Traceability Matrix (RTM) output from SIT will be updated with tests added in IT and the revised version will be supplied to the DCC.Test Stage overlapSP UAT Test Stage with Interface Test StageAs part of the programme re-plan in February 2014, the decision was taken to overlap the SP UAT Test Stage of SIT with the Interface Test Stage in order to minimise the impact of Great Britain Companion Specification (GBCS) delays on the go-live date. This overlap introduces a risk that any significant defects found in SP UAT will cause re-work and delay in Interface Testing, however this risk is small given that SP UAT is merely a repeat of a subset of the tests run in the Solution Test Stage. This risk will be mitigated by scheduling the high risk tests for the start of SP UAT.Solution Test Stage with Interface Test StageGiven that SIT will be conducted on a CSP Region by Region basis, the DCC may, after consultation, recommend to the Secretary of State that Interface Testing for one or more Regions is started before Solution Testing is completed.The DCC will publish any such proposal and consultation results on its website.Where the Secretary of State agrees with the DCCs recommendation, Interface Testing shall commence from the time recommended for the identified Regions.Interface Test Stage with End to End Test StageEnd to End Testing is currently scheduled to follow Interface Testing but the DCC, after a period of consultation, may recommend to the SEC Panel that End to End Testing starts before the end of Interface Testing.The DCC will publish any such proposal and consultation results on its website.Technical ReadinessService ProvidersAll SPs will undergo a Technical Readiness assessment before participating in IT, which will confirm:the impact on IT of any outstanding Solution Test defects, and whether the dates in the Work-Off Plan are acceptablethat the relevant IT environments have been: built configured loaded with the required solution elements (including Test Stubs, tools and devices) connected up (e.g. firewalls opened up and validated) smoke testedthat the base test data required to commence IT has been loaded to the relevant data storesthat the Test Management tool has been set up and made available to the relevant personnelthat guidelines for the Test Management tool have been developed and that the relevant SP and supporting SEC Party personnel have been made familiar with its use.SEC PartiesAll SEC Parties will undergo a Technical Readiness assessment as defined in the CTSD/SREPTSD.RDPsThe Technical Readiness assessment will consist of verifying that each has supplied the data necessary to support the testing.Testing PrinciplesNon-discriminationThe DCC will support SEC Parties performing UEPT in IT and the DCC will facilitate the concurrent testing of these SEC Parties to the extent reasonably practicable. To facilitate this, SEC Parties are required to notify the DCC of their intention to participate in IT five months ahead of the start of IT execution so that Test Lab facilities and support resources can be expanded where necessary.Should unexpected problems be encountered, Energy Suppliers will be given priority to test in their gas supplier and electricity import supplier roles in order that IT can complete in a timely manner. See Section 7.1 - Introduction, for details on current assumptions made regarding the number of Users who will participate.Use of Devices/Device stubsIT will be conducted with physical Devices rather than stubs. However, if Devices are not available at the start of IT, stubs will be used. Should Devices become available in sufficient volume during the conduct of IT, they will be introduced into IT. However, the DCC will first verify that these Devices can interoperate with the DCC, which may require testing in the SIT environment or against some other suitable test tool. Before introducing Devices into IT, the DCC will determine if by doing so it introduces significant delay (i.e. because of the need for retesting) to the completion of IT or adversely impacts any SEC Party that is conducting UEPT. In choosing to use stubs as the basis of IT, the DCC will first confirm that the stub will provide a suitable basis for the conduct of UEPT.Testing with one CSPEach SEC Party will be expected to conduct their UEPT tests with a single CSP, as allocated by the DCC. Every effort will be made to accommodate a preference where one is expressed. For Network Operators, the CSP allocated will be in the geographical area covered by that SEC Party.Having completed the tests with one CSP, the SEC Party will not be expected to repeat them with the other CSP. Testing by RegionThe SEC enables IT to start in one Region in advance of another Region.As set out in Section 6.4.3, the DCC will make every effort to allocate SEC Parties to the Region of their choice for the purpose of conducting UEPT. However, where the difference in the start date for the two Regions is more than 2 weeks, SEC Parties will be allocated to the first available Region. At the point at which the second Region becomes available, those SEC Parties who have already completed UEPT (or are still undertaking UEPT) on the first Region will be invited to conduct selected tests on the second Region. SEC Parties yet to start their UEPT will be allocated across both Regions, subject to capacity constraints.Release managementReleases of fixes and configuration changes will be scheduled for each week during IT, unless otherwise agreed by the DCC (frequency to be reviewed in the light of experience). The UIT Issue Manager will agree the contents of these releases with the SP IT Managers and the DCC Service User Integration Test Manager. Should an emergency Release be required (e.g. because testing is seriously impeded or halted), the UIT Manager will decide on the timing after consulting with the SP IT Managers and the DCC Service User Integration Test Manager. Each Release into the IT environment will be accompanied by a Release Note describing the contents of the Release. All fixes and configuration changes included in a Release will undergo PIT and SIT with the relevant SPs, which includes an appropriate degree of regression testing according to the principles laid out in the PIT and SIT Approaches. Once a Release has been installed in the IT environment, it will be subject to: a smoke test, that will be undertaken by the DCC; testing of the constituent fixes and configuration changes, that will be undertaken by the relevant SPs and SEC Parties; and an IT regression test, based on assessment of the risks that the new release will cause features previously working in IT to stop working that will be undertaken by the DCC.Configuration managementThe DSPs Configuration Manager owns the master Configuration Plan which defines a) the Configuration Items (CIs) comprising the DCC Solution and b) the inter-dependencies between these CIs. This configuration management process is part of the test preparation activities. See Figure 7 Timetable for test for the timeline of this.

Smart Metering Implementation ProgrammeInterface Testing Approach

Page 1 of 2303/10/14 V1.0DCC Public

TimetableThe high level timetable for IT can be divided into the following sections: Test Initiation Test Execution Test Completion.The tables below list the activities that must be undertaken in each section. They do not duplicate the information relating to UEPT activities in the CTSD/SREPTSD but describe additional activities.1.1.1 IT Test InitiationThe table below sets out the steps that must be undertaken by either the DCC or relevant Test Participant seeking to undertake IT and the timeframes within which such steps must be completed.RefBy WhenActionFromToInformation RequiredMethod

123/07/2014Publish P&C Software Architecture SpecificationDCCSEC PartiesDCC website

26 months prior to the start of ITPublish IT ApproachDCCSEC PartiesSEC Panel approvalDCC website

36 months prior to the start of ITPublish CTSD/SREPTSDDCCSEC PartiesIncorporated into the SEC as a subsidiary documentDCC website

46 months prior to the start of ITPublish SMKI Interface Specification DCCSEC PartiesDCC website

56 months prior to start of ITPublish Interface Testing PlanDCCSEC PartiesDCC website

66 months prior to start of ITPublish Test Data Plan DCCSEC PartiesDCC website

76 months prior to start of ITPublish Environment PlanDCCSEC PartiesBy secure email

85 months prior to the start of ITConfirm intention to participate in IT SEC PartiesDCCConfirmation of intention, and supporting plan By email

96 months prior to the start of ITOrder Network ConnectionSEC PartiesDCCCompleted Order Formemail

104 months prior to start of ITP&C software V2 available DCCSEC PartiesDCC website

112 months prior to start of ITPublish Testing Issue Management Tool GuideDCCSEC PartiesDCC website

122 month prior to start of ITPublish Schedule of IT executionDCCSEC PartiesDCC Website

132 months prior to start of ITPublish Testing Issue Resolution and Disputes ProcessDCCSEC PartiesDCC website

142 months prior to start of ITPublish IRB Terms of ReferenceDCCSEC PartiesDCC website

1530 working days prior to the start of ITSet up Individual test environmentsALLCompleted Checklistemail

1630 working days prior to the start of ITSupply meters for Test LabsDCCInventory listemail

1725 working days prior to the start of ITConfigure and connect InterfacesDCCALLCompleted Checklistemail

181 month prior to the start of ITPopulate test environments with test dataSEC PartiesDCCConfirmationemail

191 month prior to the start of ITDevelop Test Stubs (if needed)SEC PartiesDCCConfirmationemail

201 month prior to the start of ITSet up Testing Issue Management Process & ToolDCCSEC PartiesCompleted Checklistemail

211 month prior to the start of ITSet up Configuration Management Process & ToolDCCSEC PartiesCompleted Checklistemail

221 month prior to the start of ITProvision Network ConnectionDCCSEC PartiesCompleted Checklistemail

231 month prior to the start of ITSet up Test LabsDCCSEC PartiesCompleted Checklistemail

241 month prior to the start of ITDeliver Test ArtefactsSEC PartiesDCCHigh level test design, test scripts, regression test packBy email

2515 working days prior to the start of ITValidate eco-systemDCCALLCompleted Checklistemail

265 working days prior to the start of ITComplete Readiness VerificationDCCALLCompleted Checklistemail

271 working day prior to the start of ITSatisfy IT Entry CriteriaSEC PartiesDCCQuality Gate checklistby email

28Start of ITCommence IT ALL

Figure 7 Timetable for test initiationOne of the key events in the above section is Confirm intention to participate in IT (Ref 8), which is when SEC Parties planning to participate in IT must declare their intention to do so. Declaration of intention to test must be accompanied by a plan, against which the Users preparation progress will be monitored. The date of declaring intention to test is when the Service Providers need to know whether or not the planned Test Lab and support facilities will be adequate for the expected demand: for further details of this and the assumptions made, see Section 7 - Environments, Networks and Test Labs. 1.1.2 IT Test ExecutionThe table below sets out the steps that must be undertaken during test execution by either the DCC or Testing Participant and the timeframes within which such steps must be complete.RefBy WhenActionFromToInformation RequiredMethod

1[10 working days] after the start of ITUser Connectivity Validated (first 2 users)DCC

2[1 month] after the start of ITNorth Region : Connectivity Test completedSEC PartiesDCCTest Completion Reportemail

3[2 months] after the start of ITNorth Region : UEPT completedSEC PartiesDCCTest Completion Reportemail

4[2 months] after the start of ITNorth Region: Additional Functionality Tests completedDCCTest Completion Reportemail

5[30 working days] after the start of ITCentral & South Region : Connectivity Test completedSEC PartiesDCCTest Completion Reportemail

6[50 working days] after the start of ITCentral & South Region: UEPT completedSEC PartiesDCCTest Completion Reportemail

Figure 8 Timetable for test executionAnother key milestone is User Connectivity Validated (first 2 users) (Ref 1). This is where it has been demonstrated that two SEC Parties in each Region can successfully connect to the DCC System, by the successful sending of a Service Request and subsequent receipt of a Response. (All SEC Parties will conduct Connectivity Testing, but for the purposes of milestone planning, the DCC will assume that user connectivity is validated when two SEC Parties have demonstrated they can successfully connect to the DCC system.)1.1.3 IT Test CompletionThe table below sets out the steps that must be undertaken during test completion by either the DCC or Testing Participant seeking to undertake IT and the timeframes within which such steps must be completed.

RefBy WhenActionFromToInformation RequiredMethod

110 working days before the end of ITDraft IT Completion ReportDCCSEC PanelAs described in Joint Test Strategy [2]email

2End of ITFinal IT Completion ReportDCCSEC PanelAs described in Joint Test Strategy [2]DCC website

Figure 9 Timetable for test completion

Smart Metering Implementation ProgrammeInterface Testing Approach

DependenciesThere are a number of dependencies within IT preparation and execution, which are described in the following tables. In each case, the milestones and activities listed in the tables are taken from the above Timetables: Figure 7 Timetable for test initiation and Figure 8 Timetable for test execution.MilestoneDependent onNotes

Interface Test Approach Published Completion of industry consultation SEC Panel approval

Interface Test Plan PublishedInterface Test Approach

Test Data plan PublishedDefinition of data

Test Data Ready Issuing of Test Data Plan Population of data Verification of data

Test Tools Ready Development of test stubs Set-up of Testing Issue Management tool

Parse & Correlate Parse & Correlate v2 S/W being Available Provision of network connection Set-up of test labs Set-up/configuration of individual environments Configuration of interfaces Verification of eco-system - smoke test, penetration test

Test Artefacts Delivered Development of detailed test scripts Development of other test artefacts

Interface Test Entry Criteria SatisfiedSee Section 6.9.1 - Entry Criteria

Interface Test Started Publication of Interface Test Approach Publication of Interface Test Plan Declaration of Intention to Test Test Data being Ready Test Tools being Ready Test Infrastructure & Environments being Ready Interface Test Entry Criteria being SatisfiedThe Interface Test Approach must be published 6 months before the start of IT

Intention to Test must be declared 5 months before the start of IT

User Connectivity ValidatedCompletion of Connectivity TestThe Connectivity Test must be successfully completed by two Large Suppliers of each energy type in at least one CSP Region to pass this milestone.

Interface Test CompleteSee Section 6.9.2 - Exit Criteria

Figure 10 Milestones and their dependenciesStage Entry and ExitEntry CriteriaGeneral entry criteria for the Stage: Interface Test Approach (this document) published 6 months previously; Authority to Proceed certificate issued by DCC; Interface Test Specifications (and supporting test data) prepared; Technical Readiness Assessments (see Section 6.4) completed for each SP and RDPs have provided the data necessary; Service Provider test environments and Test Labs prepared; Service Provider resources are available to support the testing; and confirmation that SPs have implemented the security controls defined in the Code of Connections for the DCC User Gateway Interface [10] and Self Service Interface [11], where the IT test environment differs from that used for SIT.For SEC Parties participating in UEPT: confirmation from the DCC Service User Integration Test Manager that the SEC Party has met their Entry Criteria, as defined in the CTSD/SREPTSD [7]; SEC Party resources are available to support DSP-led testing.Exit CriteriaFor the Interface Test Stage SIT has successfully completed for both CSP North and CSP South/Central; SIT Phase Completion Certificate issued; Connectivity Test and UEPT successfully completed by at least two large suppliers of gas (who are not affiliates of one another) in their role as gas supplier and two large suppliers of electricity (who are not affiliates of one another) in their role as import supplier, for each of CSP North and CSP South/Central (successful Connectivity Test is defined as all planned tests run; successful UEPT is defined as issue of Completion Certificate); Connectivity Test and UEPT successfully completed using the DCC-supplied Parse & Correlate software by at least one supplier (successful is defined as confirmation from the UIT Manager that the suppliers evidence of the use of Parse & Correlate is acceptable); Where the Secretary of State has required the Network Parties to be ready to participate in IT, Connectivity Test and UEPT successfully completed by at least one network party in their role as electricity distributor or at least one network party in their role as gas transporter, for each of CSP North and CSP South/Central; All planned Additional Functionality Tests run, or any exceptions documented and agreed with the DCC; The number and severity of outstanding testing issues are within agreed thresholds and the work-off plan has been published (see Section 9.4 - Testing Issue Severities); DCC-produced IT Stage Completion Report issued; and DCC-produced IT Stage Completion Certificate issued.For SEC Parties participating in UEPT: confirmation from the DCC Service User Integration Test Manager that the SEC Party has met their Exit Criteria, as defined in the CTSD/SREPTSD [7].SEC Panel approvalHaving consulted with the RDPs and the SEC Parties, the DCC will present the IT Stage Completion Report and the IT Stage Completion Certificate to the SEC Panel for approval when the DCC considers that the Exit Criteria have been met. The DCC will clearly demonstrate how each exit criterion has been met and the level of assurance that has been achieved. If the Panel rejects the recommendation, then the DCC will undertake further tests as directed by the SEC Panel and re-submit the evidence.Should one Region complete IT before the other, then the DCC will present an IT Region Completion Report to the SEC Panel. The contents of the report will be the same as the IT Stage Completion Report.Environments, Networks and Test LabsIntroductionThe DCC test environment for IT comprises a set of networked environments provided and supported by each Service Provider, and Test Labs at each of the CSPs premises furnished with communications hubs and Devices.The environments, Test Labs and support have been designed on an assumed number of testing participants (a SEC Party conducting testing for a particular Role) taking part in Interface Testing. The assumption is that there will be 120 testing participants, of which 90 will be assigned to test in CSP Central/South and 30 assigned to CSP North.Should the number of SEC Parties wishing to test simultaneously exceed these numbers, then the requirements will be assessed, the technical feasibility validated, the relevant approvals sought, the delivery planned and then the Test Labs and technical support can be increased accordingly. The long lead times involved mean that an early date for notification of intention to test has to be given (see section 6.7.1 - for the relevant date).DCC Enterprise environmentsDCC Enterprise will provide a test environment which contains Billing and Reporting functionality.DSP environment and networkThe DSP will provide a test environment containing all the DSP-delivered components, including the Self Service Interface (SSI), as well as the network connections used by the SEC Parties. CSP environments and networksBoth CSPs will provide a test SMWAN network, together with test systems connected to the DSP. If SIT Solution Test has exited with Device stubs and physical smart Devices are still not available in time to use in IT, CSPs will also provide the Device stubs used in SIT.TSP environmentThe TSP will provide a test environment containing all the TSP-delivered components.Parse & CorrelateThe Parse & Correlate software will be provided to the SEC Parties, for installation on their test environments.SEC PartiesSEC Parties will provide and support their own test environments for use in IT, and are responsible for establishing connectivity with the DSP network. The DCC will provide support to the SEC Parties as requested and as appropriate.CSP Test Labs (for devices)Each CSP will provide a secure physical environment in which to house the relevant number of communications hubs and Devices. The CSP will also provide the communications hubs (Devices will be provided by the DCC in collaboration with the meter manufacturers as described in the Device Selection Methodology [6]);Each Test Lab will contain sets of devices, with a set consisting of a minimum of: one communications hub one electricity meter one gas meter,and other Devices or stubs as required to complete UEPT.Network operators will be allocated 2 Device sets and all other SEC Parties will be allocated 5 device sets.Details of support provided by the CSPs during testing are yet to be finalised[footnoteRef:5], and the intention is that the type and level of support offered by the two CSPs will be similar. It is expected to include support for checking the Devices during testing, as well as facilities which will allow SEC Parties to house a limited number of their own staff on site and allow them controlled access to the test labs. The CSP-provided support for checking the Devices will be via a call-logging system, which will operate two different support methods, using: [5: The level of support will be discussed with Test Participants via the TAG and TDEG. Any comments will be taken into account to the extent appropriate in the final version of this document and the results of the discussions and any issues raised will be presented to the TAG when it conducts its final review on the document prior to making a recommendation to the SEC Panel on whether this document should be approved.]

requests for support at a specified time in the future (following working day onwards); and requests for immediate support.Once a CSP support team member has started working on a particular request, then contact will be directly between the CSP staff member and the relevant SEC Party staff member, either by phone or by email. Email contact between the Test Participants will be secured by TLS or an equivalent method.Figure 11 Number of device sets shows the numbers of devices available:Number of Device SetsCSP North (Arqiva)CSP Central/South (Telefonica)

Initial Provision *75225

Expansion Capability (number that can be added) 25150

Total Possibly Available100375

Figure 11 Number of device sets*A portion of the Initial Provision (final percentage to be confirmed, current estimate is 10%), will be used for Systems Integration Testing, that occurs in parallel with IT (and therefore is not available for IT)Devices will be available for electricity export testing. The devices will be subject to load simulation to the extent necessary to support UEPT, noting that the functionality of the devices will not be tested.Sufficient numbers of spare devices will be held in reserve so that faulty devices can be replaced immediately, as laid out in the Device Selection Methodology [6]. This will ensure that the number of operative Devices is kept at the level shown in Figure 11 Number of device sets.It is intended that where possible, Devices and communications hubs will be re-used by different SEC Parties, as some SEC Parties complete their testing and possibly others begin their testing. This may be done by using a simple Change of Supplier process. In addition, each Energy Supplier will need some devices provided in an initial state as delivered by the meter manufacturer (in order to perform an Install and Commission Test) and this will be catered for.De-selection and substitution of DevicesWhere it becomes evident that a Device that is used in IT for the purpose of UEPT has a defect, the DCC will de-select that Device in accordance with the Device Selection Methodology [6].In doing so, the DCC will first determine if the defect relates to one specific Device or to all Devices of that specific model. Where the defect relates to one Device, that Device will be substituted with one of the same type. If the defect relates to all Devices of that model, the DCC will substitute Devices of that model type with another Device model according to the Device Selection Methodology. In so doing, the DCC will undertake whatever testing it deems appropriate to prove that the substitute Device model can communicate with the DCC in respect of all relevant messages. If it is not possible to substitute the defective Device with either another of the same model or of a different model, then the DCC will use a test stub in place of the defective Device.Smart Metering Implementation ProgrammeInterface Testing Approach

Summary of test facilities providedStandard facilities are as described in the table below. Reasonable requests for out-of-hours support will be considered on a case-by-case basis.DCC Enterprise SystemsDSP Systems/ DCC User GatewayTSP SystemCSP Systems/ SMWANParse & CorrelateTest Labs CSPTest Labs DCC/meter manufacturer

Hours of support availability/ access to labs08:00-18:00 on working days08:00-18:00 on working days08:00-18:00 on working days08:00-18:00 on working days08:00-18:00 on working days08:00-18:00 on working days08:00-18:00 on working days

Figure 12 Summary of test facilities providedA working day is defined as Monday-Friday, excluding England & Wales Bank Holidays.

Smart Metering Implementation ProgrammeInterface Testing Approach

Page 36 of 7403/10/14 V1.0DCC PublicTest DataTest Data PlanFull details of the test data to be used will be specified as part of a separate exercise and documented in the Test Data Plan (see Section 6.8.1 for the timeline for production of this Plan).The Test Data Plan will cover: Principles How data is allocated to each SEC Party How data allocation is communicated How co-ordination will ensure no overlap/conflict between SEC Parties when testing (and between DSP-led tests and SEC Parties doing UEPT) Responsibilities of each SP/SEC Party Timetable, including When data must be provided Dependencies Test Data Management Process ToolData Specification - for each item of data needed for system set-up: Test Participant responsible for providing/maintaining How it relates to other data in the DCC Systems Source and content of the data Quantity of data requiredThe production of this Test Data Plan and subsequent provision of test data for IT will be a collaborative exercise between all SPs and the SEC Parties participating in TDEG. This will ensure that SEC Parties have adequate and correctly-functioning data with which to test.Test data principlesThe principles stated in the Test Data Plan will include the following: Data provided will be adequate to support the planned number of participating SEC Parties Data provision and maintenance will be co-ordinated and managed by the DSP, who may instruct an SP or SEC Party to carry out an update or correction to the data for which they are responsible Test data will comprise a cut of old Live data if necessary, with the proviso that the Supplier to an MPxN is not divulged to another Supplier Customer names will not be included in the data provided Some test data will be reserved for DSP-led testing Each SEC Party will be allocated data with which to test (which will be enough to cover the number of Device sets they are using for IT); this will be done by consultation between the DCC and the SEC Party The data one SEC Party is using for testing will not be revealed to another SEC Party; that SEC Party will have exclusive use of the allocated data during IT (e.g. meter points allocated) Data will comply with the provisioning and management guidelines in the Security section (Section G) of SEC [3]. Data will be reviewed before testing starts Use of data will comply with the UK Data Protection Act Data will be able to be restored (if needed) at some point during the testing.

Responsibilities for data provisionFigure 13 High-level data types and responsibilities for provision shows the main types of test data in the system and which Test Participant has the responsibility for defining, providing and maintaining this data.Data TypeAccountable Party

Service Requests/Responsesn/a

RegistrationRDPs

SMS InventoryDSP

SMKI Repository: Organisation Certificates Device Certificates Certificate Revocation ListsTSP

SSI/SSMI User Security CredentialsDSP

CSP network coverageCSP

RDP listDSP

Mapping: Post code to CSP region User role to Service Request GBCS Use Case device capability to Service RequestDSP

DCC Reference (manufacturers, device types, models)DSP

User Manuals and FAQs (for SSI)DSP

Network configuration: User Gateway SMWAN SMKI DSP CSPs TSP

System configuration: Billing ReportingDCC Enterprise Provider

Figure 13 High-level data types and responsibilities for provisionTesting Issue ManagementPrinciplesThe DCC will provide licences for HP Quality Centre (HP QC), for use by all SEC Parties and SPs participating in Interface Testing, for logging and resolving testing issues. A single licence will be provided to each SP and SEC Party and they will be required to record testing issues directly on the HP QC database.HP QC is a test management tool, which has facilities to create and manage the following: Business requirements Test scenarios and detailed test scripts Test Phase and Cycle execution details Testing Issues.It allows links to be established between the items listed, which facilitate the production of quality assurance documentation such as Requirements Traceability Matrices (RTM) and test execution progress reports. Its set-up will be done as shown in Figure 7 Timetable for test .HP QC will be used to: manage DSP-led testing demonstrate traceability between SP Requirements, test scripts and issues.SEC Parties will use HP QC only for logging testing issues. The HP QC Issue Repository will comprise several different (and segregated) Projects: Interface Testing Issue Repository (IT IR) testing s relating to DCC Systems uncovered in any of the tests conducted as part of IT (i.e. Connectivity, UEPT, Additional Functionality Tests) SEC Party Issue Repository (SEC Party IR) one for each participating SEC Party, containing testing issues relating to that SEC Partys systems (i.e. testing issues which the SEC Party needs to resolve before exiting their UEPT) Device Issue Repository (Device IR) one for each manufacturer with a device being used in IT.This segregation ensures that one SEC Party cannot see testing issues relating to the systems of another SEC Party, and that one Device manufacturer cannot see testing issues relating to the Devices of another manufacturer. However, all Test Participants involved in the testing will have access to IT IR. IT IR will be used to record only testing issues attributed to the DCC Systems, and the testing issues in IT IR will be used to determine whether or not the IT Exit Criteria have been met.If a testing issue recorded in IT IR is subsequently attributed to a SEC Party or Device manufacturer it will be closed on IT IR and a new testing issue opened on the appropriate IR. The UEPT for an individual SEC Party will be managed using their SEC Party IR, and the testing issues in this repository will be used for determining whether or not the SEC Party has met its UEPT Exit Criteria.Testing Issue Management toolThe HP QC instance will be hosted and administered by the DSP. The DCC will ensure that sufficient licences are available for use by the various Test Participants involved in IT. This HP QC instance must be used for recording testing issues.SEC Parties may be using their own HP QC systems for managing their testing issues internally. In order to support this, IT IR will be set up so as to accept input of new testing issues by either: Online input - by being typed into the tool online directly by SEC Party/Service Provider, or Excel import a template will be provided by the DSP to SEC Parties, to enable them to capture testing issues once only in their own system, and then have them imported direct into the DSPs central HP QC system (these templates will be transferred between the SEC Party and the DSP via secure email).Issue Review BoardA Terms of Reference[footnoteRef:6] will be created for the IRB, which will meet twice daily during test execution to review testing issues recorded on the IT IR. Each new testing issue will: [6: This will include who can attend, what will be discussed, who will be expected to attend (with reference to technical triage), what is the process including outcome, what information is expected to be presented at the meeting to enable successful triage and how confidential information is treated.]

be classified as one of: testing issue: that prevents execution of a test that causes an unexplained or unexpected outcome or response to a test not a testing issue (e.g. a misunderstanding) duplicate change need more information. have its Severity and Priority set be assigned to the relevant resolver group.The IRB will also review: the outstanding Severity 1 and Severity 2 testing issues to ensure they are being resolved at the requisite speed outstanding actions from previous IRBs.The IRB will be chaired by the UIT Issue Manager and attended by SP Issue Managers DCC Service User Integration Test Manager (and relevant SEC Party representative as required) SP Design Authorities Meter Manufacturer SMEs UIT Manager DCC Test Assurance SP Test Managers as required.Where any testing issues are identified that impact the SEC or any subsidiary documents, they will be managed in accordance with the Issue Resolution Procedure set out in the SEC and the conclusions that are reached through this process will be incorporated into a Release Note where appropriate. Testing Issue Severities and PrioritiesEach testing issue raised will be assigned a Severity and a Priority.The Severity classifications are defined in the Joint DSP/CSP Test Strategy.The Priority classifications are defined in Figure 14 - Testing Issue Priorities, below, lists the standard Testing Issue Priorities:Testing Issue PriorityDescription

1All test progress is blocked by the testing issue.

2Testing not completely blocked by the testing issue but the impact on test progress is significant.

3Testing can proceed but the work-around for the testing issue has moderate impact on test progress.

4Testing can proceed and the testing issue has little/no impact on test progress.

Figure 14 - Testing Issue PrioritiesThe following table lists the planned target response times for testing issues, to be measured from the point at which the testing issue is logged in HP QC.

PriorityInitial response completedTriage completedAssessed by resolver groupFix time assessedTarget Release identified

11 hour4 hours5 hours13 hours17 hours

21 hourNext IRBNext IRB + 1 hourNext IRB +8 hoursNext IRB +8 hours

3N/ANext IRBNext IRB + 1 hourNext IRB +16 hoursNext IRB +8 hours

4N/ANext IRBNext IRB + 1 hourAs required to meet defect thresholdsAs required to meet defect thresholds

Figure 15 - Target testing issue response timesTesting Issues and exiting ITFigure 16 Outstanding testing issues and exiting IT, below shows the maximum number of testing issues of each severity which are allowed in order to exit IT. Note that the numbers are given on a per SEC Party per Service Provider basis.SeverityNumber of Outstanding Testing Issues per SEC Party per Service Provider

10

20

35

415

530

Figure 16 Outstanding testing issues and exiting ITTesting Issue reportingReporting of testing issues by SEC Parties in relation to UEPT will be undertaken as specified in the CTSD/SREPTSD [7].Reporting of testing issues relating to the DCC Systems will be done by the UIT Manager and based on the information contained in the IT IR and also in the Device IR. The reports will contain as a minimum: Current Testing Issue Status: Count of testing issues by status (total since beginning of test) Count of testing issues by Severity (total since beginning of test) Count of Open testing issues by Severity. Testing Issue Trend: For Severities 1+2 combined, the trend over time of number of testing issues, shown by status: Total of all testing issues Separately for each Service Provider (showing testing issues raised against that Service Provider) For all Severities combined, the trend over time, of number of testing issues, shown by status: Total of all testing issues Separately for each Service Provider (showing testing issues raised against that Service Provider)

Test Tracking & ReportingIntroductionDetails on Test Tracking and Reporting for both test preparation and test execution activities are described below.SPs and SEC Parties responsible for documentation deliverables listed in Section 5 - Deliverables will report on progress from the point at which this Interface Test Approach document is published. During the last month of Preparation, the frequency of reporting changes from monthly to weekly.The progress report from each Test Participant will be addressed to the UIT Manager. The UIT Manager will consolidate the information into a single report, which will be made available to the DCC and all Test Participants (information will be anonymized where necessary). The DCC will also make this report available to the SEC Panel. The report will be distributed via email and its classification will be assigned according to the DCC Classification Standard; the classification is likely to be either DCC CONFIDENTIAL or DCC CONTROLLED.Test Preparation ReportingGeneralDuring this part of the Stage, reporting is as follows:

Report NameFrequencyFromToMethod

Test Preparation Progress ReportMonthly(weekly during the final month)SEC PartiesSPsRDPsDCC

TBC

Test Readiness ReportWeekly From 40 working days prior to start of test execution SEC PartiesSPsRDPsDCC

TBC

Figure 17 Test Preparation ReportingTest Preparation takes place from the date of approval of this document until test execution commences. This means that in the final 2 months before test execution starts, SEC Parties and the DSP will need to provide both of these reports. There will be minimal overlap, given that the Test Preparation Report is at a higher level than the Test Readiness Report.Test Preparation Progress ReportThe Test Preparation Progress Report will be delivered on the last Friday of each month. It relates to the Test Participants preparation for the testing. The Test Participant may not be executing any tests itself (e.g. a CSP) but will have responsibilities in the run-up to the start of testing.The report will contain: Executive Summary, including overall RAG status Key Risks, Assumptions, Issues and Dependencies Key Achievements this Period Key Plans for next Period Deliverables: Date Planned Date Forecast RAG Status.A simple Excel spreadsheet will be made available to facilitate and standardise this reporting.Each report will be discussed in a short (30 minute) bilateral meeting in the first week of each month, chaired by the UIT Manager and attended by the reporting Test Participant as well as the DCC. Any issues or risks of relevance to other SEC Parties that require escalation will then be identified and discussed at the next TDEG. At this point, the TDEG meetings will be held monthly.Test Readiness ReportThe Test Readiness Report is as described in the CTSD/SREPTSD [7].For the DSP-led testing, the DSP will submit a Test Readiness Report in the same way as SEC Parties.Test Readiness Reports relating to UEPT will be submitted to the relevant DCC Service User Integration Test Manager.Test Execution Reporting

Report NameFrequencyFromToMethod

Test Execution DashboardDailySEC PartiesSPsRDPsDCC

TBC

Test Support Progress ReportWeeklyDCCDECCBy email and published on DCC Website

Test Completion ReportEnd of StageDCCDECCBy email and published on DCC Website

Figure 18 Test Execution ReportingThe Test Execution Dashboard is as described in the CTSD/SREPTSD [7]. It is delivered to the relevant DCC Service User Integration Test Manager, who will make it available to the UIT Manager. The DSP will also submit this report for its IT tests to the UIT Manager. The UIT Manager will produce a consolidated report once per week, which will not contain any SEC Party-specific information and will be available to all Test Participants involved in the testing and to the SEC Panel. This report will include a summary of test progress, number of testing issues raised (by Severity), number of testing issues outstanding (by Severity) and any risk to the timely completion of testing. The Test Support Progress Report will be delivered each Thursday by CoB, enabling a consolidated report to be produced each Friday (which will be redacted where necessary). Its contents will be the same as for the Test Preparation Progress Report.The Test Completion Report is as described in the CTSD/SREPTSD [7]. It is first provided in draft form 10 working days ahead of the planned end of the testing.

Roles & ResponsibilitiesGeneralAll parties involved in IT shall:Comply with the SEC and follow Good Industry Practice when participating in IT i.e. the exercise of that degree of skill, diligence, prudence and foresight which would reasonably and ordinarily be expected from a skilled and experienced person engaged in a similar type of undertaking as that Party under the same or similar circumstancestake all reasonable steps to facilitate achievement of the IT Objective.Roles and ResponsibilitiesRoles of Test Participants

Test ParticipantRole

DSP (Systems Integrator role) Responsible for management and delivery of Interface Testing, (including its planning, control and tracking) Provision of test execution and environment usage schedules Provision of Interface Test Plan Provision of Test Data Plan Management of testing issue resolution, provider of HP Quality Centre Operation of the master Configuration Management Plan Operation of the Environment Plan Provision of test scenarios and test scripts for Connectivity and Additional Functionality Tests Provision of resource to carry out Additional Functionality Tests Maintenance of IT RTM

DCC Provision of test assurance activities (see Section 11.5) Production of CTSD/SREPTSD Provision of Devices for test labs, working with Meter Manufacturers (see below) Provision of progress reports to the SEC Panel

DSP (SP role) Provision of DSP component of UIT test environment and related data Provision of test stubs to simulate SEC Party input (e.g. to support CoS testing) Provision of support for DSP solution elements and DSP component of UIT test environment Provision of fixes required in DSP solution elements Provision of resource to support SEC Parties testing

CSPs Provision of CSP component of UIT test environment and related data Provider of support for CSP solution elements and CSP component of UIT test environment Provision of fixes required in CSP solution elements Provision of test lab facility and communications hubs Provision of Device stubs (if necessary) Provision of support for test lab Provision of resource to support SEC Parties testing Provision of support to the DSP as Systems Integrator in: management and delivery of IT design and creation of test scenarios, test scripts, test data and test environments preparing test execution and environment usage schedules triaging testing issues creation and operation of the master Configuration Management Plan creation and operation of the master Release schedule creation and maintenance of the Environment Plan

SEC Parties Provision of detailed Connectivity Test and UEPT test scripts Provision of SEC Party test environment and related data Provision of resource to carry out the UEPT, Connectivity and Additional Functionality tests Provision of resource to triage testing issues Provision of any fixes required in SEC Parties systems Provision of Communication links to UIT test environment Provision of test evidence as set out in the CTSD/SREPTSD

RDP Provision of initial set of registration data to be used in testing Provision of resource to triage testing issues

TSP Provision of SMKI component of UIT test environment and related data, including test certificates Provision of support for SMKI solution elements and SMKI component of test environment Provision of fixes required in SMKI solution elements Provision of resource to support SEC Parties testing Provision of support to the DSP as Systems Integrator in: management and delivery of IT design and creation of test scenarios, test scripts, test data and test environments preparing test execution and environment usage schedules triaging testing issues creation and operation of the master Configuration Management Plan creation and operation of the master Release schedule creation and maintenance of the Environment Plan

DCC Enterprise Provision of billing/reporting component of UIT test environment and related data Provision of support for billing/reporting component of test environment Provision of fixes required in billing/reporting solution elements Provision of resource to support SEC Parties testing Provision of support to the DSP as Systems Integrator in: management and delivery of IT design and creation of test scenarios, test scripts, test data and test environments preparing test execution and environment usage schedules triaging testing issues creation and operation of the master Configuration Management Plan creation and operation of the master Release schedule creation and maintenance of the Environment Plan

Critical Software Provision of support for standard DCC Parse & Correlate software Provision of fixes required in Critical Software solution elements

Meter manufacturer (together with DCC) Supplier of meters to furnish the test labs, working with the DCC Provision of support for Devices both in labs and when returned to factory Provision of resource to triage testing issues

SEC Panel Approval of IT Approach Document Approval of IT Stage Completion Report Recipient of escalated testing issues from DCC Notify testing issues to the Secretary of State Resolve of testing issues where directed by the Secretary of State Recipient of progress reports

DECC Resolve testing issues notified by the SEC Panel Recipient of progress reports where provided by the SEC Panel

Ofgem Resolve disputes on IT Approach Document unless otherwise directed by SEC Panel

Figure 19 Roles of participating Test ParticipantsLarge Supplier Parties have an obligation to take all reasonable steps to be ready to start UEPT at the beginning of Interface Testing. Network Parties have an equivalent obligation, if directed to do so by the Secretary of State.Each Test Participant participating in test execution activities is responsible for ensuring that the staff it provides are adequately trained and have adequate facilities to enable them to perform the tests.

Smart Metering Implementation ProgrammeInterface Testing Approach

Page 57 of 7403/10/14 V1.0DCC PublicReporting structureThe reporting structure for IT is shown in Figure 20 IT main roles and structure, below:

Figure 20 IT main roles and structureThe UIT Manager will maintain a list of the contact details of the person filling each role for each Test Participant.

Roles of Test Management

RoleResponsibility

User Integration Test Manager

The User Integration Test Manager is the person responsible for overall planning, preparation and execution of Interface Testing. This role encompasses co-ordination, control, management and tracking/progress reporting for the testing. It includes directing the Connectivity and Additional Functionality Tests. The UIT Manager is provided by the DSP, in its role as overall systems integrator and reports via the Integration and Acceptance Test Manager to the DCC Test Programme Manager.

User Integration Testing Issue Manager

The User Integration Testing Issue Manager is the person responsible for overall management of testing issues. This includes responsibility for the repository used to manage testing issues, and chairing the Issue Review Board meetings. The UIT Issue Manager reports to the UIT Manager and is provided by the DSP, in its role as overall systems integrator.

DCC Service User Integration Test Manager The DCC will provide the DCC Service User Integration Test Manager to manage UEPT. This role is described in the CTSD/SREPTSD [7].

SP and SEC Party IT Manager The SEC Party IT Manager is the person who has overall responsibility for their Test Participants planning, preparation and conduct of the testing, including responsibility for the successful and timely completion of the testing. The SP IT Manager is the person who undertakes the planning and provisioning, and is responsible for providing support to those executing the tests. The SP and SEC Party IT Managers also have responsibility for the tracking and management of all testing issues relating to their organisation (alternatively they may delegate this to an Issue Manager). This person attends the daily Issue Review Board and is responsible for ensuring that testing issues observed by their organisation are recorded in a timely manner and resolved as agreed at the IRB.

SP SME/SEC Party SME

The Subject Matter Expert (SME) from the SP and from the SEC Party is available to support IT Manager in resolving testing issues. The SMEs are able to pin-point the cause of a testing issue and to collaborate with their counterparts in the Service Providers as well as other SEC Parties in order to determine responsibility for a testing issue (whether or not it is a testing issue relating to their own system), as well as the best way to resolve it.

Figure 21 Roles of Test Management

GovernanceThe DCC will provide overall governance of IT and liaise with the following stakeholders:DECCSEC PanelSEC PartiesEnergy UKEnergy Networks AssociationOfgem.The Forums and Boards relevant to IT are shown in the following diagram.

Figure 22 - IT GovernanceThe Test Design and Execution Group (TDEG) consists of Service Providers, SEC Parties, Ofgem and DECC. The UIT Manager will report plans and progress at each TDEG meeting, thus keeping all stakeholders informed and receiving their input.CommunicationThe DCC Programme Test Manager will report plans and progress at each TDEG meeting to ensure that SEC Parties, SPs, Ofgem and DECC are kept informed. Communication of plans and progress to the wider community will take place via a report at each Implementation Management Forum meeting (presented by the DCC Implementation Manager). The DCC will hold bilateral meetings with SEC Parties as part of the communication process.Progress meetingsA regular progress meeting for DSP-led testing will be held by telephone, chaired by the UIT Manager and attended by SP IT Managers and DCC Service User Integration Test Manager. This meeting will review the consolidated progress report and discuss any testing issues arising. The frequency will match the frequency of the progress reports: Monthly during most of test preparation (in the first week of each month) Weekly from one month prior to the start of testing and during the test execution itself (each Friday).Progress meetings for UEPT will be as described in CTSD/SREPTSD [7].For the further details and the contents of the progress reports, see section 10 - Test Tracking & Reporting.Testing Issues and their resolutionThe testing issue management process is described in general in Section 9 - Testing Issue Management.However, there may be urgent testing issues which require immediate communication with the Test Participants involved in testing. For this reason, all SEC Parties and SPs are expected to be available on email or the phone at all times during the hours of testing. If, for example, the system has to be taken out of service, then this will be communicated by the UIT Manager to the SPs and to the DCC Service User Integration Test Manager by both email and text message. The email will be marked High Importance (with !) and will be sent to an email address designated in advance. The text message will be sent to the designated phone number. A response is required within 30 minutes (at least an acknowledgement) to such communications. If there is no response, then the UIT Manager will place a phone call to the designated phone number. If there is still no response, then the necessary action will be implemented regardless. The UIT Manager will have a dedicated phone for this purpose. Email exchange will be secured by TLS or an equivalent method.Test AssuranceIntroductionThe UIT Manager will assure the test scripts for Connectivity and Additional Functionality Tests.The DCC will assure IT as described in the Joint DSP/CSP Test Strategy [2], using the following methods: Quality Gate Reviews Test Quality Audits Product inspections Document Review.Quality Gate ReviewsThere will be a Quality Gate Review between the Solution Test Stage of SIT and the Interface Test Stage. This Review will be chaired by the DSP, approved by the DCC and attended by the Service Providers.As approver, the DCC will set the outcome of the Review as one of the following: Solution Test Stage can close, Interface Test Stage can start, only minor (if any) remedial actions required, or Solution Test Stage cannot close until remedial actions have been completed, Interface Test Stage can start, or Solution Test Stage can close, Interface Test Stage cannot start until remedial actions have been completed, or Solution Test Stage cannot close, Interface Test Stage cannot start, until remedial actions have been completed.The Quality Gate Review meeting will be a short, checklist-driven event at which previously assembled and validated evidence relating to the Exit and Entry Criteria is considered and decisions made to close Solution Test and start Interface Test. It is expected that Quality Gate Review meetings will be dry-run to enable testing issues to be identified and resolved in a timely manner, and thereby avoid impacting the start date for Interface Test.The Solution Test Stage will complete (and achieve its Milestone) on attainment of its Exit Criteria. The Interface Test Stage will commence (and achieve its Milestone) on attainment of its Entry Criteria. Test Stage Complete certificates will be issued by the DCC as follows:

Item No.Test Stage Exit CriteriaCircumstance in which Test Stage Exit Criteria achievedResulting Certificate

1Interface Testing (North Region) Stage Exit CriteriaTest Success Criteria achieved in respect of Interface Testing in North Region with two (2) Large Supplier Parties in Great Britain; and one (1) Network Party if Network Parties are obliged to be ready to commence testing from the start of IT; andall other Stage Exit Criteria achieved Interface Testing (North) Test Stage Complete Certificate

2Interface Testing (Central Region) Stage Exit CriteriaTest Success Criteria achieved in respect of Interface Testing in Central Region with two (2) Large Supplier Parties in Great Britain; and one (1) Network Party if Network Parties are obliged to be ready to commence testing from the start of IT; and all other Stage Exit Criteria achieved Interface Testing (Central) Test Stage Complete Certificate

3Interface Testing (South Region) Stage Exit CriteriaTest Success Criteria achieved in respect of Interface Testing in South Region with two (2) Large Supplier Parties in Great Britain; and one (1) Network Party if Network Parties are obliged to be ready to commence testing from the start of IT; and all other Stage Exit Criteria achieved Interface Testing (South) Test Stage Complete Certificate

Figure 23 Test Stage CertificatesTest WitnessingThe DCC will agree with the DSP a witness execution schedule for the Connectivity and Additional Functionality Tests.The protocol for this witnessing is described in the Joint Test Strategy.Test Quality AuditsBy prior agreement with the Service Providers on the timing, duration and scope, the DCC may perform Test Quality Audits of Interface Testing.The protocol for conducting these Audits and addressing queries/issues is described in the Joint DSP/CSP Test Strategy [2].Such Audits will include reviewing that the Exit Criteria for Interface Testing have been met. Reviewing that Exit Criteria have been met may include a review of the IT Stage Completion Report and the testing issues raised during IT. Upon issue of the IT Stage Completion Report, the DCC will review it within 10 working days, or within a mutually-agreed period.Product inspectionsBy prior agreement with the Service Providers on the timing, duration and scope, the DCC may conduct on-site product inspections.The protocol for conducting these product inspections and addressing queries/issues is described in the Joint DSP/CSP Test Strategy [2].Documentation reviewThe DCC may undertake a review of any documents (including Design documents) used in testing by the Service Providers.The protocol for conducting these reviews and addressing queries/issues is described in the Joint DSP/CSP Test Strategy [2].Operational Acceptance TestingOperational Acceptance Testing (OAT) is a separate testing activity and will have its own scope, Test Approach and Test Plan documents. Its purpose is to verify that the solution:can be installed and configured in the production environmentcan be operated by the Service Management function under normal and exceptional conditionscomplies with the non-functional requirementswill meet its Service Level Agreements.Such verification includes but is not limited to:installation and configuration testingend to end security testing, including penetration testing and the Security Operation Centresservice monitoring and reportingBCDR testing: Business Continuity Planning testing resilience and failover testing of solution components (e.g. equipment, networks, databases) Disaster Recovery testingperformance testingtesting of the Service Management processesfunctional testing of service management toolsbackup and restore testing.OAT will be performed in the production environment before go-live, in parallel with the SIT and IT phases. Where practical, the test results will be provided in the SIT and IT Completion Reports, otherwise they will be reported separately to the SEC Panel for information purposes..

AppendicesTest ScenariosConnectivity Test ScenariosFor EIS, EES, GIS, RSA SEC Party roles:Test Scenario

Title:Connectivity to the DCC for EIS, EES, GIS or RSA SEC Party roles

Prerequisite: Energy Relevant Party holds the role of EIS, EES, GIS or RSA Connection to DCC Test Laboratory Appropriate data (i.e. device details provided by the DCC) SEC Party completion of Registration and Enrolment of Organisations and their representatives to the DCC Registration Authority following the processes described in the Registration Authority Policies and Procedures (RAPP) in order to acquire access to the SMKI using the SMKI Portal and also to the SMKI Repository.

StepsDescriptionObjectiveActionsAcceptance Criteria

1Device Pre-notification Populate Smart Metering Inventory1. Complete the following Service Request to support connectivity of device:DUGIDS SR 12.2 - Device Pre-notification SEC Party will receive the success I0 response code as defined in DUGIDS.

2Read Inventory Check device is registered on the Smart Metering Inventory1. Complete the following Service Request to support connectivity of device:DUGIDS SR 8.2 Read Inventory The SEC Party will receive the read the response that will contain all data item in the SMI for the device performed in Step 1.

3Test verification Verify that Service Request has been performed1. Query Service Audit via the SSI for the Service Requests performed in steps 1 and 2:i. DUGIDS SR 12.2 - Device Pre-notificationii. DUGIDS SR 8.2 Read Inventory The SEC Party will be required to provide a screen print of the audit trail query to the DCC via email.

For ENO, GNO or OU SEC Party roles:Test Scenario

Title:Connectivity to the DCC for ENO, GNO or OU SEC Party roles

Prerequisite: Energy Relevant Party holds the role of ENO, GNO or OU Connection to DCC Test Laboratory Smart Metering Inventory populated with a Device in own region

StepsDescriptionObjectiveActionsAcceptance Criteria

1Read Inventory Check device is registered on the Smart Metering Inventory1. Complete the following Service Request to support connectivity of device:DUGIDS SR 8.2 Read Inventory The SEC Party will receive the read the response that will contain all data item in the SMI for the device(s).

2Test verification Verify that Service Request has been performed1. Query Service Audit via the SSI for the Service Request performed in steps 1:DUGIDS SR 8.2 Read Inventory The SEC Party will be required to provide a screen print of the audit trail query to the DCC via email.

Additional Functionality Test ScenariosChange of Supplier (On-Demand) Test Scenario for EIS SEC Party RoleTest Scenario

Title:Change of Supplier (On-Demand) for EIS SEC Party Roles.

Prerequisite: Energy Relevant Party holds the role of EIS. Connection to DCC Test Laboratory. Dummy DCC Supplier setup as old supplier for take on the specified Devices. Appropriate data (i.e. device details provided by the DCC). Registration Test Data setup to match test scenario for the CoS date.

StepsDescriptionObjectiveActionsAcceptance Criteria

1Change of Supplier request Change of Supplier for each specified Electric meter devices:i. Electric Smart Meter (3x certificate updates)ii. HAN Connected Auxiliary Load Control Switch (1x certificate update)1. Complete the following Service Request to initiate the Change of Supplier process for each specified device updating the required certificates:DUGIDS SR 6.23 Update Security Credentials (CoS) Acknowledgement received for the DUGIDS SR 6.23 Update Security Credentials (CoS) Service Request by requesting SEC Party. Response is then delivered to the new supplier and the old (losing) supplier is notified of completion of the process via a DCC Alert N27.

2Update tariff Update of import tariff (on-demand) for the specified Electric Smart Meter device post CoS.1. Complete the following Service Request to support verification of completion of Change of Supplier:DUGIDS SR 1.1.1 Update Import Tariff (Primary Element) Acknowledgement received for the DUGIDS SR 1.1.1 Update Import Tariff (Primary Element) Service Request by requesting SEC Party. Response is received for Update Import Tariff by requesting SEC Party.

3Update billing calendar Update of billing calendar for the specified Electric Smart Meter device post CoS.1. Complete the following Service Request to support verification of completion of Change of Supplier:DUGIDS SR 6.8 Update Device Configuration (Billing Calendar) Acknowledgement received for the DUGIDS SR 6.8 Update Device Configuration (Billing Calendar) Service Request by requesting SEC Party. Response is received for the Update Device Configuration by requesting SEC Party.

4Retrieval of Debt and Credit Billing Data Log Retrieve Debt and Credit Billing Data Log post CoS.1. Complete the following Service Request to support verification of completion of Change of Supplier:DUGIDS SR 4.4.1 Retrieve Debt and Credit Billing Data Log Acknowledgement received for the DUGIDS SR 4.4.1 Retrieve Debt and Credit Billing Data Log Service Request by requesting SEC Party. Response is received for the Debt and Credit Billing Data Log by requesting SEC Party.

5Test verification Verify that Service Requests have been performed1. Query Service Audit via the SSI for the Service Requests performed in steps 1, 2, 3 & 4.2. Save screen print of audit trail query. The SEC Party will be required to provide a screen print of the audit trail query to the DCC via email.

Change of Supplier (On-Demand) Test Scenario for GIS SEC Party RoleTest Scenario

Title:Change of Supplier (On-Demand) for GIS SEC Party Roles.

Prerequisite: Energy Relevant Party holds the role of GIS. Connection to DCC Test Laboratory. Dummy DCC Supplier setup as old supplier for take on of the specified Devices. Appropriate data (i.e. device details provided by the DCC). Registration Test Data setup to match test scenario for the CoS date.

StepsDescriptionObjectiveActionsAcceptance Criteria

1Change of Supplier request Change of Supplier for each specified Gas meter devices:i. Gas Smart Meter (3x certificate updates)ii. Gas Proxy Meter (2x certificate updates)1. Complete the following Service Request to initiate the Change of Supplier process for each specified device updating the required certificates:DUGIDS SR 6.23 Update Security Credentials (CoS) Acknowledgement received for the DUGIDS SR 6.23 Update Security Credentials (CoS) Service Request by requesting SEC Party. Response is then delivered to the new supplier and the old (losing) supplier is notified of completion of the process via a DCC Alert N27.

2Update tariff Update of import tariff (on-demand) for the specified Gas Smart Meter device post CoS.1. Complete the following Service Request to support verification of completion of Change of Supplier:DUGIDS SR 1.1.1 Update Import Tariff (Primary Element) Acknowledgement received for the DUGIDS SR 1.1.1 Update Import Tariff (Primary Element) Service Request by requesting SEC Party. Response is received for Update Import Tariff by requesting SEC Party.

3Update billing calendar Update of billing calendar for the specified Gas Smart Meter device post CoS.1. Complete the following Service Request to support verification of completion of Change of Supplier:DUGIDS SR 6.8 Update Device Configuration (Billing Calendar) Acknowledgement received for the DUGIDS SR 6.8 Update Device Configuration (Billing Calendar) Service Request by requesting SEC Party. Response is received for the Update Device Configuration by requesting SEC Party.

4Retrieval of Debt and Credit Billing Data Log Retrieve Debt and Credit Billing Data Log post CoS.1. Complete the following Service Request to support verification of completion of Change of Supplier:DUGIDS SR 4.4.1 Retrieve Debt and Credit Billing Data Log Acknowledgement received for the DUGIDS SR 4.4.1 Retrieve Debt and Credit Billing Data Log Service Request by requesting SEC Party. Response is received for the Debt and Credit Billing Data Log by requesting SEC Party.

5Test verification Verify that Service Requests have been performed1. Query Service Audit via the SSI for the Service Requests performed in steps 1, 2, 3 & 4.2. Save screen print of audit trail query. The SEC Party will be required to provide a screen print of the audit trail query to the DCC via email.

Self Service Interface Test Scenarios1. A SEC Party wishes to request the Postcode Coverage Report via the SSI using the DCC IDP (Identity Provider).Test Scenario

Title:Self Servic