. n v e. n s·.~ s· . .

TM i r) v ' e . r) S · .!:l s ·

Operations Management Triconex Project: Purchase Order No.:

Project Sales Order:

PG&E PROCESS PROTECTION SYSTEM REPLACEMENT 3500897372 993754

PACIFIC GAS & ELECTRIC COMPANY

NUCLEAR SAFETY-RELATED PROCESS PROTECTION SYSTEM

REPLACEMENT DIABLO CANYON POWER PLANT

v &V DESIGN PHASE SUMMARY REPORT PPSI

Document No. 993754-11-861

Revision 3

January 15,2016

Author: Reviewers:

A roval: Kevin Vu

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 2 of 44 Date: 01/15/2016

Document Change History Revision Date Change Author

0 11/14/13 Initial issue S. Phan

1

04/08/14

Updated to incorporate Rev 9 Design phase V&V activities L. Nguyen

Revise MCL revision to 38 in Table 4-31a to includes this report. L. Nguyen

2 08/07/14

Cover Page -Changed document number from “993754-1-861” to “993754-11-861” and added “PPSI” in title per SVVP Rev 5. Added “PPSI” in Sections 2.0, 3.1, 3.2, and 4.1. Fixed formatting.

M. Montellano

3 01/15/16

Minor formatting and editorial corrections. -Table of Contents - Fixed the section numbers -Added “PPSI” to header title -Added Section number 6.1.2.2 -Section 6.2.1 –changed the font to bold.

M. Montellano

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 3 of 44 Date: 01/15/2016

TABLE OF CONTENTS

1.0 PURPOSE ..................................................................................................... 5

2.0 SCOPE .......................................................................................................... 6

3.0 TASKS PERFORMED ............................................................................... 7

3.1 PHASE INPUTS ........................................................................................................... 7

3.2 PHASE TASKS (RE-ENTER OF DESIGN PHASE) .................................................. 8

4.0 TASK RESULTS ....................................................................................... 10

4.1 RESULTS SUMMARY .............................................................................................. 10

4.2 TASK AND PHASE ITERATION ............................................................................. 12

4.3 PHASE OUTPUTS ..................................................................................................... 13

5.0 DESIGN PHASE ANOMALIES .............................................................. 14

6.0 SOFTWARE QUALITY ........................................................................... 16

6.1 SOFTWARE QUALITY METRICS .......................................................................... 16

6.1.1 Defect Count 16

6.1.1.1 SDD Defect Count ........................................................................................... 16

6.1.1.2 PTM Defect Count .......................................................................................... 17

6.1.2 Defect Density 18

6.1.2.1 SDD Defect Density ........................................................................................ 18

6.1.2.2 PTM Defect Density ........................................................................................ 20

6.2 V&V EFFECTIVENESS METRICS .......................................................................... 22

6.2.1 V&V Review Completeness 22

6.2.2 V&V Reporting Accuracy 23

6.2.2.1 SDD V&V Reporting Accuracy ...................................................................... 23

6.2.2.2 PTM V&V Reporting Accuracy ...................................................................... 24

6.3 QUALITY EVALUATION SUMMARY................................................................... 25

7.0 SOFTWARE SAFETY METRICS .......................................................... 28

8.0 TECHNICAL AND MANAGEMENT RISKS (RE-ENTER OF DESIGN PHASE) ...................................................................................... 30

9.0 RECOMMENDATIONS ( RE-ENTER OF DESIGN PHASE) ........... 31

10.0 REFERENCES........................................................................................... 32

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 4 of 44 Date: 01/15/2016

10.1 INDUSTRY DOCUMENTS....................................................................................... 32

10.2 INVENSYS TRICONEX DOCUMENTS .................................................................. 32

11.0 APPENDICES ............................................................................................ 33

Appendix 1: SDD Software Quality Metrics Data 33

Appendix 2: PTM Software Quality Metrics Data 33

12.0 ATTACHMENTS ...................................................................................... 44

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 5 of 44 Date: 01/15/2016

1.0 PURPOSE The purpose of this V&V Design Phase Summary Report is to summarize the results of the V&V tasks performed for the design phase of the application life cycle for the Nuclear Safety-Related (Class 1E) Diablo Canyon Power Plant (DCPP) Process Protection System (PPS) Replacement Project in order to fulfill the V&V reporting requirements specified in the Software Verification and Validation Plan (SVVP) [Ref.10.2.2].

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 6 of 44 Date: 01/15/2016

2.0 SCOPE The PPSI V&V design phase activities include architectural and detailed design for the implementation of the software and the plan for testing the software to ensure correct implementation of the requirements. The objectives of the V&V efforts are to ensure that the design is correct, consistent, complete, accurate, readable, testable, traceable and unambiguous and traceable to the requirements.

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 7 of 44 Date: 01/15/2016

3.0 TASKS PERFORMED 3.1 PHASE INPUTS The tables below contain the design phase inputs, the associated document numbers, revisions, approval dates. The Technical Requirements List (TRL) [Ref.10.2.6] lists document revisions, deletions or additions, which are considered as customer inputs during all project development phases. It provides a complete listing of the customer input documents while the SVVP provides a generic guidance. Therefore, this report is not intended to duplicate those documents listed in the TRL. This report summarizes the project inputs applicable for PPSI V&V design phase (Table 3-1a) and re-enter of PPSI design phase tasks on 02/25/14 (Table 3-11a)

Table 3-1a. Design Phase Inputs

Document ID Document Name Revision Date 993754-1-808 Technical Requirements List (TRL) 9 03/07/2013 3500897372 Purchase Order #3500897372, dated 12/20/2011 6 12/20/2011 08-0015-SP-001 PG&E PPS Replacement Functional Requirements Specification

(FRS) 5 04/17/2012

N/A PG&E PPS Replacement Interface Requirements Specification (IRS)

6 06/07/2012

10115-J-NPG PPS Controller Transfer Functions Design Input Specification 2 04/19/2012 08-0015-D Series PG&E PPS Function Block Diagram (FBD) 3 to 6 03/21/2012 993754-1-813 Validation Test Plan 2 12/18/2012 993754-1-915 Safety (Requirements Criticality/hazard/Risk/Interface) Analysis 1 10/24/2012 993754-1-803 Master Configuration List (MCL) 21 04/03/2013 993754-1-804 Project Traceability Matrix (PTM) 2 06/03/2012 993754-11-807 Protection Set I Hardware Requirements Specification (HRS) 2 10/07/2013 993754-12-807 Protection Set II Hardware Requirements Specification (HRS) 1 09/12/2012 993754-13-807 Protection Set III Hardware Requirements Specification (HRS) 1 09/12/2012 993754-14-807 Protection Set IV Hardware Requirements Specification (HRS) 1 09/12/2012 993754-11-809 Protection Set I Software Requirements Specification (SRS) 3 10/25/2013 993754-12-809 Protection Set II Software Requirements Specification (SRS) 2 10/17/2012 993754-13-809 Protection Set III Software Requirements Specification (SRS) 2 10/17/2012 993754-14-809 Protection Set IV Software Requirements Specification (SRS) 2 10/17/2012 993754-1-860 V&V Requirements Phase Summary Report 1 10/30/2012

Table 3-11a. Re-enter of Design Phase Inputs

Document ID Document Name Revision Date Document ID Document Name Revision Date 993754-1-808 Technical Requirements List (TRL) 11 01/29/2014 3500897372 Purchase Order #3500897372, dated 12/20/2011 6 12/20/2011

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 8 of 44 Date: 01/15/2016

CWA 3500897372 Contract Work Authorization – Change Order Rev9 0 12/18/2013 08-0015-SP-001 PG&E PPS Replacement Functional Requirements Specification

(FRS) 9 11/15/2013

N/A PG&E PPS Replacement Interface Requirements Specification (IRS)

9 11/15/2013

10115-J-NPG PPS Controller Transfer Functions Design Input Specification 4 11/15/2013 993754-35R Project Letter (DCPP operational data and initial tunable

parameter settings. N/A 12/13/2012

993754-64R Project Letter (PG&E Project Letter PPS-IOM-13-021-TCM Configuration Setting Design Input)

N/A 9/25/2013

993754-78R Project Letter (CD-ER 993754-24; Restore Streaming Factor Calculation.)

N/A 12/9/2013

993754-79R Project Letter (PPS Set I, II, III & IV Rev 9 Updated FAT Drawings – Revised Letter)

A 12/19/13

993754-82R Project Letter (CD-ER 993754-25; Interface Requirement Specification; updated Flux values and incorrect paragraph numbering.)

N/A 12/19/13

993754-91R Project Letter (CD-ER 993754-26; Suppress Reactor Coolant Pressure Alarm output when the channel is out of service).

N/A 1/21/2014

993754-92R Project Letter (DIT 68001801-1-0, PPS TSAP Constraints .and Input Scaling Adjustment Factors – Initial Values)

N/A 1/23/2014

08-0015-D Series PG&E PPS Function Block Diagram (FBD) 3 to 6 06/08/2012 993754-1-813 Validation Test Plan 3 01/28/2014 993754-1-915 Safety (Requirements Criticality/hazard/Risk/Interface) Analysis 3 01/28/2014 993754-1-803 Master Configuration List (MCL) 36 01/30/2014 993754-1-804 Project Traceability Matrix (PTM) 4 01/28/2014 993754-11-807 Protection Set I Hardware Requirements Specification (HRS) 3 01/16/2014 993754-12-807 Protection Set II Hardware Requirements Specification (HRS) 2 01/16/2014 993754-13-807 Protection Set III Hardware Requirements Specification (HRS) 2 01/16/2014 993754-14-807 Protection Set IV Hardware Requirements Specification (HRS) 2 01/16/2014 993754-11-809 Software Requirements Specification (SRS) 4 01/21/2014 993754-1-860 V&V Requirements Phase Summary Report 2 01/30/2014

3.2 PHASE TASKS (RE-ENTER OF DESIGN PHASE) The following are phase tasks associated with the PPSI design phase. Document review results are summarized in section 4.1 Results Summary and reported by the Document Review/Release (DRR) process. Various other document verifications listed in the SVVP are inherent to the documentation process.

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 9 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 10 of 44 Date: 01/15/2016

4.0 TASK RESULTS 4.1 RESULTS SUMMARY The tables below contain a summary of PPSI design phase task results, the documents associated with each task, the revision of each document reviewed, the DRR number where each review was documented, and the date each document was issued. The Document Review/Release (DRR) processes documented all required review for both initial releases and approved changes to the design phase documents during the design phase. Required V&V reviews of certain task outputs are noted in the task descriptions of Table 4-11a as per section 3.2 of this report.

Note: The changes on the MCL do not require the use of a DRR. The revisions processed without DRRs are reported for information purposes.

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 11 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 12 of 44 Date: 01/15/2016

4.2 TASK AND PHASE ITERATION When phase inputs are changed, or issues with phase outputs are discovered, tasks are required to be re-executed. Table 4-2a and 4-21a outlines the iterations, the date iterations were initiated, the reason the iterations were initiated and the repeated tasks and/or revised phase outputs. There is (2) iteration since last restart of the project design phase last February 2013.

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 13 of 44 Date: 01/15/2016

4.3 PHASE OUTPUTS The following table lists the final output documents for the Design Phase.

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 14 of 44 Date: 01/15/2016

5.0 DESIGN PHASE ANOMALIES Anomalies or deficiencies are documented on System Integration Deficiency Report (SIDR). Action Request Reports (ARRs) are used to document internal programmatic deficiencies associated with project activities that needs corrective or preventive action. Anomalies or deficiencies that were identified during the completion of Design phase tasks are listed in Table 5-1a.

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 15 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 16 of 44 Date: 01/15/2016

6.0 SOFTWARE QUALITY There are two categories of metrics considered in the phase summary reports to evaluate the software quality:

1. Software Quality Metrics– provides the metrics for evaluating the software development processes and products. These metrics are calculated using the defect count and defect density methods. Per the Invensys PPM [Ref.10.2.1], software documentation defects are classified as technical, editorial, or procedural.

Defect count is the number of defects in software products identified during the Design

phase. Defect density is the defect count divided by the number of opportunities for error (OFE).

2. V&V Effectiveness Metrics– provide the metrics for evaluating the degree of completeness and accuracy of the work executed by the IV&V organization. Note that effectiveness does not include efficiency and productivity, which relate to the resources consumed to execute the activity. These metrics measures the V&V review completeness and reporting accuracy. V&V Review Completeness is the percent of the phase software products that Nuclear IV&V

reviews. V&V Reporting Accuracy is the number of proven defects divided by the number of reported

defects. Acceptance Criteria: IV&V recommends proceeding to the next phase if no technical defects remain at the end of the current phase.

6.1 SOFTWARE QUALITY METRICS

6.1.1 Defect Count

Defect Count = number of defects identified in the SDD and PTM IV&V review

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 17 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 18 of 44 Date: 01/15/2016

6.1.2 Defect Density Defect Density = Defects (SDD) / Opportunities for Error (OFE)

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 19 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 20 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 21 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 22 of 44 Date: 01/15/2016

6.2 V&V EFFECTIVENESS METRICS

Software V&V effectiveness metrics are calculated using two methods:

1. V&V Review Completeness – the amount (%) of contained material in the document that has undergone software V&V review.

2. V&V Reporting Accuracy – the measure of the number of reported defects that are ultimately proven to be defects.

Acceptance Criteria: IV&V recommends proceeding to the next phase only if the review percentage is 100%.

6.2.1 V&V Review Completeness

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 23 of 44 Date: 01/15/2016

6.2.2 V&V Reporting Accuracy %Accuracy = # of Software Product Defects (Actual) x100%

# of Reported Product Defects (Potential Defects)

6.2.2.1 SDD V&V Reporting Accuracy

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 24 of 44 Date: 01/15/2016

6.2.2.2 PTM V&V Reporting Accuracy

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 25 of 44 Date: 01/15/2016

6.3 QUALITY EVALUATION SUMMARY Table 6-9a below lists the quality evaluation summary for the Design phase.

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 26 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 27 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 28 of 44 Date: 01/15/2016

7.0 SOFTWARE SAFETY METRICS The software safety effort and its effectiveness in the design phase are assessed in the metrics below. The safety metrics were outlined in the Software Safety Plan (SSP) [Ref. 10.2.7]. The safety metrics are divided into four categories to show the collected safety numbers and their justifications.

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 29 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 30 of 44 Date: 01/15/2016

8.0 TECHNICAL AND MANAGEMENT RISKS (RE-ENTER OF DESIGN PHASE)

All of the design phase tasks are complete, deficiencies are resolved, and output documents are issued.

Nuclear IV&V reviewed the Software Design Description (SDD) for Protection Set I Rev 1 based on IEEE 1012-1998 software requirements evaluation criteria and determined that the following Requirement-Phase open items are satisfied and closed:

1) Performance criteria for precision (e.g., precision of decimal places in floating-point calculations).

2) System, device, and software control that specify initialization of the Protection Set’s critical points (e.g., setting setpoints and tuning constants to initial values).

3) Logic, computational, and interface precision that specify truncation and rounding in floating-point computations and data sent to external applications.

All documents released by the Nuclear Delivery in the design phase have been correctly developed and updated in the MCL described in the Software Configuration Management Plan (SCMP) [Ref. 10.2.8] There are no technical issues associated with the design phase outputs or the phase completion.

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 31 of 44 Date: 01/15/2016

9.0 RECOMMENDATIONS ( RE-ENTER OF DESIGN PHASE) Software Quality Metrics – There are no technical defects remain unresolved V&V Effective Metrics – Review percentage is 100% Software Safety Metrics – There are no technical defects remain unresolved. Nuclear IV&V recommends exiting the Design phase.

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 32 of 44 Date: 01/15/2016

10.0 REFERENCES 10.1 INDUSTRY DOCUMENTS

10.1.1 IEEE 1012-1998, Standard for Software Verification and Validation

10.2 INVENSYS TRICONEX DOCUMENTS

10.2.1 Invensys Project Procedures Manual (PPM)

10.2.2 Software Verification and Validation Plan (SVVP), 993754-1-802

10.2.3 Project Management Plan (PMP), 993754-1-905

10.2.4 Software Quality Assurance Plan (SQAP), 993754-1-801

10.2.5 Safety Analysis (Criticality/Hazards/Risks/Interfaces), 993754-1-915

10.2.6 Technical Requirements List (TRL), 993754-1-801

10.2.7 Software Safety Plan (SSP), 993754-1-911

10.2.8 Software Configuration Management Plan (SCMP), 993754-909

10.2.9 Protection Set I Software Requirements Specification (SRS), 993754-11-809

10.2.10 Protection Set II Software Requirements Specification (SRS), 993754-12-809

10.2.11 Protection Set III Software Requirements Specification (SRS), 993754-13-809

10.2.12 Protection Set IV Software Requirements Specification (SRS), 993754-14-809

10.2.13 Master Configuration List (MCL), 993754-1-803

10.2.14 Validation Test Plan (VTP), 993754-1-813

10.2.15 Protection Set I Software Design Description (SDD), 993754-11-810

10.2.16 Hardware Design Description (HDD), all four Protection Sets, 993754-11-917

10.2.17 Project Traceability Matrix (PTM), 993754-13-804

10.2.18 Software Verification Test Specification (SVTS), 993754-13-869

10.2.19 Validation Test Specification (VTS), 993754-13-813

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 33 of 44 Date: 01/15/2016

11.0 APPENDICES The following appendices include supporting data used in the software quality metrics calculations.

Appendix 1: SDD Software Quality Metrics Data

Appendix 2: PTM Software Quality Metrics Data

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 34 of 44 Date: 01/15/2016

APPENDIX 1: SDD SOFTWARE QUALITY METRICS DATA 11.1a SDD Defect Count Data Defect Count for the SDD = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 35 of 44 Date: 01/15/2016

11.1b SDD Defect Density Data Defect Density (1) = Defects/Opportunities for Error(1)

Where: PTM OFE(1) = # of Design Elements

Density of Editorial defects (DDedit1) = Nedit / OFE(1)

Density of Procedural defects (DDproc1) = Nproc / OFE(1)

Density of Technical defects (DDtech1) = Ntech / OFE(1)

Density of all defects (DDtot1) = Ntot / OFE(1)

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 36 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 37 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 38 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 39 of 44 Date: 01/15/2016

APPENDIX 2: PTM SOFTWARE QUALITY METRICS DATA 11.2a PTM Defect Count Data Defect Count for the PTM = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 40 of 44 Date: 01/15/2016

11.2b PTM Defect Density (1) Data Defect Density (1) = Defects/Opportunities for Error(1)

Where: PTM OFE(1) = # of Traces

Density of Editorial defects (DDedit1) = Nedit / OFE(1)

Density of Procedural defects (DDproc1) = Nproc / OFE(1)

Density of Technical defects (DDtech1) = Ntech / OFE(1)

Density of all defects (DDtot1) = Ntot / OFE(1)

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 41 of 44 Date: 01/15/2016

11.2c PTM Defect Density (2) Data Defect Density (2) = Defects/Opportunities for Error(2)

Where: PTM OFE(2) = # of Cells

Density of Editorial defects (DDedit2) = Nedit / OFE(2)

Density of Procedural defects (DDproc2) = Nproc / OFE(2)

Density of Technical defects (DDtech2) = Ntech / OFE(2)

Density of all defects (DDtot2) = Ntot / OFE(2)

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 42 of 44 Date: 01/15/2016

11.2d PTM V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100 Ndefects = Number of software product defects (actual defects) Nrep = Number of reported product defects (potential defects)

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 43 of 44 Date: 01/15/2016

Document: 993754-11-861 Title: V&V Design Phase Summary Report PPSI Revision: 3 Page: 44 of 44 Date: 01/15/2016

12.0 ATTACHMENTS None