wecc cip-101 cip-002 mockaudit 09242013 · it manager & power operations manager ! 20 years...

CIP-‐101 September 24-‐25, 2013

(c) 2013 Dr. Joseph B. Baugh 1

Joseph B. Baugh, PhD, PMP, CISA, CISSP, CRISC, CISM

Senior Compliance Auditor – Cyber Security WECC: Vancouver WA Office

CIP-101: CIP-002 v3 to v5 Transition WECC Office: Salt Lake City UT

September 24-25, 2013

2

•  Dr. Joseph B. Baugh o  40 years Electrical Industry Experience

§  Transmission Lineman §  NERC Certified System Operator §  Information Infrastructure Design & Implementation §  IT Manager & Power Operations Manager §  20 years Information Technology & Security Experience §  Project Manager & IT Program Manager §  PMP, CISSP, CISA, CRISC, CISM, NSA-IAM/IEM certifications

o  17 years Teaching Experience (Multiple Schools) §  Degrees: PhD, MBA, BS-Computer Science §  Information Technology and IT Security courses §  Business Strategy, Leadership, & Management courses §  PMP, CISSP, CISA, CISM, ITIL, & Cisco certification prep

courses §  Project Management courses

Speaker Introduction

CIP-‐101 September 24-‐25, 2013


3

•  The WECC Cyber Security team has created a mythical Registered Entity, Billiam Power Company (BILL) and fabricated evidence to illustrate key points in the CIP audit processes.

•  Any resemblance of BILL to any actual Registered Entity is purely coincidental.

•  All evidence presented, auditor comments, and findings made in regard to BILL during this presentation and the mock audit are fictitious, but are representative of audit team activities during an actual audit.

WECC CIP-101 Disclaimer

4

•  Class Introductions o Name, Title, Organization, Interest in CIP-002

•  CIP-002-3 Mock Audit Overview •  Review CIP-002-5 Transition Guidance •  Review CIP-002-3 Requirements •  Review CIP-002 Team audit approach •  The BILL Mock Audit •  Questions

Agenda

CIP-‐101 September 24-‐25, 2013


5

•  BILL has identified and documented a list of Critical Assets through an application of the CIP-002-5 Impact Rating Criteria (IRC) per the recent NERC v5 Transition Guidance

•  BILL has identified associated Critical Cyber Assets

•  BILL requires a full Compliance audit on CIP-002-3 through CIP-009-3 o First week: Discovery phase at WECC offices o Second week: Compliance audit at BILL office

CIP-101 Mock Audit Overview

6

•  Mock Audit squeezes 2 weeks of audit activities into a few hours.

•  Sample DR’s •  Mock Interview •  Site Visits •  Use the RSAW as the guiding document •  Present and review evidence for each

requirement •  What do YOU think is the appropriate finding

for each requirement?

CIP-101 Mock Audit Overview

CIP-‐101 September 24-‐25, 2013


7

•  CIP-002-3 is the first step in the CIP Compliance trail •  All Registered Entities who perform the BA, GO, GOP,

LSE, TO, TOP, and/or TSP registered functions are required to be compliant with CIP-002-3.

•  CIP-002-5 replaces LSE with the DP function, TSP function drops out. o  However, for this mock audit, we are only using the

CIP-002-5 R1 and accompanying Attachment 1 IRC to identify and document a list of Critical Assets and remain compliant with CIP-00x-3, so the v3 functions are still valid.

•  Some entities find they are only required to be compliant with CIP-002-3 & CIP-003-3 R2. o  Typically requires a reduced scope audit that is conducted at

WECC offices or other locations as necessary.

CIP-002-3 Overview

8

•  R1: Identify and document a risk-based assessment methodology (the RBAM). o Include procedures and evaluation criteria

(R1.1) o Consider all BES Assets, pay close attention to

those assets listed in R1.2 (see R1.2.1 - R1.2.7).

Current CIP-002-3 Requirements: R1

CIP-‐101 September 24-‐25, 2013


9

CIP-002-3: R1, R1.1, R1.2

10

CIP-002-3: R1.2.1-1.2.7 Use these asset types

as represented by your inventory of BES

Assets in your application of the

CIP-002-5 IRC during the transition period.

CIP-‐101 September 24-‐25, 2013


11

•  Apply the RBAM to a list of your BES Assets to identify and document a list of Critical Assets.

•  Review the list of Critical Assets at least annually and update as necessary.

CIP-002-3 Requirements: R2

12

CIP-002-3: R2

CIP-‐101 September 24-‐25, 2013


13

•  Cyber Security Standards Transition Guidance (NERC, 2013 Sept 5, p. 2)

CIP v5 Transition Guidance

14

•  Cyber Security Standards Transition Guidance (NERC, 2013 Sept 5, p. 2)

•  BILL chooses Option 2 to identify and document a list of Critical Assets from its inventory of BES Assets.

•  The CIP Senior Manager documents this choice prior to implementation.

CIP-00x-5 Transition Guidance

CIP-‐101 September 24-‐25, 2013


BILL Documents Its CAID Choice

16

•  Each Responsible Entity shall implement a process that considers each of the following assets for purposes of parts 1.1 through 1.3: [Violation Risk Factor: High][Time Horizon: Operations Planning] o  i. Control Centers and backup Control Centers; o  ii. Transmission stations and substations; o  iii. Generation resources; o  iv. Systems and facilities critical to system restoration, including

Blackstart Resources and Cranking Paths and initial switching requirements;

o  v. Special Protection Systems that support the reliable operation of the Bulk Electric System; and

o  vi. For Distribution Providers, Protection Systems specified in Applicability section 4.2.1 above. (Not applicable for transition)

•  Ensure all asset types described in CIP-002-3 R1.2.1 through R1.2.7 are included in the above categories. If not, add them to the evaluation process (per slide 10).

CIP-002-5 Transition Changes

CIP-‐101 September 24-‐25, 2013


17

•  Each Responsible Entity shall implement a process that considers each of the following assets for purposes of parts 1.1 through 1.3: o  1.1. Identify each of the high impact BES Cyber Systems

according to Attachment 1, Section 1, if any, at each asset;

o  1.2. Identify each of the medium impact BES Cyber Systems according to Attachment 1, Section 2, if any, at each asset; and

o  1.3. Identify each asset that contains a low impact BES Cyber System according to Attachment 1, Section 3, if any (a discrete list of low impact BES Cyber Systems is not required).

•  CIP-002-5 R1.1-R1.3 are not applicable for the transition period.

CIP-002-5 Transition Changes

18

•  After identifying and documenting a list of Critical Assets by applying the IRC to BILL’s inventory of BES Assets, the process reverts to the current mandatory and enforceable CIP-002-3 R3 processes.

•  Use the list of Critical Assets (CA’s) developed by applying CIP-002-5 R1 and the IRC to develop a list of Cyber Assets associated with each Critical Asset and apply the current Critical Cyber Asset Identification [CCAID] methodology to determine if any Cyber Assets are essential to the operation of the Critical Asset.

CIP-002-3 R3

CIP-‐101 September 24-‐25, 2013


19

CIP-002-3: R3

20

•  For each such Cyber Asset that is deemed essential, consider: o R3.1: Does it use a routable protocol to

communicate outside the ESP? or o R3.2: Does it use a routable protocol within a

control center? or o R3.3: Is it dial-up accessible? o  If any of the above are true, the Cyber Asset is a

CCA. •  Review the list of CCAs at least annually and

update as necessary.


CIP-‐101 September 24-‐25, 2013


21

•  The senior manager or delegate (as defined in CIP-003-3 R2) must approve at least annually: o The RBAM (not applicable under Option 1 or 2) o The list of Critical Assets o The list of CCAs, even if such list is null.

•  The entity may determine it has no Critical Assets or associated CCAs

•  The entity must maintain signed and dated records of the approvals listed above.


22

CIP-002-3: R4

CIP-‐101 September 24-‐25, 2013


23

•  Audit to the Standard. •  Review the Evidence:

o  Current RBAM o  Current list of Critical

Assets o  Current list of CCAs, even

if such list is null. o  Records of current and

prior approved versions of the above documents (the Bookends)

•  DR for additional information, as needed.

CIP-002-3 Audit Team Approach

Review the application of the IRC to identify and document a list of Critical Assets

24

•  Use a methodical approach to deliver consistent results across all entities

•  Use the RSAW supplied by the entity as working papers to document the audit and findings

•  Review Initial Evidence package supplied by the entity o Attachment G

WECC Audit Team Approach

CIP-‐101 September 24-‐25, 2013


25

Initial Evidence: Attachment G Prior to the selection of an option, provide all versions of the RBAM in force during the audit period up to the date of selection. After a transition option is declared, entities should attach a copy of the CIP Senior Manager statement and the annual application of either the BLC or the IRC – depending on choice – in lieu of the RBAM.

26

•  Submit Data Requests (DR’s) for any additional information that will support the entity’s compliance efforts, e.g.: o One-line diagrams (we’ll see the BILL one-line

later) o Prior documentation to provide bookends o Initial list of Cyber Assets at each Critical Asset

identified in R2. o Address any questions or concerns


CIP-‐101 September 24-‐25, 2013


27

•  Review the RBAM or application of the IRC (R1), list of CA’s (R2), lists of CCA’s, even if such lists are null (R3)

•  If full Compliance audit: o  Hold interviews with the entity’s CIP SMEs o  Site visits (Trust, but Verify)

•  Validate annual approval documentation (R4) •  Submit DR’s, if needed, to clarify compliance •  Determine findings (NF, PV, or OEA) •  Discuss findings with entire Cyber Security Team •  Complete RSAW •  Prepare CIP audit report (ATL & CPC)


28

•  Walk through audit process in more detail •  Explain the differences between a reduced

scope off-site audit and a full Compliance audit

•  The Mock Audit simulates a Compliance audit of Billiam Power Company [BILL]

•  BILL is registered with NERC as a BA, GO, GOP, LSE, TO, TOP, TP, and TSP.

CIP-101 Mock Audit

CIP-‐101 September 24-‐25, 2013


29

•  Received from the entity in the initial evidence package

•  Response to data requests in Attachment G •  Information contained in entity response to the

RSAWs •  Sets the stage for the initial audit review

o Discovery phase at the WECC offices •  Followed up by additional Data Requests as

needed

Review Initial Evidence

30

•  Billiam Power Company’s (hereafter referred to by its NERC acronym, BILL) Balancing Authority (BA) area is effectively within the boundaries of the three counties on the western edge of Some State, bordered by Another State on the north and the Almost Mountains on the East and South. These three counties occupy about 15% of the land area of the state and contain about 20% of the state's population.

•  BILL is registered as a BA, DP, GO, GOP, LSE, TO, TOP, TSP

The BILL System (from entity report)

CIP-‐101 September 24-‐25, 2013


31

•  BILL’s primary generation station is located in eastern Whatchamacallit County. The BILL generation station has two 1,000 MW fossil fuel generating units. The output of these units supports BILL’s native load and any available excess energy is marketed throughout the WECC Interconnection.

•  BILL owns and operates nine Combustion Turbines (averaging 30 MWs each) located near various consumer load centers throughout the service territory. These CT’s are primarily used as peaking units and for voltage and frequency support during the summer months.

•  BILL also owns and operates the BILL-3 Hydroelectric plant on the Sweet William River. BILL-3 has a nameplate rating of 100 MW. This hydro unit is Blackstart capable and is connected to the BILL Generation Station through a dedicated 115 kV line that runs 87 miles from Sub3 to Sub1.

•  Total BILL generation capacity is 2,380 MWs.

The BILL System (continued)

32

•  There are two synchronous 345 kV interties with adjacent BA’s that define the BILL BA area. These ties are with XXXX Electrical Utility and YYYY Federal Power District at Sub1, which is adjacent to the BILL Generation Station. The BES portion of BILL's BA area, its 345 kV, 230 kV, and 115 kV facilities, include 190 miles of 345 kV transmission lines, 450 miles of 230 kV lines, and 973 miles of 115 kV lines. BILL owns and operates two 345kV substations, 25 230 kV substations, and 52 115 kV substations throughout its service territory. BILL serves its native residential and commercial load through its 115 kV and 230 kV transmission facilities.

•  The Generation and Transmission facilities are monitored and managed from the Primary Control Center (PCC) located at the corporate headquarters in Big Bill City. BILL also maintains a hot stand-by Back-up Control Center (BUCC) located in its operations center in Little Bill City, which is approximately 50 miles from the PCC.

•  BILL is a summer peaking BA and BILL's BA all-time area peak load was recorded on July 20, 2010 at 2,482 MWs.

The BILL System (continued)

CIP-‐101 September 24-‐25, 2013


33

BILL One-Line Diagram

34

•  The first step in a normal CIP-002-3 audit is to review the RBAM.

•  The second step is to review the Critical Asset Identification Methodology [CAID].

•  The CAID is typically included as part of the RBAM, but the audit team will review the application of the IRC under this scenario o  Starts with an overall list of entity BES Assets. o  Uses the IRC to identify and document a list of Critical Assets.

•  Review BILL’s 2013 list of Critical Assets derived from the IRC and compare it to the previous lists derived from the RBAM.

•  Were applicable BES Assets evaluated relative to IRC criteria 2.3. 2.6. or 2.8? [If Option 1 selected, then 1.3, 1.8, 1.9, 1.10] o  Did BILL demonstrate coordination with the applicable registered

function(s)? o  If not, should we submit a data request?

BILL’s Critical Asset Identification

CIP-‐101 September 24-‐25, 2013


35

BILL BES Assets: 2012 Control Centers

36

BILL BES Assets: 2013 Control Centers

CIP-‐101 September 24-‐25, 2013


37

BILL BES Assets: 2012 Substations

38

BILL BES Assets: 2013 Substations

CIP-‐101 September 24-‐25, 2013


39

BILL BES Assets: 2012 Generation

40

BILL BES Assets: 2013 Generation

CIP-‐101 September 24-‐25, 2013


41

BILL BES Assets: 2012 Special Systems

42

BILL BES Assets: 2013 Special Systems

CIP-‐101 September 24-‐25, 2013


43

BILL BES Assets: 2012 Critical Assets

44

BILL BES Assets: 2013 Critical Assets

CIP-‐101 September 24-‐25, 2013


45

•  Control Centers o No change

•  Substations o Add 4 (Subs 4, 7, 8, 11) o Drop 1 (Sub 3, related to blackstart)

•  Generation Units o Drop blackstart unit

•  Special Protection Systems o No change

2012-2013 Critical Assets – Net Changes

46

•  Did BILL apply the IRC appropriately? •  Does BILL need to confer with its RC, PA, or TP to

consider any Critical Assets relative to Criteria 2.3, 2.6, or 2.8?

•  Did BILL review its list of Critical Assets at least annually?

•  Did BILL update the list as necessary? •  Application Questions

o  Did BILL consider all BES Assets in R1.i through R1.vi? o  Did BILL review and evaluate all BES Assets through the

IRC? o  Did BILL clearly identify and document all Critical Assets?

•  Is any additional information necessary? o  If so, do we submit a DR?

R2: Critical Asset Review Questions

CIP-‐101 September 24-‐25, 2013


47

•  The third step in a CIP-002-3 audit is to review the Critical Cyber Asset Identification Methodology [CCAID].

•  Under this scenario, the CCAID should be maintained as a discrete document.

•  Starts with the identified list of Critical Assets. •  Uses the CCAID procedures and evaluation criteria

to identify and document a list of Critical Cyber Assets, even if such list is null.

•  Review the BILL Critical Cyber Asset Identification Methodology

•  Review List of Critical Cyber Assets

BILL’s Critical Cyber Asset Identification

48

2012 CCAs: Primary Control Center

CIP-‐101 September 24-‐25, 2013


49

2013 CCAs: Primary Control Center

50

2012 CCAs: Backup Control Center

CIP-‐101 September 24-‐25, 2013


51

2013 CCAs: Backup Control Center

52

2012 CCAs: SUB1

CIP-‐101 September 24-‐25, 2013


53

2013 CCAs: SUB1

54

2012 Null Lists CCAs: Generation & Subs

CIP-‐101 September 24-‐25, 2013


55

2013 Null Lists CCAs: Generation & Subs

56

•  Did BILL use the Critical Asset list developed in R2 to identify Critical Cyber Assets?

•  Did BILL apply its Critical Cyber Asset Identification Methodology [CCAID] appropriately to consider all Cyber Assets supporting the reliability function of the Critical Asset?

•  Did BILL review the list at least annually and update the list as necessary?

•  Application Questions o  Did BILL consider all Cyber Assets located at its Critical Assets for

evaluation through the CCAID? o  Did BILL consider R3.1-R3.3 for all Cyber Assets considered

essential to the operation of the Critical Asset o  Did BILL clearly identify and document all Critical Cyber Assets?

•  Are any DR’s necessary? o  If so, what additional information is required?

R3: Critical Cyber Asset Review Questions

CIP-‐101 September 24-‐25, 2013


57

•  The fourth step in a CIP-002-3 audit is to review the annual approvals of the RBAM, the list of Critical Assets, and the lists of Critical Cyber Assets, even if such lists are null.

•  Review the BILL 2012 Annual Approvals •  Review the BILL 2013 Annual Approvals

BILL’s Annual Approvals

58

•  Did the BILL CIP Senior Manager or delegate approve at least annually the RBAM, the list of Critical Assets, and the lists of Critical Cyber Assets, even if such lists are null?

•  Application Questions o Did BILL provide evidence of annual reviews and

approvals? •  Are any DR’s necessary?

o  If so, what additional information is required?

R4: Annual Approval Review Questions

CIP-‐101 September 24-‐25, 2013


59

•  Set up through an interview DR the prior week •  Typically held on Monday of the on-site week

immediately after the opening presentation •  Examines the entity’s understanding of and

approach to R1-R4 •  Cover any areas of concern raised through the

initial evidence review •  Schedule follow-up interview(s), if needed,

after the site visits

On-Site Activities: The Interview

60

•  Need four volunteers o You are BILL SMEs o No, you don’t get to practice

•  We will ask a series of questions that we generally ask all CIP-002 SMEs

•  Also ask questions of concern, if indicated by the initial review of the evidence

•  The Interview Question Set

On-site activities: Mock Interview

CIP-‐101 September 24-‐25, 2013


61

•  What did we learn from the interview? •  What was the key issue from an audit

perspective? •  Should we find a PV for this issue? •  Why or why not?

On-site activities: Mock Interview

62

•  Set up through a site visit DR the prior week •  Itinerary determined through review of the

initial evidence •  Trust, but verify. Why? •  Depending on entity size, 100% validation or a

statistical sampling •  Where?

o Control Centers o Generation Facilities o Transmission Facilities

On-Site Activities: Site Visit

CIP-‐101 September 24-‐25, 2013


63

•  Who? o CIP-002-3 Sub-Team

§  Validates lists of CCAs, even if such lists are NULL §  Works in conjunction with CIP-005-3a sub-team

o CIP-005-3a Sub-Team §  Validates Electronic Access Points [EAPs] and Electronic

Access Control and Monitoring devices [EACMs]. §  Confirms ESP boundaries

o CIP-006-3c Sub-Team §  Validates PSPs and Physical Access Controls, such as

PACS, cameras, logs, etc. § My colleague, Wally Magda, provided an overview on

CIP-006-3c audit activities earlier.


64

•  What? o Validate lists of CCAs o Validate null lists of CCAs o Look for aberrations from the lists o Hold informal interviews with entity SMEs

•  When? o Sometimes during the off-site week. o Typically on Tuesday of the on-site audit o May also be on Wednesday depending on sites

visited, distances traveled, etc.

On-Site Activities: CIP-002-3 Site Visit

CIP-‐101 September 24-‐25, 2013


65

•  Visit the Primary and Backup Control Centers o 100% validation of CCAs in both locations o Talk to Operators & SMEs

•  Visit the BILL Generation Station, SUB1, SUB2, SUB4, SUB7, SUB8, and SUB11. o Validate the Null Lists of CCAs o Talk with entity SMEs

•  Site Visit Questions o Why validate all CCAs at a given site? o Why validate Null lists of CCAs? o Why ask questions of entity SMEs?

On-Site Activities: BILL Site Visits

66

•  Visited the Primary Control Center o 100% validation of CCAs o Found nothing out of the ordinary.

•  Visited the Backup Control Center o 100% validation of CCAs o Found nothing out of the ordinary.

BILL Site Visits: Control Centers

CIP-‐101 September 24-‐25, 2013


67

•  Visited BILL Generation Station o Validated Null list of CCAs o Found nothing out of the ordinary.

Site Visits: Generation Units

68

•  Visited Sub1 o 100% validation of CCAs o Found nothing out of the ordinary.

•  Visited Sub2 o Validated Null list of CCAs o Noticed something strange here.

•  Visited Subs 4, 7, 8, & 11 o Validated Null list of CCAs o Noticed something strange at each of these

substations too.

Site Visits: Substations

CIP-‐101 September 24-‐25, 2013


69

Site Visits: What Did We See? What is this device and what is it doing here in the subs?

70

•  What did we learn from the site visit? •  Why do we validate Null lists of CCAs? •  What was the main concern with the

unexpected devices? •  Should we DR for additional information?

o Tour Notes DR •  Would another interview be more effective? •  Does this situation call for an R3 PV finding? •  Why or why not?


CIP-‐101 September 24-‐25, 2013


71

•  Discuss with whole Cyber Security Team •  Is there a PV for the undocumented devices?

o R2: Undeclared Critical Assets §  The Combustion Turbines §  Does the entity have documentation from its TP or PA/PC that

exempts the CTs from Criterion 2.3? o R3: Undeclared Critical Cyber Assets

§  The Substation Modems •  Determine the scope of a PV

o How do we do this? •  Complete the CIP-002-3 Findings Table in RSAW •  Submit to the ATL and CPC for the Closeout

Presentation

Discussing the Findings

72

•  WECC Audit Teams never Prescribe Solutions, but we do describe: o Brief entities on findings o Encourage good security practices o Discuss examples of industry best practices o  Identify areas of concern, which may not be

violations, but which could stand improvements o Provide suggestions, when appropriate

•  Support development of a sustainable compliance culture

Value-Added Activity: Feedback

CIP-‐101 September 24-‐25, 2013


73

Audit Documentation: The RSAW

•  An auditor is judged by the quality of his or her working papers. o Complete the

RSAW o Document findings o DR for any final

needed information

74

•  Auditors review evidence, find facts, and report findings o Turn PVs over to the Enforcement team o Enforcement team depends heavily on the

quality of auditor documentation •  Be Literate, be Concise, but above all

else, Be Accurate. •  If it’s not written down, it didn’t happen.

Audit Documentation

CIP-‐101 September 24-‐25, 2013


75

•  The Audit Report o Work with ATL & CPC o Verify findings and other information related to

audited standard(s) •  Document findings in webCDMS

o PV & OEA findings only •  Work with WECC Enforcement personnel to

support Investigations as SME for audit processes and findings

Post-Audit Auditor Activities

76

•  Participate in entity Outreach activities, such as this event and CIPUG meetings

•  Be available to address entity questions/comments

•  Work at National level o CCWG o Drafting teams o Comment on new Standards, CANs, etc. o Attend and present at conferences

Post-Audit Auditor Activities

CIP-‐101 September 24-‐25, 2013


77

Summary

•  Audit to the Standard •  Provide useful feedback to the entity •  Prepare a valid report •  Be available to CIP personnel at the

entities •  Work at National level

78

Remember the Auditor’s Mission

Just the facts, Ma’am,

Just the facts!

CIP-‐101 September 24-‐25, 2013


79

•  NERC. (2013 September 5). Cyber Security Standards Transition Guidance (Revised). Retrieved from http://www.nerc.com/pa/comp/Resources/ResourcesDL/Cyber%20Security%20Standards%20Transition%20Guidance%20(Revised).pdf

References

Joseph B. Baugh, Ph.D., PMP, CISA, CISSP, CRISC, CISM

Senior Compliance Auditor - Cyber Security Western Electricity Coordinating Council (WECC)

7400 NE 41st Street, Suite 160 Vancouver, WA 98662

jbaugh (at) wecc (dot) biz (C) 520.331.6351 (O) 360.567.4061

Questions?