A T ti l A h t NERC CIP 010 5A Tactical Approach to NERC CIP-010-5 Compliance

Walter Sikora, Vice President Security Solutions of Industrial Defender

Remote Monitoring and Control Conference & ExpoRemote Monitoring and Control Conference & ExpoSeptember 18-19, 2012

Denver, CO

Abstract

NERC has moved quickly to address shortcomings and lack of clarity in previousshortcomings and lack of clarity in previous versions of CIP standards. While this was a positive move, overall, it also presents some

i h ll f A t O Thiunique challenges for Asset Owners. This presentation will briefly cover changes and challenges presented with NERC CIPv5 adoptionchallenges presented with NERC CIPv5 adoption and deliver tactical steps that Asset Owners can take to fulfill requirements and achieve continuous

li h dd i CIP 010 “C bcompliance when addressing CIP-010-5 “Cyber Security – Configuration Management and Vulnerability Assessment”

8/16/2012 2

Vulnerability Assessment .

A Tactical Approach to Compliance for CIP-010-5

• FERC issues order approving NERC CIP-002-4 on 4/20/2012– 4/1/2014

• For version 5, the latter of: – NERC CIP version 5 effective date July 1, 2015– Or first calendar day of the ninth calendar quarter after the effective y q

date of the order providing applicable regulatory approval (2.5yr)• Timeline

– Now: Q2 2012/Q3 2012Now: Q2 2012/Q3 2012– In-between: ???– Ready for compliance: Q3 2014

Eff ti D t Q3 2015– Effective Date: Q3 2015• 2016?

8/16/2012 3

NERC CIP v4 v. NERC CIP v5

Source: http://www.nerc.com/docs/standards/sar/Webinar_Slides-Project_2008-06-April_10,_2012.pdf

Version 4 Version 5e s o e s o 542 requirements; 113 parts 37 requirements; 148 partsNo contextual information Includes background, rationale, and

guidelines and Technical Basis

Measures on high level requirementonly

Measures for each requirement, including parts

14 requirements with Technical 12 requirements with TFE triggering qFeasibility Exception (TFE) triggering

language

q gg glanguage

Undefined periodic terms Clear periodic requirements: initial requirements in Implementation Plan

8/16/2012 4

requirements in Implementation Plan

Many binary Violation Severity Levels (VSLs)

More gradated VSLs

Implications of the Changes

Change ImplicationLess requirements, more parts More clarity, more coverageIncreased context, background, rationale

More clarity

Measures for each requirement and t

More clarity, more examplespartLess potential for TFEs More mitigation, workarounds, or

additional solution(s)Cl l d fi d i di t M l it l f i diClearly defined periodic terms More clarity, less room for periodic

errorsMore gradated VSLs More flexibility

8/16/2012 5

High, Medium, and Low Impact

• CIP-002-5 Attachment 1– Impact Rating Criteria

R t BES C b S t b– Rates BES Cyber Systems by:• High Impact Rating (H)• Medium Impact Rating (M)• Low Impact Rating (L)Low Impact Rating (L)

– Criteria listed for each rating level– Rating level will determine which requirements/sub-requirements a BES

Cyber System owner will have to meety y– Rating level(s) are associated with each requirement

8/16/2012 6

Strategic Goals for complying with CIP-010-5

• High-level strategy– Continuous compliance– Reduce overhead– Improved resource allocation

• How do you get there?y g– Are you there yet?

8/16/2012 7

Tactics for CIP-010-5

• Gain visibility– Be EVERYWHERE or at least in a lot of places

Redundant and supporting information– Redundant and supporting information– Passive – listen on the network– Active – local or remote interaction with asset

Use a tomated data collection• Use automated data collection– Automatic data collection– Minimum – a script requiring manual initiation

Id l h d l d t t d i f ti th i– Ideal – scheduled, automated information gathering• Automate Analysis

– Answer requirements, capture decisions and prioritiesL t th t d h d k– Let the computers do your hard work

• Tie into Workflow– Tie analysis results back into system performance and work tasks

R di t d fi i i

8/16/2012 8

– Remediate any deficiencies

Quick Review of CIP-010-5 – Configuration Management and Vulnerability AssessmentsVulnerability Assessments• R1 – Configuration Change Management

– R1.1 – Baseline configuration for:• OSOS• Commercial or Open-source Application(s)• Custom software• Logical network accessible ports• Security Patches

– R1.2 – Authorize and document changes that deviate from the existing baseline configuration

– R1 3 – For deviations update baselines and documentation required by CIP-007– R1.3 – For deviations, update baselines and documentation required by CIP-007 and CIP-005 as necessary within 30 calendar days

– R1.4 – For changes that deviate• R1.4.1 – Determine potentially impacted cyber security controls prior to change• R1.4.2 – After change, verify controls are not adversely affected• R1.4.3 – Document results of verification

8/16/2012 9

Quick Review of CIP-010-5 – Configuration Management and Vulnerability Assessments continuedVulnerability Assessments - continued• R1 – Configuration Change Management

– R1.5 – For each change that deviates from the baseline• R1.5.1 – Test changes in test environment to ensure no adverse affectsR1.5.1 Test changes in test environment to ensure no adverse affects• R1.5.2 – Document test results and test v. prod variations.

• R2 – Configuration Monitoring– R2.1 – Where technically feasible, monitor continuously or periodically (not greater

than once every 35 calendar days) for changes to baseline. Document and investigate unauthorized changes

• R3 – Vulnerability Assessments– R3.1 – Conduct a paper or active vulnerability assessment to determine extentR3.1 Conduct a paper or active vulnerability assessment to determine extent

cyber security controls are implemented correctly and operating as designed (calendar year to 15mo)

– R3.2 – Perform active vulnerability assessment in test or production R3 3 Active vulnerability assessment for new Cyber Asset– R3.3 – Active vulnerability assessment for new Cyber Asset

– R3.4 – Document results and plan to remediate or mitigate findings

8/16/2012 10

Tactical ReferenceAutomatedAutomated

Hard, Automated

Easy, Automated

EasyHard

Hard, Manual

Easy, ManualManual Manual

8/16/2012 11Manual

R1.1 – Develop Baseline Configuration

• Baseline creation– Bootstrap from existing baseline(s)– Bootstrap from existing production assetsBootstrap from existing production assets– Bootstrap from representative test or development environment

• Enumerate– OS info & version

• E.g. Windows: psinfo, systeminfo, Linux: uname– All installed software

• Windows: wmic product list full, Linux: rpm -qaLogical network accessible ports– Logical network accessible ports

• Windows: netstat –anb, Linux: netstat -an• Filter out localhost

– Security patchesSecurity patches• Windows: wmic qfe list full, Linux: rpm –qa

• Check baseline validity across all assets• Baselines are set in stone, except when they’re not

8/16/2012 12

R1.1 – Develop Baseline Configuration Tactics

• Option 1– Manually run commands and collect on every asset

Option 2• Option 2– Manually initiated, scripted data collection run locally on assets

• Option 3Man all initiated scripted data collection r n remotel on assets– Manually initiated, scripted data collection run remotely on assets

• Option 4– Fully automated, “scripted” data collection run remotely or locally on assets

P t ti l Pitf ll• Potential Pitfalls– Synchronous v. asynchronous data collection– Devices without or with limited remote, interactive interfaces

D t ll ti i j t th fi t t– Data collection is just the first step

8/16/2012 13

R1.2 – Authorize and document changes that deviate from the existing baseline configurationthe existing baseline configuration• R1.1 Baselines compared to a baseline check gathered AFTER R1.1

baseline is “final”H d d t i h t th h ?• How do you determine what the changes are?– This is where Configuration Change Management starts getting

“hard”– Gathering the baseline data is easy. Checking and verifying it

against a baseline, across time is hard.• How do you monitor and find deviations for settings which have a

range of acceptable values? (E.g. dynamic ports)• How do you prioritize and tie your results back into your workflow?

8/16/2012 14

R1.2 – Authorize and document changes that deviate from the existing baseline configuration Tacticsthe existing baseline configuration - Tactics• This is where “Do It Yourself” starts to get hard• Option 1

Manually and visually compare asset configuration output– Manually and visually compare asset configuration output• Option 2

– Manually instantiate programmatic comparison of asset configuration outputoutput

• Option 3– Automatically instantiate programmatic comparison of asset configuration

outputoutput• Option 4

– Results from Option 3 get output in a review queue with differences highlighted for easy consumptiong g y p

• Option 5– Option 4 results that get prioritized by asset and/or system criticality

8/16/2012 15

R1.3 – Update the baseline configuration and other documentation within 30 calendar daysdocumentation within 30 calendar days• By now, if you are doing this internally without dedicated software

development you are spending a lot of time on reviewing and updating baselinesbaselines– How many other jobs/tasks do your resources have to complete in that 30

day period?– What about their “normal” jobs?j– Most organizations have resources with technical, process-related jobs

that ALSO double has OT/Compliance/Tech Support– See Industrial Defender survey*

• 30 day timeline is the key– How much of that 30 day timeline is taken up by…

• Updating and testing your new baseline?• Updating your other documentation?• Determining all of the disparate sources of documentation which need

updating

8/16/2012 16

R1.3 – Update the baseline configuration and other documentation within 30 calendar days Tacticsdocumentation within 30 calendar days - Tactics• Recognize these options?• Option 1

– Manually run commands and collect on every assetManually run commands and collect on every asset• Option 2

– Manually initiated, scripted data collection run locally on assets• Option 3

– Manually initiated, scripted data collection run remotely on assets• Option 4

– Fully automated, “scripted” data collection run remotely or locally on assetsOption 5• Option 5

– Update your actual baseline files in your central repository• Potential Pitfalls

– Synchronous v. asynchronous data collectionSynchronous v. asynchronous data collection– Devices without or with limited remote, interactive interfaces– Data collection is just the first step

8/16/2012 17

R1.4 – For deviations, determine security controls and determine any adverse affectsdetermine any adverse affects• Make your control list and check it twice• How many tools will it take to gather all of that data?• Do your existing tools adequately gather data for analysis?• How do you test each individual control?• Do you test each individual control?y• Ensuring working controls means demonstrating failure

– How do you introduce failure without affecting production?– What existing sources of failure can you draw from? Without introducingWhat existing sources of failure can you draw from? Without introducing

additional failure?

8/16/2012 18

R1.4 – For deviations, determine security controls and determine any adverse affects Tacticsdetermine any adverse affects - Tactics• Compiling & comparing control lists

– Option 1: Gather all enabled controls from all cyber assets– Option 2: Start with NIST control list, remove what isn’t in your

environment– Compare your list vs. what is required in all CIPs

• Verify that controls are not adversely affected– Option 1: Manually evaluate for any differences between pre &

post-change configuration settingsp g g g– Option 2: Manually driven, programmatic evaluation of differences– Option 3: Automatic, programmatic evaluation of differences

• Document results• Document results– Option 1: Manual documentation– Option 2: Automatic documentation

8/16/2012 19

R1.5 – For deviations, test changes in test environment, document results document variances in environmentsdocument results, document variances in environments

• Establish your test environment– Shouldn’t be the same thing as your “backup” workstation/server– Leverage virtualization if you can

• Greater flexibility and ability in “test” environment the closer to y yproduction you can be– Hint: deploy your baselines to your test environment

8/16/2012 20

R1.5 – For deviations, test changes in test environment, document results & variances in environments Tacticsdocument results & variances in environments - Tactics

• Option 1: Using “backup” machines a “test” environmentO ti 2 D li t d ti i t i li it d h d• Option 2: Duplicate production environment in limited hardware

• Option 3: Virtualize test environment with images from production or from vendor

• All options then require:– Before/After configuration settings gathering process– Before/After results comparisonp– Results demonstrating adverse or not affects on cyber security

controls– Documentation of differences and test resultsDocumentation of differences and test results– Are you able to account for all differences between test and

production?

8/16/2012 21

R2.1 – Configuration Monitoring, monitor continuously or periodically (35 day window) for changes documentperiodically (35 day window) for changes, document, investigate• Manual completion will take place 12 times a yearp p y

– Can you gather all of the configuration data, analyze, investigate, and document in one day a month?

– How many days? – 2 days? 5 days? 7 days?

• Automatic completion can take place 12 or 24 times a day

8/16/2012 22

R2.1 – Configuration Monitoring, monitor continuously or periodically (35 day window) for changes documentperiodically (35 day window) for changes, document, investigate - Tactics

O ti 1• Option 1– Manual, visual confirmation per device

• Option 2– Manual, visual confirmation via remote administration

• Option 3– Programmatic monitoring manual monitoringProgrammatic monitoring, manual monitoring

• Option 4– Programmatic monitoring, programmatic monitoring

O ti 5• Option 5– Option 4 with changes automatically brought into workflow and

documentation

8/16/2012 23

R3.1 – Calendar year/15mo, paper or active VA

• Paper Vulnerability Assessments are a bad idea– Unless it’s your only real option

A d th it’ till b d id– And even then, it’s still a bad idea• Active Vulnerability Assessments are the way to go

– Can be performed in a controlled manner– Can include the use of credentialed methods– Credentialed methods turn vulnerability scanners into administrative tools

8/16/2012 24

R3.1 – Calendar year/15mo, paper or active VA - Tactics

• Use a tool(s)– Vulnerability scanners galore, find some, get to know them and how to use

themthem– Get the most bang for your buck but it’s likely that “one size DOES NOT fit

all”– Use credentialed scans wherever possible!Use credentialed scans wherever possible!– Progressively and incrementally scan your network and cyber assets

• DO NOT scan everything at once• DO NOT scan all cyber assets of a particular type at onceDO NOT scan all cyber assets of a particular type at once• DO NOT scan redundant assets at the same time• DO NOT scan embedded or legacy devices in production• DO NOT give your scanned unlimited resourcesDO NOT give your scanned unlimited resources

• Ensure your tools are actually testing your controls! (Not just looking for vulnerabilities)

8/16/2012 25

R3.2 – Every 36 calendar months, active VA on test or production environmentproduction environment• Active Vulnerability Assessments are the way to go (remember this?)

– Can be performed in a controlled manner– Can include the use of credentialed methods– Credentialed methods turn vulnerability scanners into administrative tools

• Test environments– Full-scale test environments mirroring production are rarely seen– In once case, this test environment doubled as a training environment and was shared by

multiple sites– Most test environments usually looks like one or more machines with all of the environment’s

software installed• Production environments

– Modern OSes (XP and newer) can handle an active VA with negligible overhead on the cyber assets

– However, if something will fall over during an active VA it is likely:• Vendor services• Legacy device (including some network gear)• Embedded device (including some network gear)

8/16/2012 26

R3.2 – Every 36 calendar months, active VA on test or production environments Tacticsproduction environments - Tactics• Buy a vulnerability scanning application and learn how to use it

– Make sure it can do credentialed scanning (most can)• Perfect practice makes perfectly repeatable results

– Understand what your scanner’s options do and how they work• Test potentially unreliable devices and services in test firstp y

– If production is your only option, limit testing to one of your assets of that type. NOTE: don’t scan a PLC/embedded/non-redundant device in production!p

– Set your backup PLC/embedded/non-redundant device up on a test bench and scan

8/16/2012 27

R3.3 – Active VA on new Cyber Assets (except “Exceptional Circumstances” or replacements w/ existing baseline config)Circumstances or replacements w/ existing baseline config)

• “Like replacements”Id ll hi l k f– Ideally, this means less work for you

– BUT:• You already know how to do active vulnerability assessments…• The device hasn’t been deployed to production yet…• So, why not?

– Make active VA of your replacement assets a part of your deployment process

• Requirement description reads like these scenarios apply:– New types of assets

“Lik l t ” th t d ’t/ ’t h t d b li d l d t– “Like replacements” that don’t/can’t have a stored baseline deployed to them prior to production deployment

8/16/2012 28

R3.3 – Active VA on new Cyber Assets (except “Exceptional Circumstances” or replacements w/ existing baseline config)Circumstances or replacements w/ existing baseline config) -Tactics

• Make active VA a part of your asset deployment process• Utilize active VA tools and techniques• It’s ok to knock over a device or service in a test environment• It s ok to knock over a device or service in a test environment

8/16/2012 29

R3.4 – Document Assessment results and develop action plan for remediation/mitigationplan for remediation/mitigation

• You’ve been doing this since v3• NERC’s Find, Fix, Track (FFT) is probably relevant here

– FFT applies to low-risk “potential violations”– FERC has conditionally accepted NERC’s FFT proposal which is aimed

at increasing reliability of the Bulk Electric System, reducing auditing overhead and allowing Registered Entities the ability to self-report and mitigate low-risk “Potential Violations”. “Potential Violations” will follow a “Find Fix and Track” lifecycle to demonstrate remediationFind, Fix, and Track lifecycle to demonstrate remediation.

8/16/2012 30

R3.4 – Document Assessment results and develop action plan for remediation/mitigation Tacticsplan for remediation/mitigation - Tactics

• Good record keeping– Tracking:

• Results of the review or assessment• Action items

Documented proposed dates of completion• Documented proposed dates of completion• Status of action items

8/16/2012 31

Guidelines and Technical Basis

• Where “test environment” mentioned…– “Model” the baseline configuration not duplicate it exactly

All f bi ti f l / t d th lti t f– Allows for combination of roles/assets and the resulting set of characteristics

• Automated monitoring is the intentB t k ll f h th t i t ibl– But makes allowances for cases where that is not possible

8/16/2012 32

Guidelines and Technical Basis

• Paper Vulnerability Assessment– Basically a v3 VA

N t k di ( i f AP t th ESP)– Network discovery (review of APs to the ESP)– Network port and service identification– Vulnerability review

Wi l i– Wireless review• Active Vulnerability Assessment

– Paper VA performed with “active” and “scanning” tools

8/16/2012 33

Complete Story

• My organization doesn’t have|has change control.• My organization cannot|can detect change• My organization cannot|can accurately assess change• My organization cannot|can accurately address & manage change• My organization’s ability to perform and manage change control is non-y g y p g g

existant|in its infancy| is mature• My organization doesn’t use|uses vulnerability assessment.• My organization performs|hires a third party to perform vulnerabilityMy organization performs|hires a third party to perform vulnerability

assessments.• My organization uses vulnerability assessments to address

compliance|security requirementscompliance|security requirements• My organization is not confident|confident that our vulnerability

assessments accurately assess our use of security controls.

8/16/2012 34

Conclusion

• Combining a traditionally internal process(change control & management) with a traditionally external process (vulnerability assessment) to achieve security that is driven by complianceassessment) to achieve security that is driven by compliance requirements

• Too much to do in too little timeL t ti t ll i t h t• Leverage automation to alleviate resource shortages

• Don’t take humans out of the loop, but put them at the right place in the loop

8/16/2012 35

Contact Us

If you have further questions, please contact me at:

Walt Sikora, wsikora@industrialdefender.com

16 Chestnut St., Suite 300 Foxboro, MA 02035Telephone: +1 508.718.6700

Visit www.industrialdefender.com

8/16/2012 36