dr. samuel liles - selil and sv...

Metricsofprecisionforleadersofsecurity

programsDr.SamuelLiles

Caveat:Thesearemyviews,youcanhaveyourownviews,butthesearemine.Myemployerscurrentandprevioushaveviews,andopinionstoo.Thesearenottheirviews,opinions,orotherwise.I’mhererepresenting thedisciplineofinformationsecurityasappliedtonationalsecurity.Iamnotrepresentinganyagency,organization,orentity.Otherthanmyself.

Agenda• Goal:Givereasonable,actionable,andrealisticmetricsforsecurityofanenterpriseforseniorleaders

• Scope:ThereisFISMA,FITARAandvariousothercompliancedrills.Thisisnotaboutthose

• Topics:Risk,vulnerabilities,investment,workforce,policy

Complianceisnotsecurity.Manyorganizationshavebeenfullycompliantandbreached.Complianceisaboutmeetingrequirements.Securityisaboutbeingfreefromdangerorthreat.Compliancecanbedemonstratedwhereassecurityisaprocessthatincludesadaptionandinnovationbeyondcompliancetorequirements.Requirementshavetobedescribedanddefinedbeforetheycanbecompelled.• http://blog.kaseya.com/blog/2014/09/03/home-depot-yet-another-retail-breach/

• https://pciguru.wordpress.com/2011/08/30/compliance-is-not-security-%E2%80%93-busted/

• http://www.csoonline.com/article/2995924/data-protection/compliant-does-not-equal-protected-our-false-sense-of-security.html

• https://www.schneier.com/essays/archives/2000/04/the_process_of_secur.html

4/28/16 UNCLASSIFIED 2

Partiallybasedon:Ryan,JulieJ.C.H.andDanielJ.Ryan,PerformanceMetricsforInformationSecurityRiskManagement,IEEESecurityandPrivacy,vol.6no.5,Sep/Oct2008,pp.38-44

4/28/16 UNCLASSIFIED 3

CybersecurityforExecutives:APracticalGuide1stEdition• GregoryJ.Touhill• C.JosephTouhillFromAmazon.com: Practicalguidethatcanbeusedbyexecutivestomakewell-informeddecisionsoncybersecurityissuestobetterprotecttheirbusinessEmphasizes,inadirectanduncomplicatedway,howexecutivescanidentify,understand,assess,andmitigaterisksassociatedwithcybersecurityissues• Covers'WhattoDoWhenYouGetHacked?'includingBusinessContinuityandDisasterRecoveryplanning,PublicRelations,LegalandRegulatoryissues,andNotificationsandDisclosures

• ProvidesstepsforintegratingcybersecurityintoStrategy;PolicyandGuidelines;ChangeManagementandPersonnelManagement

• Identifiescybersecuritybestpracticesthatexecutivescanandshouldusebothintheofficeandathometoprotecttheirvitalinformation

(2014)Touhill,G.,Touhill C.J.,Cybersecurityforexecutives:ApracticalGuide,Wiley,IAChE4/28/16 UNCLASSIFIED 4

Measure1: HowVulnerableAreOurSystems?Purpose:IdentifyriskassociatedwithknownvulnerabilitiesMeasure1A:Numberofunpatchedknownvulnerabilities

• Whattomeasure:High,Medium,andLowvulnerabilitiesfromtheCVElist. HighvulnerabilitiesiswhatexecutivesintheC-suiteworryaboutandarewhereyouaremostvulnerable.

• Whentomeasure:Subordinatesshouldbelookingatthiscontinuously(seeDHSContinuousDiagnostics&MitigationeffortfortheUSgovernment)mostexecutivesshouldbelookingatthisatleastmonthly.Well-informedboardsandC-suitesshouldseethisatleastquarterly

• Whymeasure:Badactorsviewexploitationofknownvulnerabilitiesaslow-hangingfruittobeplucked. Properlypatchedandconfiguredsystemsarenotattractivetargets.

• Decisionsthismeasuredrives:Accept,mitigate,avoid,ortransferrisk. Forexample,fixingthehighvulnerabilitieswithinarecommendedtimeframe,addressmediumsasresourcespermit,andacceptlowvulnerabilitiesastheenvironmentdictates. Seniorsshouldknowwhereriskexistsanddictatetheriskappetite,notthetechnicians. Showingstaffcapacitytoaddressmeasures1Aand1BwillgarnerC-suitesupporttoinvestinreinforcements/augmentationto “buydown”risk.

V

(2014)Touhill,G.,Touhill C.J.,Cybersecurityforexecutives:ApracticalGuide,Wiley,IAChE

4/28/16 UNCLASSIFIED 5

MITRECVE:Data1999-2011

V

4/28/16 UNCLASSIFIED 6

1020

4638

6612

Younan,Y.(2013).25YearsofVulnerabilities:1988-2012.SourcefireVulnerabilityResearchTeam.

Dataset:BlackBeltCyberProject,2011-2012

DataderivedfromMITRECVEdatabase.

V

4/28/16 UNCLASSIFIED 7

Arora,A.,Krishnan, R.,Nandkumar,A.,Telang,R.,&Yang,Y.(2004,May).Impactofvulnerabilitydisclosureandpatchavailability-anempiricalanalysis.InThirdWorkshopontheEconomicsofInformationSecurity (Vol.24,pp.1268-1287).

McQueen,M.A.,McQueen,T.A.,Boyer,W.F.,&Chaffin,M.R.(2009,January).Empiricalestimatesandobservationsof0dayvulnerabilities.InSystemSciences,2009.HICSS'09.42ndHawaiiInternationalConferenceon (pp.1-12).IEEE.

1)In2006approximately2500zerodaysinexistenceonanygivenday2)Averagelifespanfromcreationtopatch169days3)ChangesovertimetotheCVEdatabase(backlog,prioritization,exclusion)tendtoinaccuratelyskewpredictiveestimates(downwards!)

Measure1B:Amountofout-of-datesoftware• Whattomeasure:Numberofsystems(e.g.servers,clients,andmobiledevices)whosesoftwareisnotconfiguredwiththelatestversion

• Whentomeasure:Sameas1Aabove• Whymeasure:Similarto1A. Properlypatchedandconfiguredsoftwaregenerallyhasbettersecuritycontrolsthanpreviousversions

• Decisionsthismeasuredrives:Accept,mitigate,avoid,ortransferrisk. Aswith1A,out-of-datesoftwarehasbecomeatargetofchoiceforbadactors.Whilehavingaplantokeepyoursoftwareup-to-datewiththelatestversionsisimportant(andrecommended),runningout-of-datesoftwareoftenmakessenseforsomeorganizationsaslongastheyhavecompensatingcontrolsinplace. KnowingtheriskandarticulatingittoyourboardandC-suiteinamannertheyunderstandiscriticallyimportant.

V

(2014)Touhill,G.,Touhill C.J.,Cybersecurityforexecutives:ApracticalGuide,Wiley,IAChE

4/28/16 UNCLASSIFIED 8

Imageontheleft:YearX/OSILayerYImageontheright:YearY/OSILayerX

KeyTakeAway:NoticeimageonrightdefinitetrackingofCVE’stodifferentlayersovertheyears.Showsbroadtrendsinmovementfromdatalinktoapplicationlayervulnerabilities

Dataset:BlackBeltCyberProject,2011-2012

V

4/28/16 UNCLASSIFIED 9

Measure2:HowVulnerableIsOurWorkForce?Purpose: Identifyriskassociatedwithaproperlytrainedand“cyberaware”workforceMeasure2A:WorkForceCybersecurityTraining• Whattomeasure:Percentageofworkforcecurrentontheirorganizationalcybersecuritytraining

• Whentomeasure:Considerquarterlyatyourlevelandmonthlytosupervisors

• Whymeasure:Atrainedworkforcethatisawareofcybersecurityissuesandhowtopreventthemislesslikelytomakemistakesthatexposeyourorganizationanditsinformationtotrouble. Forexample,trainedpersonnelarelesslikelytofallpreytosocialengineeringandotherhumanfactors.Thisreducestheorganizationalriskexposure(noteitisnotstatedthatiteliminatesrisk,justreducesit)

• Decisionsthismeasuredrives:Accept,mitigate,avoid,ortransferrisk. Mostorganizationsmitigatethisriskbymakingcybersecuritytrainingmandatory. Thekeyhereistohaveaneffectiveandmeaningfultrainingprogramwhileholdingallpersonnel(includingseniorleaders)accountabletobeproperlytrained.

V

(2014)Touhill,G.,Touhill C.J.,Cybersecurityforexecutives:ApracticalGuide,Wiley,IAChE

4/28/16 UNCLASSIFIED 10

4/28/16 UNCLASSIFIED 11

©SamuelLiles

Measure2B:ITTechnicalStaffQualifications• Whattomeasure:PercentageofITtechnicalstaffcurrentontheirtechnicaltrainingandcertifications

• Whentomeasure:Considerquarterlyreviews• Whymeasure:Awell-trainedITtechnicalstaffislesslikelytomisconfiguresystemssuchasgrantingunauthorizedpermissions(i.e.leastprivilege,etc.),notimplementingapplicationwhitelisting,punchingholesinfirewalls,etc.

• Decisionsthismeasuredrives:Accept,mitigate,avoid,ortransferrisk.Itcanbesuccessfullyarguedforandadditionalresourcesreceivedfromseniorstomaintaintechniciantrainingandcertifications. Thisinturnhasledtobettermorale, retention,andperformanceinorganizations.

V

(2014)Touhill,G.,Touhill C.J.,Cybersecurityforexecutives:ApracticalGuide,Wiley,IAChE

4/28/16 UNCLASSIFIED 12

Youshould lookattrainingandskillassessmentasanadaptiveproblemneedinganswered.Thisisaprocessandsustainmentissue.

4/28/16 UNCLASSIFIED 13

©SamuelLiles

Measure3:AreWeDoingtheRightThings?Purpose:DemonstrateduecareandduediligenceMeasure3A:Well-definedanddocumentedpoliciesandprocedures• Whattomeasure:Percentageofcurrentorganizationalpoliciesandprocedures

• Whentomeasure:Annually• Whymeasure:Well-definedanddocumentedpoliciesandproceduresarethestartofgoodorderanddisciplineandarefoundationaltoduecareandduediligence. Toomanycompaniesinvolvedinlitigationwheretheydidnotfollowbestpractices(withtheNISTCyberFrameworkcontinuingtogainmomentumasanexemplar),didnothave policiesandproceduresdefined,ordidn’tfollowtheirownprocedures. Aleadingindicatorishavingasetofcurrent,up-to-date,andmeaningfulpoliciesandproceduresforyourworkforce.

• Decisionsthismeasuredrives:Disciplineinarticulatingstandards. Thisisanareawheretheoutsideauditorsshouldevaluatethepoliciesandproceduresatleastonceayear. Theyshouldbereviewedforcompletenessandcurrency.

C

(2014)Touhill,G.,Touhill C.J.,Cybersecurityforexecutives:ApracticalGuide,Wiley,IAChE

4/28/16 UNCLASSIFIED 14

Measure3B:WorkForcePolicyAcknowledgement• Whattomeasure:Percentageofworkforcethathasacknowledgedthepoliciesandprocedures.

• Whentomeasure:Quarterly• Whymeasure:PoliciesandproceduresthatarepostedonaSharepoint siteandNOBODYreadsorunderstandsthemareworthless.Havingtheaffectedworkforceacknowledgethepoliciesandproceduresfostersbothbettercomprehensionaswellasasenseofaccountability. Anexampleisyour “AcceptableUsePolicy”,butthatshouldn’tbetheonlyoneyouhave!

• Decisionsthismeasuredrives:Workforcetrainingandaccountability

C

(2014)Touhill,G.,Touhill C.J.,Cybersecurityforexecutives:ApracticalGuide,Wiley,IAChE

4/28/16 UNCLASSIFIED 15

Measure3C:AdherencetoPolicyandProcedures• Whattomeasure:NumberofCyberIncidentsresultingfromfailuretofollowstandards

• Whentomeasure:Monthly• Whymeasure:Peoplewhofollowgoodpoliciesandproceduresreducethecybersecurityriskexposureoftheorganization. Spotlightingthelinkagebetweensoundpolicy,adherence,ANDaccountabilityisapotentmeasure.

• Decisionsthismeasuredrives:Thedecisionsrangefromchangingpoliciesandprocedureswhentheyarenolongereffective,refocusingtrainingefforts,toaddressinghowpersonnelareheldaccountable

C

(2014)Touhill,G.,Touhill C.J.,Cybersecurityforexecutives:ApracticalGuide,Wiley,IAChE

4/28/16 UNCLASSIFIED 16

Measure4:AreWeEfficient?Purpose:Makingsurethattheorganizationisproperlybalancedandprovidesagoodreturnoninvestmentbasedontheorganization’sriskappetiteMeasure4A:InformationAssetValuation• Whattomeasure:Percentageofinformationmaintainedbytheorganizationhasbeenassigneda “value”

• Whentomeasure:Annually• Whymeasure:Informationhasavalueyetmostorganizationsdonotconsideritasanassetontheirbalancesheets. Asaresult,techniciansintheserverroomsareleftwithoutdirectionastowhatthepriorityinformationassetsareandtrytodefendeverythingequally. Thatapproachnolongerisviablenorcosteffective.

• Decisionsthismeasuredrives:Adisciplinedapproachoninformationassetvaluationleadingtobetterdecisionsregardinghowtoapportionresourceswhilemanagingrisk

I

(2014)Touhill,G.,Touhill C.J.,Cybersecurityforexecutives:ApracticalGuide,Wiley,IAChE

4/28/16 UNCLASSIFIED 17

4/28/16 UNCLASSIFIED 18

©SamuelLiles

Poweristheabilitytoinfluenceresults

Bennis,W.G.,Berkowitz,N.,Affinito,M.,&Malone,M.(1958).Authority,power,andtheabilitytoinfluence.HumanRelations,11(2),143-155.

Cast,A.D.(2003).Powerandtheabilitytodefinethesituation.SocialPsychologyQuarterly,185-201.

Balanceinallthingscreatesequanimity

Measure4B:InformationCost/BenefitAnalysis• Whattomeasure:TotalCostofOwnershipvsTotalAssetValue

• Whentomeasure:Quarterly• Whymeasure:Manyorganizationsspendtoomuchprotectingtrifleswhilespendingfarlessthantheyneedtoontreasures. Onceyouunderstandthevalueofyourinformation(see4A),youcancompareyouractualTCOagainstthevaluetoensurethe “juiceisworththesqueeze”.Manyareshockedwhentheyfindouthowtheystandinthisarea…

• Decisionsthismeasuredrives:Accept,mitigate,avoid,ortransferrisk. Thedecisionsherearebusiness101itemsandordinarilyaretransparentcorporateresourceallocationandapportionmentissues. BoardsandC-suitesappreciateawell-reasonedand auditableapproachtoinformation;theydon’twanttospend$50protectingtencentsworthofinformation.Youshouldn’teither.

I

(2014)Touhill,G.,Touhill C.J.,Cybersecurityforexecutives:ApracticalGuide,Wiley,IAChE

4/28/16 UNCLASSIFIED 19

4/28/16 UNCLASSIFIED 20

• Generalizedspendingtrendstendtobeinacurate.Considerhowthisgraphchangesassystemcriticalityisaddedasafactor.

• GartnerusesperuserandpercentofITbudgetasmetricsforsecurityspendrate(budget).HowdoesthatfitwithaTCO/ROIanddifferentsystemcriticallevels?

Measure5:AreWeReadyandResilient?Purpose:Makingsuretheorganizationispreparedforacyberincidentandresilienttorecover;i.e.can “takeacyberpunchandkeepgoing”• Measure5A:BusinessContinuityandDisasterRecoveryPlanning

• Whattomeasure:Currencyandcompletenessofanorganizationalbusinesscontinuityanddisasterrecoveryplan

• Whentomeasure:Annually• Whymeasure:Duecareandduediligence.Thebesttimetorespondtoanincidentisbeforeitoccurs. SeeChapter9.0inthebook.

• Decisionsthismeasuredrives:Creationandregularmaintenanceofaplanhelpsidentifyandmanagerisks. Gettingitbeforeseniorleadersisessentialsothatriskisappropriatelyaddressedattherightlevel.

I

(2014)Touhill,G.,Touhill C.J.,Cybersecurityforexecutives:ApracticalGuide,Wiley,IAChE

4/28/16 UNCLASSIFIED 21

ISO/IEC27035:2011providesastructuredandplannedapproachto:1.detect,reportandassessinformationsecurityincidents;2.respondtoandmanageinformationsecurityincidents;3.detect,assessandmanageinformationsecurityvulnerabilities;and4.continuouslyimproveinformationsecurityandincidentmanagementasaresultofmanaginginformationsecurityincidentsandvulnerabilities.

Preparation, identification, containment, eradication, recovery, and lessons learned.

Incident triage, incident coordination, incident resolution

ISO/IEC27035:2011:InformationSecurityIncidentManagement

SANS:CreatingandManaginganIncidentResponseTeam

RFC2350:ExpectationsforComputerSecurityIncidentResponse

CERT: Handbook for Computer Security Incident Response Teams (CSIRTs)

NIST800-61:ComputerSecurityIncidentHandlingGuide

4/28/16 UNCLASSIFIED 22

Measure5B:ResiliencyEffectiveness• Whattomeasure:Numberofdrillsandexercisesthattestthebusinesscontinuityanddisasterrecoveryplan

• Whentomeasure:Monthly• Whymeasure:AsVinceLombardistated, “PerfectPracticeMakesPerfect”. Makesureyouroperationalandtacticallevelleadersroutinelyconductdrillsandexercisesandreviewtheirfindingsandfixeswiththem. Instillacultureofcontinualimprovementandencouragepeopletofindandfixweaknesses.Whenyoudothat,youwillbebetterpreparedforwhentheyou-know-whathitsthefan.

• Decisionsthismeasuredrives:Inadditiontoresourceallocationandapportionmentdecisions,thismeasurealsodrivesdecisionsregardingorganizationalalignment,rolesandresponsibilities,andliabilities

I

(2014)Touhill,G.,Touhill C.J.,Cybersecurityforexecutives:ApracticalGuide,Wiley,IAChE

4/28/16 UNCLASSIFIED 23

Questions?

4/28/16 UNCLASSIFIED 24