how to achieve success with cyber risk assessment … · how to achieve success with cyber risk...
TRANSCRIPT
How to Achieve Success with Cyber Risk Assessment and Analysis
October 24, 2014Orlando, Florida
www.issa.org
Ben Tomhave
Research Director, Gartner
@falconsview
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Starting With the End in Mind
1. Context and Process Are Essential
2. Start Somewhere, Then Operationalize It
3. Employ a Two-tier Approach
4. Find Your Inner Quant
5. Good Tools Enable Good Practices
Applicable research:
• Planning and Executing Successful IT Risk Assessments
• Comparing Methodologies for IT Risk Assessment and Analysis
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
"Context Is Everything."
—Anton Aylward
"Priority Is a Function of Context."
—Stephen Covey
1. Context and Process
Are Essential
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
ISO 31000 as a High-Level Template
Risk Management
Risk Assessment
Risk Analysis
Treatment
Assessment
Identification Analysis Evaluation
Context
Mo
nito
ring
and
Re
vie
w
Co
mm
un
ica
tio
n
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Critical: Process Presence & Alignment
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Meaningful Output Is Essential
• Meaningful … to whom?
• Looks like …?
• GIGO?
Your risk is 72.
Likely loss of $0.5-1M.
Your risk is high.
Image Credit (CCby2.0): Thoth (https://www.flickr.com/photos/thoth-god/4078908973/sizes/z/)
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
"The Journey of a Thousand MilesBegins With One Step."
— Lao Tzu
2. Start Somewhere, Then
Operationalize It
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Ramp-up Time and Cycle Time
Short Medium Long
Qua
ntita
tive
FAIR
RiskSafe
ISF IRAM Ramp-up Time
Cycle Time
COBIT 5
MAGERIT
NIST SP 800-30
OCTAVE Allegro
Medium-long
Eith
er/B
oth
Qua
litat
ive
Time
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Common Errors and Misconceptions
• Estimation and calibration — avoid point values!
• Inadequate business input
• Bad assumptions and GIGO
• Gaming the system
• Use of "mathmagic"
Image Credit (CCby2.0): acidpix (https://www.flickr.com/photos/acidpix/4795721175/sizes/z/)
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Setting a Bad Example
High Medium Low *Score
Acctg. 2 5 11 36
CRM 3 2 9 30
Web 1 7 22 48
TOTAL 6 14 42 38
*High=5, Medium=3, Low=1
What does this mean??
All equal?
Context???
Is this meaningful?
Context??
Stats violation!!!
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Abusing Stats for Fun and Misadventure
• Correlation is not causation
• Correlating measurements is important
• No math on categorical data!
• A 2-star review + 3-star review != 5-star review
• Measurement issues…– What if the population changes?
– Don’t assume, drill down on percentage changes!
– Victim of your own success?
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Scale and Continuous Improvement
Image Credit (CCby2.0): SantaRosa (https://www.flickr.com/photos/santarosa/25526929/sizes/o/)
Process First!
Leverage ISO 31000!
Iterate and Evolve!
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
3. Employ a Two-tier Approach
Image Credit: Critical Illness & Trauma Foundation (http://citmt.org/Start/images/flowchart2.jpg and http://citmt.org/Start/images/startsml.jpg)
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Formal vs. Informal Methods
Triage Is Important
Prioritize Rapidly
"KISS" Principle
Image Credit (CCby2.0): Phil Manker (https://www.flickr.com/photos/philmanker/3654636770/sizes/o/)
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Example: FAIR for Qual or Quant
Productivity Loss
Response Costs
Replacement Costs
Competitive Advantage Loss
Fines & Judgments
Reputation Damage
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Pace Layering: A Lesson From DevOps
Systems of Record(Pace of Change: Slow)
Systems of Differentiation (Pace of Change: Medium)
Systems of Innovation (Pace of Change: Fast)
Application Intra-application Integration
Intralayer Integration
Interlayer Integration
Application
ServiceService
Service Service
Service
Service
ServiceService
S
S SS
Application
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Multiple Perspectives on Impact
Asset
Process
Performance
Image Credit (CCby2.0): Christiaan Triebert (https://www.flickr.com/photos/christiaantriebert/2975069958/sizes/l/)
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
"The Heart of Science Is Measurement."
— Erik Brynjolfsson
"Subjectivity Measures Nothing Consistently."
—Toba Beta
4. Find Your Inner Quant
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
"No Perfect Data"
"Unknown Unknowns"
"Your Estimate Is Bad"
"Bad Assumptions"
"You Can't Estimate Likelihood"
Don't Believe the Myths
Image Credit (CCby2.0): Beverly & Pack (https://www.flickr.com/photos/walkadog/3484426248/sizes/l)
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Why is Quant So Difficult?!
Image Credit (CCby2.0): nathanmac87(https://www.flickr.com/photos/nathanmac87/5081698065/sizes/l)
InfoSec is…
Non-deterministic
Constantly changing
About belief states
Numeracy is problematic
Frequentists vs. Bayesians
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
You Have Lots of Data!
Image Credit (CCby2.0): NASA Goddard (https://www.flickr.com/photos/gsfc/7309213060/sizes/o/)
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Examples of Data
• Logs– Infrastructure & Apps
– InfoSec (AV, IDS/IPS, etc.)
– PhysSec (badge access)
• Metrics & Measurements
• Post-mortem Reporting
• Assessments, Audits, Scans
• Business Performance Info
Image Credit (CCby2.0): JD Hancock (https://www.flickr.com/photos/jdhancock/8031897271/sizes/z/)
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Consider: The Risk Spectrum
Frequency
Impact Magnitude
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Consider: The Risk Spectrum
Frequency
Impact Magnitude
BCM
“Daily”
Concerns
“Scary”
Gray
Area
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Beware of Common Mistakes
Image Credit (CCby2.0): JustinJensen (https://www.flickr.com/photos/justinjensen/4947663237/sizes/l/)
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
"We Become What We Behold. We Shape Our Tools, and Thereafter Our Tools Shape Us."
—Marshall McLuhan
5. Good Tools
Enable
Good Practices
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Key Finding: Cultural Fit Is Important!
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Leverage Platforms (like GRC)
Image Credit (CCby2.0): NASA Goddard (https://www.flickr.com/photos/gsfc/11827553605/sizes/h/)
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Formalize Processes
KISS! Triage! Communicate!
Image Credit (CCby2.0)): JD Hancock (https://www.flickr.com/photos/jdhancock/8671399450/sizes/h/)
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Don't Forget Training!
Image Credit (CCby2.0): Official U.S. Navy Imagery (https://www.flickr.com/photos/usnavy/9718801246/sizes/z/)
Image Credit (CCby2.0): James Sarmiento (https://www.flickr.com/photos/ijames/112866960/sizes/o/)
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Beware GIGO
Image Credit (CCby2.0): dimnikolov (https://www.flickr.com/photos/dimnikolov/3471180754/sizes/z/)
Image Credit (CCby2.0): Phillie Casablanca (https://www.flickr.com/photos/philliecasablanca/2064915931/sizes/z/)
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Context and Process Are Essential
Start Somewhere, Then Operationalize It
Employ a Two-tier Approach
Find Your Inner Quant
Good Tools Enable Good PracticesImage Credit (CCby2.0): VinothChandar (https://www.flickr.com/photos/vinothchandar/6840269621/sizes/o/)
To Recap…
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Q & A
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Recommended Gartner Research
Planning and Executing Successful IT Risk AssessmentsAnne Robins, Erik Heidt (G00268438)
Comparing Methodologies for IT Risk Assessment and AnalysisBen Tomhave, Erik Heidt, Anne Robins (G00256964)
Security Information and Event Management Futures and Big Data Analytics for SecurityAnton Chuvakin, Ramon Krikken (G00255883)
Use the Pace-Layered Application Strategy to Guide Your DevOps StrategyGeorge Spafford, Cameron Haight (G00245328)
For more information, stop by Gartner Research Zone.
© 2014 Gartner, Inc. and/or its affiliates. All rights reserved.
Thank You!
Ben Tomhave
@falconsview