The Science of Risk:Overview of Risk Analysis

David Moser, Charles Yoe,

Office of the Chief Economistcyoe1@verizon.net

Institute for Water Resources2009

2

Objectives

• By the end of this training module you will be able to:– Identify and describe tasks of risk analysis– List some elements of a risk manager’s job– Identify several risk assessment tools– Describe what drives risk communication– Identify the 3 Ms of risk communication

3

Language and Models of Risk Analysis

4

Characteristics of Risk

• Risk is everywhere• Some risks are more serious than others• Zero risk is not an option• Risk is unavoidable• We need to. . .

– describe them (risk assessment)– talk about them (risk communication)– do something about them (risk

management)

Risk

• Risk is a measure of the probability and consequence of uncertain future events

• Risk includes – Potential for gain (opportunities) – Exposure to losses (hazards)

5

• We have been doing risk assessment for flood for a long time– EAD is a risk assessment metric

• Major rehab program requiring reliability analysis developed in 1992

• These focused on economic risks and expected values and deemphasize uncertainty

Risk Is Not New To Corps

Why Use Risk Analysis?

• Traditional standards based approaches are no longer enough—problems persist

• Identify uncertainties to be monitored and managed

• Science-based approaches have helped• Risk analysis is a new way of approaching

problems that integrates science and social values

• The global community is embracing risk analysis

7

Selected Corps Concerns

• Residual flood risk• Risk of infrastructure

malfunction• Risk to life, health and safety• Marine casualties• Navigation channel

sedimentation• Ice cover• Small boat harbor damages• Commercial fishery losses• Coastal Erosion

• Ecosystem restoration• Energy and power reliability• Dam safety• Loss of pool• Operation and maintenance

– Facility maintenance• Major rehabilitation• Risks to environment• Cost estimation• Project management• Budgeting Priorities• MilCon support

8

Science-Based Water Resource Analysis

Effective Modern

Water Resource Analysis

Water Resource Problems

Traditional Water Resource Analysis

Public Safety, EQ, NED and Other Benefits

ScienceInfrastructure

Risk ManagingOrientation

Risk Analysis

9

Risk Analysis

• Risk analysis is a decision-making framework that comprises three tasks: risk assessment, risk management, and risk communication.

Risk Management

• Policy and preference based

Risk Assessment

• Analytically based

Risk Communication

•Interactive exchange of information, opinions, and preferences concerning risks

Risk Analysis Is a Paradigm

• It is a way of thinking about things and organizing to solve problems– It is not an activity

• It is science based but not science

• It is the interface between science and values

• It is used for making decisions under uncertainty

12

Examples of Risk Analysis:Water Resources

• Expected annual damages

• Establishment of design levels

• Risk-informed cost estimation

• Dam safety program

• Levee safety program

• Establishment of product safety standards, performance standards, and specifications

• Scientific sampling protocols13

Why Do Risk Analysis?

• It improves the quality of our thinking before a decision is made—uncertainty is widespread

• It helps ensure a safe water resource infrastructure• It protects human, animal, and plant life and health• It provides a more reliable flow of outputs• It is better than the alternative• Because we have to-ERs, ECs, OMB, GAO

14

Context of Risk Analysis

• Risk analysis takes place within an organizational culture– COE structures and decision processes already exist– Goals, objectives and constraints are the

organization’s reality• There are decisions to be made• Risk analysis is a process and a paradigm, a way of

approaching problems and decisions that will look different in every organization that uses it

15

When To Do Risk Analysis

Extensive Risk AnalysisW/ Adaptive Management

Routine Risk Analysis

Modest Level of Risk Analysis

No Risk Analysis Required

Consequence of Being Wrong

UncertaintyMuch

Grave

Little

Minor

Risk Management-Informally

• What is the problem?• What questions do we

want risk assessment to answer?

• What can be done to reduce the impact of the risk described?

• What can be done to reduce the likelihood of the risk described?

• What are the trade-offs of available options?

• What is the best way to address the described risk?

• Is it working?

17

Working Definition

• Risk management is the process of problem finding and initiating action to identify, evaluate, select, implement, monitor and modify actions taken to alter levels of risk, as compared to taking no action.

18

Risk Management

19

Establish Decision ContextEstablish Decision Context

Identify RisksIdentify Risks

Analyze RisksAnalyze Risks

Evaluate RisksEvaluate Risks

Risk Management DecisionRisk Management Decision Mon

itor,

Eval

uate

, Mod

ifyM

onito

r, Ev

alua

te, M

odify

Com

mun

icat

e an

d Co

nsul

tCo

mm

unic

ate

and

Cons

ult

Risk

Ass

essm

ent

Risk Assessment—Informally

• What can go wrong?

• How can it happen?

• How likely is it?

• What are the consequences?

20

Working Definition

• Risk assessment is a systematic, evidence based approach for quantifying and describing the nature, likelihood, and magnitude of risk associated with the current condition and the same values resulting from a changed condition due to some action.

21

Risk Assessment StepsLook for the Hazard or Opportunity

Identify the hazards that can cause harm or the opportunities for gain that are uncertain.

Likelihood AssessmentAssess the likelihood of the various adverse and beneficial consequences. Characterize these likelihoods and their uncertainty qualitatively or quantitatively.

Consequence AssessmentDecide who or what may be harmed or benefited and in what ways. Gather and analyze the relevant data. Characterize the consequences and their uncertainty qualitatively or quantitatively.

Risk CharacterizationEstimate the probability of occurrence, the severity of adverse consequences, and the magnitude of potential gains, including attendant uncertainties, of the hazards and opportunities identified based on the evidence in the preceding steps. Characterize the risk qualitatively or quantitatively with appropriate attention to baseline and residual risks, risk reductions, transformations and transfers.

22

Good Risk Assessment

• Is science-based• Gets the right science and gets the science

right • Science provides basis for answer to risk

question(s)• Separates what we know from what we do not

know– Keeps track of uncertainties

• Assumptions and uncertainties are made clear

23

Risk Communication-Informally

• Why are we communicating?

• Who is our audience?

• What do our audiences want to know?

• What do we want to get across?

• How will we communicate?

• How will we listen?

• How will we respond?

24

Working Definition

• Risk communication is the open, two-way exchange of information and opinion about hazards and risks leading to a better understanding of the risks and better risk management decisions. Risk communication ensures that the decision makers, other stakeholders and affected parties understand and appreciate the process of risk assessment and in so doing can be fully engaged in and responsible for risk management.

25

Who Is Using Risk Analysis?

• DOD• EPA• FDA• USCG• USDA (APHIS, FSIS)• DOT• DOE

• OMB• GAO• WTO• Engineering

profession• Industry• International

organizations

26

Risk Management Overview

27

Risk Management

• Policy and preference based

Risk Assessment

• Analytically based

Risk Communication

•Interactive exchange of information, opinions, and preferences concerning risks

The Job!

• Risk managers are responsible for risk analysis; they identify or validate problems

• Risk managers need scientific information to make decisions– They ask questions that, when answered, will give them

the information they need to make decisions

• Risk assessors answer the questions and characterize the uncertainty in their answers

• Risk managers mitigate risks that are not acceptable• Risk managers make sure that risk communication takes

place

29

Test

• Which is the greater risk to human life in the U.S.?– Hazardous waste sites– Smoking

• Which is the U.S. spending about $30 billion on annually? – Cleaning hazardous waste sites– Programs to reduce smoking

• Which is the U.S. spending about $500 million on annually? – Cleaning hazardous waste sites– Programs to reduce smoking

30

Test

• Consider this scenario. A chemical solvent from a nearby plant is found in the air of one room in an elementary school. It is present at levels beneath 10 parts per billion. An advocate for the children says a fan in the window would make the school safe. Which risk management actions would you recommend?– Put a fan in the window– Require children who used to walk to school to

take the bus across snow-covered streets in rush hour to the high school

31

Test

• True or false. We could make decisions that are more rational and informed.

• True or false. We sometimes make policy based more on fear than fact.

• True or false. If we overspend on risks like pesticides or asbestos, which are real but relatively low-magnitude risks, we have less to spend on the prevention of foodborne illness, smoking, and crime.

• True or false. We often fear before we think.• True or false. Putting resources where the risks are

makes sense.

32

Can We Do Better?

• If we can improve our thinking before we make a decision under uncertainty, perhaps we can do better

• Risk analysis is designed for making decisions under uncertainty

• It all starts with risk management

33

Who Are the Risk Managers?

• In the Corps?

• Elsewhere?

34

COE Risk Management

Establish Decision ContextEstablish Decision Context

Identify RisksIdentify Risks

Analyze RisksAnalyze Risks

Evaluate RisksEvaluate Risks

Risk Management DecisionRisk Management Decision Mon

itor,

Eval

uate

, Mod

ifyM

onito

r, Ev

alua

te, M

odify

Com

mun

icat

e an

d Co

nsul

tCo

mm

unic

ate

and

Cons

ult

Risk

Ass

essm

ent

Risk Management Strategies

• Acceptance/Retention

• Avoidance

• Reduction

• Transfer

• Transformation

• Containment

• Constant monitoring

• Continuous research

• Development of substitutes

• Increasing resilience–surprises do less harm

• Precautionary principle

• Adaptive management

• Build confidence and trustworthiness

• Reduce uncertainties

• Clarify facts

• Involve affected people

• Deliberation

• Accountability

36

Principles for Choosing Risk Mitigation

• P&G– NED, PS, EQ

• Zero risk

• Weight-of-evidence

• Precautionary principle

• Sound science

• As low as reasonably achievable (ALARA) principle

• Appropriate level of protection (ALOP) principle

• Reasonable relationship

• Safety standards

37

Final Management Decision: The Best Option?

• Some possible criteria

– Prevent rather than control risk

– Offer stakeholders choice of options

– Based on best available scientific, economic, and technical information

– Economic feasibility

– Enforceable

– Consider tolerable level of risk (TLR)

38

Tolerable RiskIn

crea

sing

Ind

ivid

ual r

isks

and

soc

ieta

l con

cern

s

UnacceptableRegion

TolerableRegion

Broadly acceptableRegion

Risk cannot be justifiedexcept in extraordinary

circumstances

People and society are prepared to accept risk in

order to secure benefits

Risk regarded as insignificant, further effort

to reduce risk not required

Monitor/Evaluate/Modify

• Actively measure and evaluate success of decision, its implementation, and outcome to determine if changes are needed– See if desired risk reductions have been

achieved– Determine if new information is available

• Resources are required for these tasks

40

41

Historical Response to Risk• Risk presumed to be “managed” by engineering

design– Freeboard on levees and dams– Cost contingencies– Underkeel clearance design criteria– Channel width design criteria– Probable maximum flood spillways

• Or policy– Choose NED plan– Obligations and expenditures

Risk Assessment Tool Box

42

Risk Management

• Policy and preference based

Risk Assessment

• Analytically based

Risk Communication

•Interactive exchange of information, opinions, and preferences concerning risks

The Risk Assessment Model

• A scientifically based process consisting of the following steps:

1. Look for the hazard

2. Consequence assessment

3. Likelihood assessment

4. Risk characterization

• The definition includes quantitative risk assessment, which emphasizes reliance on numerical expressions of risk, and also qualitative expressions of risk, as well as an indication of the attendant uncertainties.

44

Look for the Hazard or Opportunity Identify the hazards that can cause harm or the opportunities for gain that are uncertain.

Likelihood AssessmentAssess the likelihood of the various adverse and beneficial consequences. Characterize these likelihoods and their uncertainty qualitatively or quantitatively.

Consequence AssessmentDecide who or what may be harmed or benefited and in what ways. Gather and analyze the relevant data. Characterize the consequences and their uncertainty qualitatively or quantitatively.

Risk CharacterizationEstimate the probability of occurrence, the severity of adverse consequences, and the magnitude of potential gains, including attendant uncertainties, of the hazards and opportunities identified based on the evidence in the preceding steps. Characterize the risk qualitatively or quantitatively with appropriate attention to baseline and residual risks, risk reductions, transformations and transfers.

Risk Characterizations

• Informal

• No set structure, minimal science (low end)

• Screening

• Separating elements into categories of interest, (e.g. significant vs. insignificant)

• Ranking• Put hazards and

opportunities in ordinal sequence

• Qualitative• Usually categorical or an

ordinal scoring system• Quantitative

• Deterministic • Probabilistic (high end)

46

Alternatives to Risk Assessment

• Ignorance

• Trial and error

• Professional judgment

• Expert opinion

• Precautionary principle

• Policy

47

Many Techniques

• These risk assessment models use many tools and techniques– Qualitative– Quantitative

• Science– Essential for all risk assessments

• Specific techniques, for example– Probability sampling– Monte Carlo process

48

Qualitative Methods Toolbox

• Increase or Decrease Risk• Risk Narratives• Evidence Mapping• Screening• Ratings• Rankings

• Enhanced Criteria Ranking

• Operational Risk Management (Risk Matrix)

• Develop a Generic Process

• Qualitative Assessment Models

• Multi-Criteria Decision Analysis

Evidence Maps• Evidence base or data• Pro and con

arguments, the warrants– Includes respective

supporting or attenuating arguments

• Conclusions of claim about existence of a hazard with remaining uncertainties

Enhanced Criteria Based Ranking

• Criteria• Ratings• All Possible Combinations of Ratings• Ranking• Evaluate Reasonableness of Ranking• Add Criteria• New Combinations of Ratings• New Ranking

ORM Risk Assessment Values

• Each risk you assess is placed in a cell and managed accordingly

Risk Levels

Risk Matrix

Quantitative Risk Assessment

• Safety Assessment

• Scenario Analysis– Deterministic Scenario Analysis– Probabilistic Scenario Analysis

• Sensitivity Analysis

• Uncertainty Analysis

• Modeling

• Vulnerability Assessment

Safety Assessment

• Identify Acceptable Risk (AR)

• Establish design standards to meet that risk

– Uncertainty factors

• Estimate Existing Risk (ER)

• ER/AR < 1 safety

56

ER/AR1.0

Safe Less Safe

Monte Carlo Simulation

0.00

0.08

0 10 20 30 40 0.0

0.4

5.0 8.8 12.5 16.3 20.0

X =

0.00

0.02

0 100 200 300 400

20 10

76.3% 0.000171615

0 0

0.8% Grounding?

1

48.1% 2.56255E-05

0 0

23.7% Allision?

1

1.0% 2.76973E-05

1 1

51.9% Collision

1

4.0% Casualty Occurs?

82

99.2% 0.029198219

81 81

72.7% Draft > Controlling Depth?

1818

96.0% 0.697278585

2502 1736 1736

Contain Oil?

74.9% 4.92731E-06

0 0

0.0% Grounding?

0

44.1% 7.29167E-07

0 0

25.1% Allision?

0

1.0% 9.23368E-07

0 0

55.9% Collision

0

4.9% Casualty Occurs?

39

100.0% 0.013376669

39 39

27.3% Draft > Controlling Depth?

684

95.1% 0.259915008

645 645

Vessels per Year

Yes

No

Yes

No

Yes

No

Yes

No

Yes

No

Yes

Yes

No

Yes

No

Yes

No

Yes

No

Yes

Probabilistic Scenario Analysis

Sensitivity Analysis

• Systematic investigation of uncertainties of key importance and/or concern to risk managers

• Creating scenarios to explore the range of possible outcomes

60

Vulnerability Assessment

CARVER is a DoD tool to assist in identifying vulnerable offensive enemy targets.

• Criticality—measure of public health and economic impacts of an attack

• Accessibility—ability to physically access and egress from target

• Recuperability—ability of system to recover from an attack • Vulnerability—ease of accomplishing attack • Effect—amount of direct loss from an attack as measured by

loss in production • Recognizability—ease of identifying target

62

Select a Process

• Identify a critical process or infrastructure and assess vulnerability across nation, or

• Assess vulnerability of components of a single process

Risk Communication Overview

64

Risk Management

• Policy and preference based

Risk Assessment

• Analytically based

Risk Communication

•Interactive exchange of information, opinions, and preferences concerning risks

66

Risk Communication Goals Tailor communication so it takes into account

the emotional response to an event. Empowers stakeholders and public to make

informed decisions. Prevent negative behavior and/or

encourage constructive responses to crisis or danger.

National Weather ServiceHurricane Ike Warning for GalvestonSeptember, 2008

"PERSONS NOT HEEDING

EVACUATION ORDERS IN

SINGLE FAMILY, ONE OR TWO

STORY HOMES WILL FACE

CERTAIN DEATH. ."

68

Communication Models

Basic Communication Model• Uni-directional or we tell “them” approach• Who says - what - when - to whom - through

what channel - with what effect

Risk Communication Model• Multi-directional• Actively involves the audience as an

information source

69

Risk Communication Outcomes

• Decrease illness, injury & deaths• Reduce property and economic losses• Build support for response plan• Assist in executing response plan• Prevent misallocation & wasting of resources• Keep decision-makers well informed• Counter or correct rumors• Foster informed decision-making concerning

risk

What Risk Communication is Not

• Spin• Public relations• Damage control• Crisis management• How to write a press release• How to give a media interview• Always intended to make people “feel

better” or reduce their fear

What Risk Communication IS

• Considers human perceptions of risk• Multi-directional communication among

communicators, publics and stakeholders• Activities before, during and after an event • An integral part of an emergency response

plan• Empowers people to make their own

informed decisions

It’s Risk Not Protection

Time in floodplain

Probability of 1 or more floods

10 years 65.132%25 years 92.821%30 years 95.761%75 years 99.963%100 years 99.997%

10-year floodplain occupant

73

Interpreting Risk• Communicating about risk is

difficult because of the way people interpret risk

• Involves competing perspectives: objective vs subjective

74

What Shapes Perceptions of Risk?

• Hazard – something that can go wrong• Probability – likelihood of it happening• Consequences – implications of hazard • Value – subjective evaluation of the

relative importance of what might be lost

FEELING

THINKING

75

Scientist - Consumer Disconnect

SCIENTISTEXPERT

knowsthinks

CONSUMER PUBLIC

feelsbelieves

Fact-based:

hazard, probabilityValue-based:

consequences, value

What Drives Risk Communication?

• Hazard—“the hazard,” the science– Experts’ concerns– Less concern to public

• Outrage—the broad context of the hazard– Public’s concern– Less concern to experts– Often cultivated by media on purpose and by

industry and government by ignorance

76

Outrage Factors Affecting Acceptability

• Catastrophic potential• Familiarity• Understanding• Controllability• Voluntary exposure• Effects on children• Manifestation of

effects• Victim identity

• Dread• Trust in institutions• Media attention• Accident history• Equity• Benefits• Reversibility• Origin

The Point

• Objective risk and perception of risk often diverge

• We are often managing fear more than we are risks

• Fear management can be found in the risk manager’s job description under “other duties as assigned”

78

79

A variety of risk comm approaches

Hazard (danger)

High

Low High

Outrage Management

Crisis /

Emergency Communication

Public

Relations

Precaution

Advocacy

Outrage(fear, anger)

80

Goal: Reduce outrage so people don’t take unnecessary precautions

Hazard (danger)

High

Low High

Crisis /Emergency

Communication

Public Relations

PrecautionAdvocacy

Outrage

Management

Outrage(fear,anger)

81

Goal: Increase concern for a real hazard to motivate preventive action

Hazard (danger)

High

Low High

Outrage Management

Crisis /Emergency

Communication

Public Relations

PrecautionAdvocacy

Outrage(fear,anger)

82

Hazard (danger)

High

Outrage(fear,anger)

Low High

Outrage Management

Public Relations

PrecautionAdvocacy

Crisis /

Emergency

Communication

Goal: Acknowledge hazard, validate concern, give people ways to act

3Ms of Risk Communication

• Message—what to say

• Messenger—who says it

• Media—how it should be presented

83

Message Development

• What are the three most important things for audience to know?

• What three things would audience most like to know?

• What three points is the audience most likely to get wrong unless we emphasize and explain them?

84

Subtle Changes

• Subtle changes in the way things are described can make a big difference.

• Imagine the U.S. is preparing for the outbreak of an unusual Mexican disease that is expected to kill 600 people. Two alternative programs to combat the disease have been proposed. Assume treatment programs with the following outcomes.

85

Choice One

• Program Options– Program A: will save 200 people– Program B: 1/3 probability of saving 600

people and 2/3 probability of saving no one

• Which do you prefer?

86

Choice Two

• Program Options– Program C: 400 people will die– Program D: 1/3 probability no one will die

and 2/3 probability 600 people will die

• Which do you prefer?

87

Principles of Risk Communication

• Know the audience

• Involve the scientific experts

• Establish expertise in communication

• Use a credible source to deliver information

• Share responsibility

• Differentiate between science and value judgment

• Assure transparency

• Put the risk in perspective

• We are all risk communicators

90

Take Away Points

• Risk analysis is a paradigm shift

• Risk analysis comprises three tasks—risk assessment, risk management, and risk communication

• Risk management is decision making that depends on– Science– Values important to stakeholders

91

Take Away Points (continued)

• Science is provided by risk assessment • Risk assessment has four steps:

– Identify the hazard – Consequence assessment– Likelihood assessment– Risk characterization

92

Take Away Points (continued)

• Risk communication is driven by hazard and outrage

• We need to communicate risk in ways people can understand and respond to effectively

93

To learn more about Actions for Change visit us on the web at

https://maps.crrel.usace.army.mil/AFC/

Questions?