Robust Decision Making

Robert Lempert

RAND

HDGC Seminar

February 13, 2004

211-15-03

How Should Climate-Change Uncertainties Be Characterized for Decisionmakers?

Key climate change uncertainties include “Basecase” emissions Behavior of perturbed climate system Value our descendants place on impacts of climate

change Costs of abatement with future technology

Climate-change decisionmakers must understand uncertainties to make effective choices

311-15-03

Analytic Tools Often Vital to Clarify Thinking, But Can Illuminate Trees Rather Than Forest

Analytic tools often vital in improving complicated decisions: Can successfully summarize vast quantities of information Help address flaws in human reasoning

Traditional analytic methods assume well-characterized risks and policy choices based on predictions

But strategic decisions can go awry if decision-makers assume risks are well-characterized when they are not

Uncertainties are underestimated Strategies can be brittle Misplaced concreteness helps blind decision-makers to surprise

Predict Act

411-15-03

Global Scenario Group offers three families of sustainability scenarios Conventional worlds Barbarization Great Transformations

These scenarios Capture a wide range of factors which may affect the

future Attempt to make an argument for a particular risk-

management strategy

Scenarios Capture and Communicate Information About Future, But Hard to Link to Actions

511-15-03

Should Analysts Put Probabilities on Scenarios

Such as Those Developed by SRES?

Pros Necessary to make

policy Others will provide

likelihood estimates if experts don’t

Cons Little evidence to support

judgments about probabilities

Arguing over likelihoods distracts from reaching consensus on near-term actions

Desire for concreteness driving IPCC towards placing probabilities on scenarios

611-15-03

Outline

Robust decision making

Example of robust decisionmaking as a means of characterizing uncertainty

Conclusions

711-15-03

Climate Change is a Problem of Decisionmaking Under Deep Uncertainty

Deep uncertainty is: When we do not know, and/or key parties to the decision do not agree

on, the system model, prior probabilities, and/or “cost” function

Under conditions of deep uncertainty, decision-makers: Often seek robust strategies, ones which perform reasonably well

compared to the alternatives across a wide range of plausible futures, evaluated with a range of values

Robust strategies are often (but not always) adaptive, that is they evolve over time in response to new information

Often use choice of strategy, not additional information, to reduce uncertainty

811-15-03

Robust Decisionmaking (RDM)

Robust decisionmaking Is an iterative, analytic process that identifies

• Strategies that perform well over a wide range of futures

• Remaining vulnerabilities of these strategies Made possible by advances in computational

capabilities Characterizes uncertainties most important to the

choice among strategies

911-15-03

Four Key Elements of Robust Decision Making

Consider large ensembles (hundreds to millions) of scenarios

Seek robust, not optimal strategies Achieve robustness with adaptivity Design analysis for interactive exploration of a

multiplicity of plausible futures

10

11-15-03

Consider Ensembles of Many Scenarios

On the occasion of the 1893 World Columbian Exposition, 74 experts wrote essays predicting what the United States would look like in 1993

Most were wrong

But some were strangely close to the truth

11

11-15-03

Use Robustness Criteria to Judge Alternative Strategies

Under deep uncertainty, decision makers often seek robust strategies that work reasonably well over a wide range of plausible futures

We measure robustness according to degree of “regret,” which is defined as the difference between

the performance of a strategy in a given future, and

the performance of the best strategy in that future

12

11-15-03

Combine Human and Machine Capabilities

Landscape ofplausible futures

Alternative strategiesX

Ensemble of scenarios

Robuststrategies

13

11-15-03

Exploratory Modeling Software Supports This Process

Exploratory modeling software enables users to navigate through large numbers of scenarios and Formulate rigorous policy arguments based on these explorations

CARsTM is java-based exploratory modeling software that: Links to virtually any type of model and/or data Supports interactive use of search and visualization to create, explore,

compare, and understand large scenario ensembles

Tools to draw Tools to draw meaning from meaning from

informationinformationUsers

Tools to Tools to represent represent

informationinformation

14

11-15-03

Outline

Robust decision making

Example of robust decisionmaking as a means of characterizing uncertainty

Conclusions

15

11-15-03

Example Application of Robust Decisionmaking

The RDM approach employed a simple method of representing information “Toy” systems-dynamics model with 41 input parameters

representing uncertainties about• future economic, demographic, and environmental trends• values and capabilities of future decisionmakers

Simple agent-based model of future decisionmakers Four value functions based loosely on UN Human Development

Index, which reflects interests of a range of stakeholders Near-term strategies affect “decoupling” rate

Example: What choice of near-term actions will help ensure Example: What choice of near-term actions will help ensure strong economic growth and a healthy environment over the strong economic growth and a healthy environment over the

course of the 21course of the 21stst century? century?

16

11-15-03

Visualizations Capture Key Relationships Among Plausible Futures

Landscape of plausible futures helps illuminate key challenges to ensuring strong economic growth and a healthy environment over the

course of the 21st century.

Economic growth rate

1.0 2.0 3.0 4.0

–1.0

0

1.0

5.0

3.0

4.0

2.0

0

India since 1960

U.S. 1890-1930

U.S. since 1950

U.S. in 20th century

China since 1960

Brazil since 1980

Russia since 1993

Conventional World scenario

Barbarization scenario

Great Transition scenario

Decoupling rate

17

11-15-03

Compare “Fixed” Near-Term Strategies Across Scenarios

Near Term

Choose policies

Assume near-term policy continues until changed by future generations

Future decision-makers recognize

and correct our mistakes

Future

18

11-15-03

Look for Robust Strategies

XLandscape of

plausible futures

Alternative strategies

Ensemble of scenarios

Robuststrategies

19

11-15-03

Strategies Should Be RobustAcross Multiple Measures of “Goodness”

Use measures inspired by UN’s Human Development Index (HDI)• Discounted, average rate of improvement in GDP/capita,

longevity, and environmental quality (but no education level) time series

• Four different weightings

N$: North GDP/capita and longevity

W$: Global GDP/capita and longevity

NG: North GDP/capita, longevity, and environmental quality

WG: Global GDP/capita, longevity, and environmental quality

20

11-15-03

Speeding Decoupling Performs Well in Many Futures Using North HDI Measure

Slight speed-upSlight speed-up

1.0 2.0 3.0 4.0–1.0

0

5.0

0

N$ W$

NG WG

1.0

3.0

4.0

2.0

Conventionalworld scenario

U.S. in 19thcentury

U.S. since 1950

U.S. in 20thcentury

Economic growth rate

Decoupling Rate

No regretMild

A lotOverwhelming

21

11-15-03

But Often Fails for Global Green Measure

1.0 2.0 3.0 4.0–1.0

0

1.0

5.0

3.0

4.0

2.0

0

N$ W$

NG WG

Conventionalworld scenario

Economic growth rate

Decoupling rate

No regretMild

A lotOverwhelming

Slight speed-upSlight speed-up

22

11-15-03

Exploration DemonstratesNo “Fixed” Strategy Is Robust

1.0 2.0 3.0 4.0

–1.0

0

1.0

5.0

3.0

4.0

2.0

0

Conventionalworld

scenario

No regretMild

A lotOverwhelming

1.0 2.0 3.0 4.0

–1.0

0

1.0

5.0

3.0

4.0

2.0

0

Conventionalworld

scenario

1.0 2.0 3.0 4.0

–1.0

0

1.0

5.0

3.0

4.0

2.0

0

Conventionalworld

scenario

1.0 2.0 3.0 4.0

–1.0

0

1.0

5.0

3.0

4.0

2.0

0

Conventionalworld

scenario

Economic growth rate

Decouplingrate

N$ W$

NG WG

Stay the CourseStay the Course Crash EffortCrash Effort

23

11-15-03

Design and Examine Additional Strategies

XLandscape of

plausible futures

Alternative strategies

Ensemble of scenarios

Robuststrategies

24

11-15-03

Start with a Milestone, but Evaluate Progress Early and Modify Milestone If Necessary (Safety Valve)

NODoes the carrying capacity change?

Choose policies to maximize

utility

Determine best policy to meet milestone

Select near-term milestone

YES

Is milestone achievable with

current approach?

Relax milestone

Present Future

YES

NO

Implement policy

25

11-15-03

“Safety Valve” Strategy Appears Highly Robust

Safety valveSafety valve

Economic growth rate (%)

Dec

ou

plin

g r

ate

(%)

1.0 2.0 3.0 4.0

–1.0

0

1.0

5.0

3.0

4.0

2.0

0

N$ W$

NG WG

Economic growth rate (%)1.0 2.0 3.0 4.0

–1.0

0

1.0

5.0

3.0

4.0

2.0

0

No regretMild

A lotOverwhelming

N$ W$

NG WG

+

WorstCase

U.S. in 19thcentury

U.S. since 1950

U.S. in 20thcentury

U.S. in 19thcentury

U.S. since 1950

U.S. in 20thcentury

26

11-15-03

Even Simple Scenario Generator Implies a High Dimensional Uncertainty Space

Uncertainties Levers

Economic Parameters (N&S)14 parameters

Demographic Parameters8 parameters

Environment Parameters (N&S)7 parameters

Future Generations (N&S)10 parameters

6 parameters

Measures Relationships

4 measures 14 state equations

27

11-15-03

RDM Employs an Iterative Process

Suggest candidate robust Suggest candidate robust strategystrategy

Initial choice is contingent on Initial choice is contingent on probability weighting across probability weighting across futuresfutures

Characterize breaking scenariosCharacterize breaking scenarios i.e., clusters of futures where i.e., clusters of futures where

strategy performs poorly strategy performs poorly independent of assumed independent of assumed weightingsweightings

Identify tradeoffs among well-Identify tradeoffs among well-hedged strategieshedged strategies

28

11-15-03

Scanning Across All Scenarios Suggests a Candidate Robust Strategy

No increase

Stay the course

All North

All South

Mostly North

North & some South

Policy

Regret

29

11-15-03

Analytic Tools Generate “Narrative” Scenarios

RDM identifies low-dimensional, easy-to-interpret regions where candidate strategy performs poorly Used Friedman and Fisher’s (1999) Patient Rule Induction Method (PRIM) “Low Global Decoupling” scenario is defined by 3 of 41 parameters Scenario suggests important data for consideration by decisionmakers

-0.03 -0.00288 0.03

0.0004 0.00812 0.04

-0.01 0.0139 0.05

North's Innovation Rate

Difference in InnovationRate bet. the N. and S.

North's Economic

Growth Rate

1950-99 (U.S.)

1960-99 (India)

1963-99 (Brazil)

1978-99 (China)

1993-99 (Russia)

1890-1930 &

1890-1930 (U.S.) 1950-99 (U.S.)

30

11-15-03

RDM Analysis Helps Policymakers Focus on a Small Number of Key Tradeoffs

Assessment of adaptive “milestone” sustainability strategies over two computer-generated scenarios

0.000 0.002 0.004 0.006 0.008 0.010 0.012

Regret in SV01.005.002 Satisficing Futures

0.00

0.02

0.04

0.06

0.08

Reg

ret

in L

ow

Glo

bal

Dec

ou

pli

ng

Fu

ture

s

SV02.010.015

Safety valve strategyMilestone strategy

M12

SV01.010.015

SV01.005.002

M12

SV02.005.015

M22

M0XM13

Regret in SV01.005.002 “Satisficing” Futures

Regret in low-global- decoupling futures

31

11-15-03

Analysis Ends by Characterizing Uncertainties which Drive Policy Choices

1:100 1:10 1:1 10:1 100:1

Stringent milestonesand lax cost constraints(SV01-1%-1.5%)

SV02-1%-1.5%

SV01-0.5%-0.2%

SV01-1%-1.5%RobustRegions}

SVab-x%y%a = N milestoneb = S milestonex% = N cost thresholdy% = S cost threshold

Lax/lax

Stringent/stringentStringent/lax

Lax milestonesand lax cost constraints(SV02-1%-1.5%)

Stringent milestonesand stringent cost constraints(SV01-0.5%-0.2%)

Relative Odds of A Low Decoupling Future

ExpectedRegret

32

11-15-03

Outline

Robust decision making

Example of robust decisionmaking as a means of characterizing uncertainty

Conclusions

33

11-15-03

Different Methods Appropriate in Different Circumstances

Scenario Planning

Robust DecisionsPredict-Then-Act

Uncertainty

Com

plex

ity

Hedging OpportunitiesWell-

characterizedDeep Many

Few

Low

High

34

11-15-03

Robust Decision Making Adds Another Means to Characterize Uncertainty for Decisionmakers

Information about future characterized by identifying robust strategies and their vulnerabilities

Complicated technology supports simple operational concept

Focus on alternative policies may require Closer coordination between analyst and

decisionmakers

Changes in process in organizations that use analysis

35

11-15-03

36

11-15-03

Landscape ofplausible futures

Alternative strategiesX

Ensemble of scenarios

Robuststrategies

What About Surprises?

37

11-15-03

The Advisory Panel Suggested Several

Potentially Stressing Surprises

Rapid technological advance that eliminates emissions

Plague that decimates population for twenty years

Future generations whose values (utility) are completely disconnected from concern about the environment

38

11-15-03

“Safety Valve” Strategy Is Still Robust, Even with Surprises

–1.0

0

1.0

5.0

3.0

4.0

2.0

No surprise

1.0 2.0 3.0 4.00–1.0

0

1.0

5.0

3.0

4.0

2.0

Population surprise

–1.0

0

1.0

5.0

3.0

4.0

2.0

Technological surprise

1.0 2.0 3.0 4.00–1.0

0

1.0

5.0

3.0

4.0

2.0

Value surprise

N$ W$

NG WG

Economic growth rate

Rate of change in emissions intensity