1

Decision making under uncertainty - is sensitivity

analysis of any use ?

Stein W. WallaceMolde University College

2

Decision making under uncertainty

• Sensitivity analysis• Parametric optimization• “What-if”-analysis• Scenario analysis

Do these work?

3

Mathematical programming, especially linear programming and related network and combinatorial methods, usually form the OR/MS deterministic subfield. It is time to recognize that this categorization is restrictive and does not serve the field well.Those of us who work in the area are, in a sense, blessed and lucky. We have in the linear programming mathematical model and in its solution by the simplex method a readily available analysis that answers important data sensitivity questions and, at the same time, yields critical related economic information. Coupling such an analysis with computationally simple studies provides a rather nondeterministic view of the modellingsituation. Thus, those of us who teach and practice mathematical programming have the means of emphasizing and answering concerns about validity, robustness, uncertain data, base case and scenario analysis, and in achieving the truism that modeling is more about gaining insight than in producing numbers. We can and do cut across the dichotomy. (Saul Gass - 1997)

4

In all LP models the coefficients of the objective function and the constraints are supplied as input data or as parameters to the model. The optimal solution obtained by the simplex method is based on the values of these coefficients. In practice the values of these coefficients are seldom known with absolute certainty, because many of them are functions of some uncontrollable parameters. For instance, future demands, the cost of raw materials, or the cost of energy resources cannot be predicted with complete accuracy before the problem is solved. Hence the solution of a practical problem is not complete with the mere determination of the optimal solution. Each variation in the values of the data coefficients changes the LP problem, which may in turn affect the optimal solution found earlier. In order to develop an overall strategy to meet the various contingencies, one has to study how the optimal solution will change with changes in the input (data) coefficients. This is known as sensitivity analysis or post-optimality analysis.

(Ravindran, Phillips and Solberg 1987)

5

ExampleYou have two lots of land that you can

developDevelopingthe land

Building theplant now

Building theplant later

Lot 1 600 200 220Lot2 100 600 660

Plants produce one unit of a good sold at price p, presently unknown

6

Nine possible solutions

No 9 is to donothing

7

Sensitivity analysisSample (or construct) possible futures, and solve for each of these possible futures. Compare (combine) solutions to find an overall solution.

Delayed constructionis never used

p < 700 : Do nothing

700 < p < 800 P > 800 No other scenario solutions arepossible

Flexibility has no value

8

Assume p can take on two different valuesPr(p=210)=Pr(p=1250)=0.5

Expected value p=730

Deterministic problem: Use p=730 and get Decision 4with profit 730-700=30

Expected value of scenario solutions:

• Decision 4: -700+0.5*210+0.5*1250=30• Decision 9: 0• Decision 7: -1500+0.5*2*210+0.5*2*1250= -40

9

Decision Invest Income ifp=210

Income ifp=1250

Expectedprofit

1 600 0 1030 -85

2 800 210 1250 -70

3 100 0 590 195

4 700 210 1250 30

5 1300 210 2280 -55

6 900 210 1840 125

7 1500 420 2500 -40

8 700 0 1620 110

9 0 0 0 0

Expected profitfor all scenarios

Delayed construction usedwheneverprofitable

10

A continuous example

zyx ++ 23max

01

1

,,

ξ

ξ

−

≥

≤

≤

≤

++

zz

yyy

xx

x Substitutes with demand summing to one

Production capacity

11

We can find the optimal solution for all possible demands

01

=

−=

=

zty

tx[ ]1,0for ∈t

All scenario solutions are infeasible when you try to calculate the expected performance

Optimal solution: 1,0 === zyx

12

What do we see ?

• All scenario solutions have x+y=1, forcing z=0. That is exactly what we do not want.

• We have solved all possible problems• All scenario solutions have the same

optimal basis. The optimal solution has a different optimal basis.

• The space spanned by scenario solutions does not contain the optimal solution

13

Connections to options

• Scenario solutions disregard flexibility. • Options (= flexible alternatives) are

worthless in a deterministic analysis, unless they are free.

• We do not have follow-up (recourse) decisions as required.

14

The IQ of hindsight

• After the fact one of the scenario solutions will turn out to be best

• The stochastic programming solution is hardly ever best

• Be careful with how decision makers are evaluated

15

An example

• A MIP model has been made to support the development of oil and gas fields in addition to the transportation infrastructure in the North Sea. The model takes more than one hour to run. 50 runs have been made with different levels of demand, prices, and field sizes.

16

• In 80% of all cases, the gas pipe from A to B has a diameter of 24”. In the other cases it is larger. Hence, we can conclude that 24” is a lower bound.

• In all cases, field C is developed between 2010 and 2020. Hence, we can safely assume that we have found a time interval in which the field is developed. Choosing 2015 is clearly a good approximation.

17

• Subfields D and E compete for the same production capacity at a platform. In all cases subfield D is chosen. Hence, we can safely disregard subfield E in the analyses to follow.

• Does anything change if we are able to make these statements for all scenarios and not just the 50 chosen ?

18

• The conclusions based on sensitivity analysis may be good or correct

• The arguments leading to the conclusions are false.

19

Option values

• All decision have embedded option values, positive or negative– A decision opens and closes doors

• The higher variance, the higher option value– If we underestimate the randomness, we

underestimate the option values• Scenario analysis gives no value to options

20

Organizational issues

• The float of information in an organization – randomness is often lost between departments

• Will result in reduced randomness and hence reduced value of flexibility.

21

Board rooms

• Leaders want clear advise and explanations

• Results in randomness being suppressed and hence flexibility being undervalued

22

Discount rates

• Many companies use very high discount rates to express that only very profitable projects will be accepted

• Investments in flexibility normally have costs early and potential income late

• Results in reduced option values and incorrect investments in accepted projects

23

Subjective issues

• Over confidence in own estimates– reduces option values

• Hindsight learning -– we have problems seeing that other scenarios

could have happened– bad learning

24

Flexibility is often important, but ...

• Disregarded by methods• Lost in organizations• Undervalued by individuals

25

Regret and robust solutions

• Expected value solutions may have a large variance.

• We may be risk averse.• Maybe we regret not having made

another decision?• We especially hate large regrets.

26

Limits on the regret• We may choose to

– maximize expected profit with a bound on the largest regret

–Maximize the expected profit minus a function of the variance

• Examples–To follow– Markowitz’ mean-variance model

27

Regret for the nine possible solutions

If we limit regret to maximum 500, we will choose Decision 4, and the expected profit will drop from 195 to 30.

28

Alternative• Buy insurance (an option) against high

prices• High prices happen with 50%

probability• Pay 10 (plus a fee) to get 20 if prices

are high

• The expected profit for Decision 3 is now 195 minus the insurance fee, and the maximal regret is down to 500.

29

Time related entities

• Period = time step in model• Stage = point in time where it makes

sense to make a new decision

New information is needed to make new decisions

New information is only interesting if new decisions can be made

30

How many stages?

• Enough to capture all the important consequences of the first stage solution

• Enough to capture a time span similar to the real decision problem

31

Correct use of sensitivity analysis

• A priori analysis, under uncertainty, of decisions that eventually will be made under certainty.

• Analysis of changes in deterministic parameters.

Mathematical programming, especially linear programming and related network and combinatorial methods, usually form the OR/MS deterministic subfield. It is time to recognize that this categorization is restrictive and does not serve the field well.Those of us who work in the area are, in a sense, blessed and lucky. We have in the linear programming mathematical model and in its solution by the simplex method a readily available analysis that answers important data sensitivity questions and, at the same time, yields critical related economic information. Coupling such an analysis with computationally simple studies provides a rather nondeterministic view of the modellingsituation. Thus, those of us who teach and practice mathematical programming have the means of emphasizing and answering concerns about validity, robustness, uncertain data, base case and scenario analysis, and in achieving the truism that modeling is more about gaining insight than in producing numbers. We can and do cut across the dichotomy. (Saul Gass - 1997)

33

In all LP models the coefficients of the objective function and the constraints are supplied as input data or as parameters to the model. The optimal solution obtained by the simplex method is based on the values of these coefficients. In practice the values of these coefficients are seldom known with absolute certainty, because many of them are functions of some uncontrollable parameters. For instance, future demands, the cost of raw materials, or the cost of energy resources cannot be predicted with complete accuracy before the problem is solved. Hence the solution of a practical problem is not complete with the mere determination of the optimal solution. Each variation in the values of the data coefficients changes the LP problem, which may in turn affect the optimal solution found earlier. In order to develop an overall strategy to meet the various contingencies, one has to study how the optimal solution will change with changes in the input (data) coefficients. This is known as sensitivity analysis or post-optimality analysis.

(Ravindran, Phillips and Solberg 1987)

34

The sad conclusion

• Sensitivity analysis, and its relatives, are not theoretically valid methods for analyzing decision making under uncertainty.

• They can lead to arbitrarily bad conclusions.

• Methods like stochastic programming and (real) option theory are needed.