Subjectivity in Decision Analysis
David L. OlsonJames & H.K. Stuart Professor in MIS
University of Nebraska Lincoln
Human Centered Processes - 2008
Rational Choice TheoryK. Morrell, Journal of Business Ethics [2004]
• Dominant model for business decision making
• Compared with Image Theory– Utilitarian ethics
• both consistent
– Kantian ethics• Image theory yes, RCT no
– Virtue-based ethics• Image theory yes, RCT no
Human Centered Processes - 2008
Ideal
• Objective measuresMax Weber
• Accurate preference input
• “Rational” decision makerAccounting: Jensen - Agency Theory
Economics: Williamson - Transaction Cost Analysis
Human Centered Processes - 2008
Basic Preference Model
Can use multiplicative model for interactions
ijs
iw
K
j
i
swValue
ij
i
K
iijij
criterion on ealternativ of score
criterion ofweight
criteria ofnumber
index ealternativ
indexcriterion 1
Human Centered Processes - 2008
Objective Measures
• Objective preferred– can measure
• past profit, after tax
• Subjective– know conceptually, but can’t accurately
measure• response to advertising
Human Centered Processes - 2008
Problems with Objective Approach
von Neumann & Morgenstern [1944] Theory of Games– utility is measurable
Georgescu-Roegen [1954] The Quarterly Journal of Economics – requires to many assumptions of rationality
Lindblom [1965] Public Administration Review– muddling through
Morgenstern [1972] Journal of Economic Literature – 13 critical points
• uncertainty
• ambiguity
• disagreement in groupsHuman Centered Processes - 2008
EMPIRICAL EVIDENCE
contrary to rational choice models
Braybrooke & Lindblom [1969]; Simon [1985] Payne, et al. [1993]
• Some problems never reach decision maker
• decision makers often have simple maps of real problems
• all alternatives not known, so decision makers do not have full, relevant information
• individual altruism
Tversky [1969]
• systematic & predictable economic intransitivities
Kahneman, Slovic & Tversky [1982]
• people use heuristics rather than follow rational model
Human Centered Processes - 2008
James G. March
Bell Journal of Economics [1978]
• Rational choice involves guesses:– About future consequences of current actions– About future preferences of those consequences
Administrative Science Quarterly [1996]
• Alternatives & their consequences aren’t given, but need to be discovered & estimated
• Bases of action aren’t reality, but perceptions of reality• Supplemental exchange theories emphasize the role of
institutions in defining terms of rationality
Human Centered Processes - 2008
Overview
• Inputs to preference models involve subjectivity– Weights are function of individual
– Scores also valued from perspective of individual
• Subjective assessment MAY be more accurate• Purpose of analysis should be to design better
alternatives
Human Centered Processes - 2008
Means to Cope
Payne, Bettman & Johnson [1993]
• strategy will differ by number of alternatives– few - focus on all relevant information
– many - noncompensatory simplifying heuristics
• $/lives tradeoff varies by context
• Hogarth: many find explicit tradeoffs uncomfortable
• PROSPECT THEORY: initial analysis simple, weed out; for selected alternatives, more detailed
• As people learn more about problem structure, construct choice strategies
Human Centered Processes - 2008
Objective/Subjective
• OBJECTIVE: what is convenient to model– ideal - eliminate bias, arbitrary judgment– extreme: cost/benefit analysis spanning years of
measuring the unmeasurable
• SUBJECTIVE: what people do to cope– value is subjective after all anyway– value is what MAUT, MCDA seeks to measure
Human Centered Processes - 2008
ACCURATE PREFERENCE INPUT
• incomplete information• uncertain measures
• uncertain preferences
• group participation
• risk• time pressure: Edwards - how can you calculate
expected utility in available time?• change competition complexity
Human Centered Processes - 2008
RELATIONSHIP TO MCDA
• We shouldn’t expect so much theoretical purity– the world has shifted away from appreciation of
numerical analysis
• Just because assumptions are not met doesn’t mean pure approach better
Human Centered Processes - 2008
MCDA Methods
• Multiattribute utility theory
• Analytic hierarchy process
• Outranking– ELECTRE, PROMETHEE
• Fuzzy, DEA, Verbal Decision Analysis
• Image Theory
Human Centered Processes - 2008
Spectrum
MAUT with strictly objective measures
MAUT with constructed measures
Likert scales
SMART - swing weighting rather than lottery tradeoffs
AHP - ratios of subjective scale
Human Centered Processes - 2008
PROMETHEE Spectrum
Class I:ordinal
Class II: step advantage
Class III: proportional advantage (in range)
Class IV: three step
Class V: proportional with indifference
range
Class VI: normal distributionHuman Centered Processes - 2008
MAUT Hierarchy
Overall
Human Centered Processes - 2008
Cost$billions
Lives lostExpected value
RiskProbability of major catastrophe
Civic improvement Families with upgraded housing
Objective Measures
Cost Lives Lost Risk Civic improvement
Nome AK 39.548 billion 61 0.0165 312 upgrades
Newark NJ 98.467 billion 143 0.0002 68,472 upgrades
Rock Springs WY
58.930 billion 41 0.0036 4,138 upgrades
Duquesne PA 60.156 billion 39 0.0069 20,653 upgrades
Gary IN 69.693 billion 86 0.0027 56,847 upgrades
Human Centered Processes - 2008
Swing Weighting
Attribute On BEST On WORST
W on Best W on Worst
Final
Lives lost 100 290 0.541 0.569 0.556
Risk 60 175 0.324 0.343 0.333
Cost 20 35 0.108 0.069 0.089
Civic 5 10 0.027 0.020 0.022
185 510
Human Centered Processes - 2008
SMART with swing weighting
Cost Lives lost Risk Civic imp RESULT
Weights 0.089 0.556 0.333 0.022
Nome AK 0.991 0.564 0.175 0.003 0.468 (4)
Newark NJ 0.026 0.007 0.990 0.685 0.351 (5)
Rock Springs WY
0.685 0.707 0.820 0.041 0.736 (1)
Duquesne PA
0.664 0.721 0.655 0.207 0.675 (2)
Gary IN 0.505 0.386 0.865 0.568 0.552 (3)
Human Centered Processes - 2008
Logical Decision
• Hierarchy of criteria• Single-attribute Utility Functions
– Worst imaginable utility = 0
– Best imaginable utility = 1
– Assess 0.5 level of either value or utility
• Tradeoffs– Pairwise comparisons
– Select preferred extreme
– Improve other until equalHuman Centered Processes - 2008
SUF for Lives Lost
PROBLEM: sensitive to limits set – may warp values more than intended
Human Centered Processes - 2008
Weight TradeoffsPROBLEM: weights reflect both choices, scale – again hard to control
• Cost < Lives Lost[0,1000]>[500,0] [0,1000]=[5,1000]
• Cost < Risk[0,1]>[500,0] [0,0.35]=[500,0]
• Cost > Civic Improvement[0,100000]>[500,0] [0,100000]=[350,0]
• Weights (including scale):– Risk 5.029– Cost 1.577– Lives Lost 29.337– Civic Improvement 64.058
Tradeoff: Cost vs. Civic Improvement
Human Centered Processes - 2008
Result
Human Centered Processes - 2008
Contributions by Criteria
Human Centered Processes - 2008
Rock Springs vs. Newark
Human Centered Processes - 2008
Subjective Ratings
Cost Lives Lost Risk Civic improvement
Nome AK Moderate Low Very high Low
Newark NJ Very high Very high Very low Very high
Rock Springs WY
High Very low Low High
Duquesne PA High Very low Medium Medium
Gary IN Higher High Low Very high
Human Centered Processes - 2008
Subjective SMART
Cost Lives Lost Risk Civic Imp RESULT
weights 0.089 0.556 0.333 0.022
Nome AK 0.8 0.7 0 0.2 0.465 (3)
Newark NJ 0 0 1 1 0.355 (5)
Rock Springs WY
0.4 0.9 0.8 0.8 0.820 (1)
Duquesne PA
0.4 0.9 0.5 0.6 0.716 (2)
Gary IN 0.2 0.2 0.8 1 0.417 (4)
Human Centered Processes - 2008
Output Comparisons
MAUT SMART SMART subjective
Nome AK 0.629 (5) 0.468 (4) 0.465 (3)
Newark NJ 0.794 (3) 0.351 (5) 0.355 (5)
Rock Springs WY 0.723 (4) 0.736 (1) 0.820 (1)
Duquesne PA 0.839 (1) 0.675 (2) 0.716 (2)
Gary IN 0.830 (2) 0.552 (3) 0.417 (4)
Human Centered Processes - 2008
Comparison with SMART
• Simpler allows decision maker to see exactly what ratings are
• MAUT– Distrusts human – masks tradeoffs in effort to make “objective”
– “Objective” here means have no idea
– Theoretically, preferences will be identical
– Does allow for nonlinear interaction, but severe impact
• My contention:– DIRECT IS BETTER THAN MACHINE
Human Centered Processes - 2008
Image Theory
INTEREST GROUP Criteria reflecting concerns
Government Cost, lives lost, risk, civic improvement
Nuclear Industry Permanent storage of nuclear wasteNuclear power demand
Local Citizens EquityCultural artifactsEmployment
General Population Nuclear power generation safetyTransportation riskLow-cost electricity
Human Centered Processes - 2008
Image Theory process
• Frame decision– Desired states
– Actions needed to attain desired states
– MORE CRITERIA
• Helpful to MCDA in structuring– Context
• Elicit participation of as many views as possible
– Identify alternatives• Design an ideal rather than settle for existing
• MORE ALTERNATIVES
Human Centered Processes - 2008
Verbal Decision Analysis
• Controlled pairwise comparisons of tradeoffs• Focus on critical criteria
– Don’t use falsely precise measures• Fuzzify – categorical ratings
• Screen alternatives– Preemptive
• Focus on critical tradeoffs
Human Centered Processes - 2008
VDA Process
• Eliminate very high lives lost– Newark eliminated
• Eliminated risk high or worse– Nome eliminated
• Rock Springs now dominates Duquesne
• FOCUS ON– Rock Springs
– Gary
• TRADEOFFS– Rock Springs – a little lower cost, improved lives lost
– Gary – civic improvement slightly better
Human Centered Processes - 2008
Inferences
• Objective can’t capture all the complexity of real decisions– OBJECTIVES ARE ALWAYS LEFT OUT
– Conventional wisdom – at most 7 matter
– BUT THERE IS NO PARETO OPTIMAL unless all considered
• When Groups are involved, THERE IS NO ONE BEST DECISION– Ward Edwards – never saw a group pick an option that was first
choice of one subgroup
– NEGOTIATION
Human Centered Processes - 2008
Conclusion
• Measures of alternative future performance, preference for that performance both subjective
• Objective measures not always better
• Focus should be on:– Learning (changing preference)– Design of better alternatives (Image Theory)