![Page 1: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/1.jpg)
Linear Models of Judgment
Judgment vs. choice Multiattribute model of judgment Actuarial model of the
environment Experts and computers Bootstrapping models
M ileage P rice
R epairs
C os t
C rash Test A B S
A irbags
S afety
Looks H andling
M arque Features
Fun
E xternal C abin
T runk
S ize
D esirability
![Page 2: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/2.jpg)
Judgment vs. Choice
Judgment = assign a score or categorye.g., How much would you pay for a one-week trip to Aspen?How much do you like Bill Clinton?What will the price of Intel be in 6 months?
Choice = pick from a set of alternativese.g., which car? investment? job?
![Page 3: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/3.jpg)
Multiattribute Choice Model Choice = select most desirable Desirability is judged from attributes Attributes can be FACTS (e.g., price),
COMPOSITES (e.g., safety), or subjective VALUES (e.g., prestige)
There is often a hierarchical structure to attributes and judgments
How to make tradeoffs, e.g., weight & add
![Page 4: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/4.jpg)
Desirability of a Car
M ileage P rice
R epairs
C os t
C rash T est A B S
A irbags
S afety
Looks H andling
M arque F eatures
F un
E xternal C abin
T runk
S ize
D esirability
![Page 5: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/5.jpg)
Lens Model
Judgment (Ys) is an attempt to represent or predict the environment from cues
There is a criterion (Ye) that allows us to estimate correctness of the judgment
YsYe
c
u
e
s
![Page 6: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/6.jpg)
Models of Decision Makers
Slovic’s study of two stockbrokers:A: near term prospects, P/E, earnings qtly trend
B: earnings yearly trend, P/E, profit margin trend
Ratings of business schools:USNWR: reputation with academics, reputation with CEOs, selectivity, placement
BusWeek: recruiters rate analytics, teaming, global; graduates rate teaching, curriculum,
placement
![Page 7: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/7.jpg)
Actuarial Environmental Models Occupation
clergy 46executive 62professional 62student 46teacher 46unemployed 33no answer 47
Job Tenure< .5 years 31.5 - 5.5 245.5 - 8.5 268.5 - 15.5 31> 15.5 years 39
Capon, J. Marketing, 1982, 46, 82-91.
Major Retailer’s Credit Scoring Table
Older economists make more extreme forecasts
![Page 8: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/8.jpg)
Comparisons Using Models
How consistent are individuals? How consensual are experts? How accurate are judges? (Ye vs. Ys)
What are judges doing? (Ysm)
What predicts the criterion? (Yem) How good are our models? Do judges understand the environment?
(Yem vs. Ysm)
![Page 9: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/9.jpg)
Graduate Admissions Example Ys = judgment of admissions committee
(1 to 5 scale) Ye = faculty ratings of performance
Ysm = prediction model of judgments
= -4.17 +.0032*GRE +1.02*GPA +.0791*QI Yem= actuarial model of performance
= -.71 +.0006*GRE +.76*GPA +.2518*QI
![Page 10: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/10.jpg)
Admission and Job Interviews
Harvard Business School stopped conducting interviews– Are interviews accurate?– Are interviews overweighted?– What is their proper role?
HBS no longer uses the GMAT HBS criteria: academics and
character
![Page 11: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/11.jpg)
Advantages of Models
Makes strategy explicit Can see how experts vary Train new judges Learn about environments Enhance or replace experts Can use the model when expert
gone
![Page 12: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/12.jpg)
Judges vs. Environment
Which should be more accurate, expert judges or actuarial models?
Judges have their experience, ability to use cues in complex ways
Actuarial models are simple, typically linear in form, consistent
![Page 13: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/13.jpg)
Judges vs. Actuarial Model
Task Judge Actuarial Model
Credit scoring .80 .95
Stock analysis .23 .80
Personnel .35 .57
Cancer survival -.01 .35
Graduate GPA .33 .69
![Page 14: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/14.jpg)
Why Don’t Experts Do Better? They have the wrong rules They don’t use their rules
- distractions- fatigue, boredom- “exceptions”- unable to make tradeoffs
![Page 15: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/15.jpg)
Bootstrapping Models
If intuitive decision makers have good rules but fail to use them consistently, can we separate signal from noise?
Consensus of judges (see groups later) Model of a judge (bootstrapping)
Judgment = Linear + Nonlinear + Noise What wins: Judge vs. linear model?
![Page 16: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/16.jpg)
A Tale of Three Models
Task JudgeBootstrapModel
ActuarialModel
Credit scoring .80 .85 .95
Stock analysis .23 .29 .80
Personnel .35 .46 .57
Cancer survival -.01 .13 .35
Graduate GPA .33 .50 .69
![Page 17: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/17.jpg)
Some Typical Results
Some tasks are much harder than others Actuarial models almost always win Bootstrapping works! Linear models correlate with any
monotonic function, work well when there is noise, positively correlated cues, work with random or unit weights
To improve on linear models, you need lots of data
![Page 18: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/18.jpg)
Experts and Models
What do experts do best? What do computers do best? How can they be combined? Should we give the model to the
expert or give the expert to the model?
![Page 19: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/19.jpg)
Batterymarch Example
Stock portfolio company Manage $12 Billion with 37
employees Experts identify variables, suggest
rules, design tests, deal with clients Computer keeps databases, runs
tests of rules, buys and sells stocks 10-12 rules identify attractive stocks
![Page 20: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/20.jpg)
Working With the Political Lens: Separating Facts and Values Selecting a bullet for Denver Police
- police want to immobilize suspects- community concerned about injuries- experts testify on each side
What kinds of information are needed? How should this decision be made?
![Page 21: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/21.jpg)
A Frame for Conflict Resolution
Facts Values Desirabilityweight Injury potentialspeedshape Stopping poweretc.
Threat tobystanders
![Page 22: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/22.jpg)
Denver Bullet Resolution Experts combine facts into
judgments on each value Constituencies compromise on how
to weight the values into overall worth
Stopping Power
InjuryPotential
. . . . . . . . . . . . . . . . . . . . P. . . . . . . . . . . . . . . . . . . . . . C. . . . . . . . .. . . . . . . . . . .
proposed by Police
proposed by Community
![Page 23: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/23.jpg)
Role of Technical Experts
Executive whose daughter had a hip deformity
One doctor said, “Wait” A second said, “Brace for 6 months” The third said, “Operate” How would you make this decision?
![Page 24: Linear Models of Judgment v Judgment vs. choice v Multiattribute model of judgment v Actuarial model of the environment v Experts and computers v Bootstrapping](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e8e5503460f94b9233c/html5/thumbnails/24.jpg)
Your Exercise #1: Job Selection What were the attributes or objectives
of jobs that mattered to you? How different were the rankings due to
intuition, weighted linear model, unit weighted model?
If the rankings differ, which do you trust? Why?
Value-added in the process, not the numbers