forecasting decisions in conflicts: best methods for supply chain, competition, union- management,...
Post on 21-Dec-2015
214 views
TRANSCRIPT
Forecasting decisions in conflicts:Best methods for supply chain, competition, union-
management, and takeover strategy problems
Ehrenberg Centre for Research in Marketing
Monday 15th November 2010 at 5:00 PM
London South Bank University
Kesten C. GreenEhrenberg-Bass Institute for Marketing Science
International Graduate School of Business University of South Australia
kestencgreen.com ForPrin.com AdPrin.com
2
Scientific* forecasting methods
Procedures for making predictions about matters currently unknown, based on:
- empirical comparisons of proper alternatives
- ex ante tests of accuracy given stated conditions(e.g. level of knowledge about the situation)
*Used interchangeably with ”evidence-based”
3
Evidence-based methods Knowledge advances when multiple hypotheses are tested, especially if hypotheses challenge accepted wisdom, e.g.:
• Market-share objectives harm profits.
• Minimum-wage laws harm low-skilled workers
• Regulation harms consumers
• Pre-announced satisfaction surveys harm satisfaction
• Anti-inflammatory drugs harm head injury patients
4
Principles for the selection and application of forecasting methods
Mid-1990s: Wharton School’s Scott Armstrong started the “principles of forecasting project” to summarize all knowledge about forecasting in the form of scientific principles.
A principle is a condition-action statement.
This project led to 139 principles as described in the Principles of Forecasting handbook.
39 authors & 123 reviewers
The principles (currently 140 in number) can guide the selection and application of methods
5
ForPrin.com features
• Descriptions of 140 forecasting principles
• The Forecasting Canon with 9 key rules
• Answers to Frequently Asked Questions
• The Forecasting Dictionary
• Forecasting Methodology & Selection Trees
• Forecasting Audit Software
• Resources for practitioners, educators, researchers
• Special Interest Groups (SIGs)
– E.g. ConflictForecasting.com
6
Why not just ask an expert what will happen?
Most decisions in business based on managers’, or other experts’, judgments about what will happen, but…
Tetlock (2005): evaluated • 82,361 forecasts • made over 20 years
• by 284 professional commentators and advisors on politics and economics
Expertise did not lead to better forecasts… (but their excuses are better than novices’!)
Tetlock’s finding is consistent with other findings from research on forecasting by unaided experts.
7
Today’s issue: “predicting decisions in
conflicts”Predicting decisions of
– parties with
– divergent interests,
– who interact
Note the focus here is on predicting decisions, not outcomes.
8
Examples of conflict situations • What reactions to a first strike in Iran?• What offer to make in a union/management
negotiation?• How to respond to demands by mob protesters?• How to best resolve legal conflicts?• How will proposed water sharing regimes work?• How to design policy for…
• benefits?• tax? • job security?• market regulation?
…all require predictions of how people will respond.
9
Forecasting decisions in conflicts:
Research problems
Artists protest: Artists stage sit-in & demand funding
Nurses dispute: Demand same pay increase
Distribution channel: Novel proposal to market appliances
Telco takeover bid: Bid for all after rejecting mobile offer
55% pay plan: NFL players demand 55% of revenue
Water dispute: Troops mass & threat to bomb dam
Employee grievance: Mediation when job down-graded
Zenith investment: Investment decision with politics
10
Water Dispute
Syria is filling a new dam on the Euphrates River, thereby slowing the flow of water into Iraq in Spring 1975.
Syria and Iraq mass troops on their border, both threatening invasion.
Saudi Arabia offers mediation.
11
Telco takeover bid
CenturyTel approaches AllTel with an offer to sell its mobile business.
AllTel rejects the offer, offers to pay a 50%+ premium for the whole business.
CenturyTel rejects counter offer, and AllTel pursues takover bid.
12
Methods for forecasting decisions in conflicts
Novices Experts
Guessing 28% 28%
Unaided judgment
Role thinking
Game theory
Structured analogies
Simulated interaction
13
Unaided forecasts from experts on…
• conflict management• political science• industrial relations• marketing• judgement & decision making• forecasting• game theory
14
Unaided judgment method
The most commonly used method for predicting decisions in conflict situations.
Appropriate when:• experts are unbiased• large changes are unlikely• relationships are known• experts get useful feedback from many similar cases
Expert and novice subjects read descriptions and made predictions.
15
Unaided judgment accuracy
3228Averages (unweighted)
7333Nurses Dispute
5033Water Dispute
3125Personal Grievance
3633Zenith Investment
182555% Pay Plan
025Telco Takeover
3833Distribution Channel
1017 Artists Protest
Bold = more accurate
Chance UJ-ExpertsPercent Accurate Forecasts
UJ-Naive
5
5
10
27
29
44
4568
29
16
Effect of experience & time-spent on accuracy
Percent correct forecasts
<5 yrs 5 yrs+Experience 36 29
<30 min30 min+
Time spent 32 35
17
Do game theorists recommend game theory for forecasting?
Yes, judging from• textbooks• papers• consultants’ advertisements
Google searches: “game theory” & “prediction” or “forecasting” . . .2,170,000 as of 15 November, 2010
adding “conflicts” to the search, yielded 789,000
18
Game theorists’ forecasts
Game theorists made predictions in response to a request that read “Using game theory to predict the outcomes of conflicts”
Hundreds invited; 23 participated
Respondents selected only some of the situations, yielding 101 forecasts
19
Game theorists’ accuracy
Telco Takeover Bid 0 0
Artists’ Protest 10 6
55% Pay Plan 18 29
Employee Grievance 31 43
Zenith Investment 36 22
Distribution Channel 38 23
Water Dispute 50 75
Nurses Dispute 73 50
Averages (unweighted) 32 31
UJ GT
Percent Accurate Forecasts
20
Use of analogies in forecasting
Analogical reasoning commonly used (informally) for prediction
58% of respondents said their organizations used analogies to forecast competitor actions (Armstrong, Brodie & McIntyre 1987)
Neustadt & May (1986). Thinking in time: The uses of history for decision makers.
Kahneman & Lovallo (1993) example
21
Structured Analogies
Domain experts individually:
1. list similar situations2. rate similarity to target situation3. match outcome to target situation
An administrator mechanically derives forecast (e.g., select outcome of the most similar situation as forecast).
Little prior evidence on structured analogies
22
Structured analogies method International Water Dispute
1) (A) In the table below, please briefly describe (i) your analogies, (ii) their source (e.g. your own experience, media reports, history, literature, etc.), and (iii) the main similarities and differences between your analogies and this situation. (B) Rate analogies out of 10 (0 = no similarity… 5 = similar… 10 = high similarity). (C) Enter the responses from question 2 (below) closest to the outcomes of your analogies.
(A) (i) description, (ii) source, (iii) similarities & differences
(B) Rate
(C) Q2
a.
b.
c.
d.
e.
2) The gist of the statement issued at the end of the meeting was? (check one , or %)
a. Midistan has decided to release additional water in order to meet the needs of the Deltalandish people [__] b. Deltaland has ordered the bombing of the dam at Mididam to release water for the needy Deltalandish people [__] c. Deltaland has declared war on Midistan [__]
23
Structured analogies accuracy
Telco Takeover Bid 0 14
Artists’ Protest 10 50
55% Pay Plan 18 80
Employee Grievance 31 50
Zenith Investment 36 50
Distribution Channel 38 67
Water Dispute 50 88
Nurses Dispute 73 50
Averages (unweighted) 32 56
UJ SA2+
Percent Accurate Forecasts
24
Use of structured analogies Percent
Correct
Unaided experts’ judgment 32
Structured analogies with experts’ predictions
42
Structured analogies withmechanical predictions
46
Structured analogies withexperts who “knew better”
25
25
Number of analogies and familiarity
% CorrectNumber of analogies (forecasts)
one 38 (53)two or more
56 (44)
Familiarity with analogiesindirect 37 (45)direct 49 (50)
+ two or more 60 (23)
ConclusionUse experts who have direct experience with analogous conflicts
26
Collaborating for analogies
Collaborators were more experienced and spent more time than solo forecasters. % correct (forecasts)Solo 44 (75)Collaborative 42 (22)
Conclusions:
• Collaboration is not effective when using analogies. • But, there is little harm if people want to discuss analogies.
27
Role thinking process
In the Fog of War, Robert McNamara concluded that one should put oneself in the shoes of an opponent.
Subjects received information about the situations along with the information about roles and were asked to think about the roles before making a prediction
28
Role thinking processInternational Water Dispute
1) A person’s role can make a big difference to how he or she views a situation, so it can be difficult to predict
what decisions will be made when people interact with each other. In the following table, please indicate which decision you think each party in the situation just described would prefer to be made and assess how likely it is that each party’s preferred decision will actually occur.
For each party in the conflict, please use your judgment to:
(A) (i) select from the following list the decision or decisions the party would prefer to see emerge from today’s meeting:
a. Midistan has decided to release additional water in order to meet the needs of the Deltalandish people b. Deltaland has ordered the bombing of the dam at Mididam to release water for the needy Deltalandish people c. Deltaland has declared war on Midistan (ii) explain why you think the party prefers that decision or those decisions
(B) (i) explain how you think the party would go about trying to achieve its most-preferred decision (ii) rate the chances that the decision will be made, out of 10 (0 = almost no chance…10 = almost certain)
Party
(A)(i) Party’s preferred decision(s) from a-c, above
(ii) Why preferred
(B)(i) How party would try to achieve most-preferred decision
(ii) Chances that most-preferred decision will be made(0-10)
Midistan (i) [______] (ii) (i)
(ii) [___]
Deltaland (i) [______] (ii) (i)
(ii) [___]
Concordia (i) [______] (ii) (i)
(ii) [___] 2) Given your analysis in Q1, which of the decisions listed in (A)(i) above is most likely? [____] a-c 3) Why will that (Q2) decision occur, and why might it not occur?
29
Role thinking accuracy
Distribution Plan 38 0 0
Artists Protest 10 8 13
55% Pay Plan 18 13 8
Telco Takeover 0 13 18
Journal Negotiations - 25 30
Personal Grievance 31 - 36
Zenith Investment 36 46 55
Water Dispute 50 75 56
Nurses Dispute 73 82 73
Averages (unweighted) 32 33 31
Unaided Judgment (Experts)
Role Thinking (Novices)
Percent Accurate ForecastsRole Thinking (Experts)
30
Simulated interaction (SI) process
Brief role description, name badge (2+ roles)
One-page situation description
Simulate interactions (i.e., role play realistic interactions)
Interactions take less than an hour
Outcome (decision) of the simulation used as a forecast
31
Simulated interaction accuracy(Naïve Subjects)
Telco Takeover Bid 0 40
Artists’ Protest 10 29
55% Pay Plan 18 60
Employee Grievance 31 60
Zenith Investment 36 59
Distribution Channel 38 75
Water Dispute 50 90
Nurses Dispute 73 82
Averages (unweighted) 32 62
UJ
Percent Correct Predictions
SI (N)
32
Summary of accuracy
Telco Takeover Bid 0 (8) 0 (7) 14 (7) 40 (10)
Artists’ Protest 10 (20) 6 (17) 50 (4) 29 (14)
55% Pay Plan 18 (11) 29 (17) 80 (5) 60 (10)
Employee Grievance 31 (13) 43 (7) 50 (6) 60 (10)
Zenith Investment 36 (14) 22 (18) 50 (4) 59 (17)
Distribution Channel 38 (17) 23 (13) 67 (6) 75 (12)
Water Dispute 50 (8) 75 (8) 88 (8) 90 (10)
Nurses Dispute 73 (15) 50 (14) 50 (4) 82 (22)
Averages (unweighted) 32 (106) 31 (101) 56 (44) 62 (105)
UJ (E) SA2+
Percent Correct Predictions
SI (N)GT
_______Experts________ Non-experts
33
Combine forecasts “mechanically”:Avoid face-to-face meetings
• A meta analysis of 30 studies found 12% error reduction (3% to 24%) with combinations always more accurate than typical individual forecasts.
• With favourable conditions (election forecasts)*, error reductions averaged between 42% and 50% compared to typical individual forecasts.
• Each component must contain some information.
*Several valid forecasting methods using different information sources allowing combining within and between methods.ReferenceGraefe, et al. (2010).
34
The logic of combining
How bad can a second forecast (F2) be and
still give an average that is no worse than
F1?
------------------------------A--------F1----
35
The logic of combining 2
How bad can a second forecast (F2) be and
still give an average that is no worse than
F1?
Answer: The error can be the same size, or up to three-times bigger if it is the opposite sign.
------- F2 -----------------------A--------F1----
36
Accuracy of combined forecasts(across 8 situations)
Game theorists 31 38 10
Structured analogies (SA) 46 63 31
Simulated interaction (SI) 62 88 68
SA & SI [63 + 88]/2 =75.5 - 88 51
Percent correctIndividual Combined
Approx. %Error
Reduction
37
Conclusions
1. Unaided experts’ judgments little better than those of college students or guessing.
2. Forecasts by game theorists no better than unaided experts’ judgments.
3. Structured analogies method provides substantial improvements in accuracy.
4. Forecasts from simulated interaction were most accurate and the method most flexible.
5. Combining further improves accuracy.
Possible applications of SA and SI for business and government
Forecast responses to alternative strategies in conflict situations:
• Labour-management disputes
• Competitor, supplier, and distributor behavior
• Customer reactions to major changes in product, price or service (e.g., “New Coke”)
• Behavior in response to new laws or regulations
• Diplomatic and national security problems
38
39
Summary• Do not rely on experts’ unaided
judgmental forecasts• Do require forecasts:
– That are scientific (i.e. from evidence-based methods – see ForPrin.com)
– For alternative policies or strategies– Of all effects– Of all costs and benefits
References
Armstrong, J. S. (2001). Principles of Forecasting: A handbook for researchers and practitioners. Norwell, MA: Kluwer.Graefe, A., Armstrong, J. S., Jones, R. J. & Cuzan, A. (2010). Combining forecasts: An application to U.S. Presidential Elections.
Working paper. Available at: http://dl.dropbox.com/u/3662406/Articles/Graefe_et_al_Combining.pdf Green, K. C. (2005). Game theory, simulated interaction, and unaided judgement for forecasting decisions in conflicts: Further
evidence. International Journal of Forecasting, 21, 463-472. Available at http://www.kestencgreen.com/gt_update_in_IJF21.pdf
Green, K. C. (2002). Forecasting decisions in conflict situations: a comparison of game theory, role-playing, and unaided judgement. International Journal of Forecasting, 18, 321-344. http://www.forecastingprinciples.com/paperpdf/Greenforecastinginconflict.pdf
Green, K. C. & Armstrong, J. S. (2007). The value of expertise for forecasting decisions in conflicts. Interfaces, 37, 287-299. Available at http://kestencgreen.com/green&armstrong2007-expertise.pdf
Green, K. C. & Armstrong J. S. (2007). Structured Analogies in Forecasting, International Journal of Forecasting, 23, 365-376. Available at http://www.forecastingprinciples.com/files/pdf/INTFOR3581_Publication15.pdf
Green, K. C. & Armstrong, J. S. (2011). Role thinking: Standing in other people’s shoes to forecast decisions in conflicts. International Journal of Forecasting, 27, 69-80. Available at http://kestencgreen.com/group_shoes-2009.pdf
Green, K. C., Graefe, A. & Armstrong, J. S. (2010). Forecasting principles. In Lovric, M. (ed.), International Encyclopedia of Statistical Science. Springer [In press].
Tetlock, P. E. (2005). Expert political judgment: How good is it? How can we know? Princeton, NJ: Princeton.
40