stakeholder preference modeling with probabilistic inversion
DESCRIPTION
Stakeholder Preference Modeling with Probabilistic Inversion. Roger M. Cooke Resources for the Future Dept Math, TU Delft June 16, 2011. Foundations Health states Risks nano -enabled food. Expert Judgment for Uncertainty Quantification: PM 2.5. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/1.jpg)
Stakeholder Preference Modeling with
Probabilistic Inversion
Roger M. CookeResources for the Future
Dept Math, TU DelftJune 16, 2011
![Page 2: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/2.jpg)
• Foundations• Health states• Risks nano-enabled food
![Page 3: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/3.jpg)
Expert Judgment for Uncertainty Quantification: PM2.5
Uncertainty in Mortality Response to Airborne Fine Particulate Matter: Combining European Air Pollution Experts Jouni T. Tuomisto, Andrew Wilson, John S. Evans, Marko Tainio (RESS 2008)
![Page 4: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/4.jpg)
Fundamental Theorem of Decision TheoryFor Rational PreferenceUNIQUE probability P which represents degree of belief:
DegBel(France wins worldcup) > DegBel(Belgium wins worldcup) P(F) > P(B)
AND a Utility function, unique up to 0 and 1, that represents values:
($1000 if F, $0 else) > ($1000 if B, $0 else) Exp’d Utility (($1000 if F, $0 else)) > Exp’d Utility (($9000 if B, $0 else))
BUT….
![Page 5: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/5.jpg)
UNLIKE Expert Judgment: There is no
• Updating utilities on observations
• Convergence of utilities via Observations
• Empirical control on Utilities
• Community of ‘Utility Experts’
• Rational consensus on Utilities
![Page 6: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/6.jpg)
Why is Preference Modeling Impoverished?
AHPMAUTMCDMELECTRAREMBRANTOUTRANKINGTHURSTONE
BRADLEY TERRYPROBITLOGITNESTED LOGITPSYCH’L SCALING
Validation???
![Page 7: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/7.jpg)
What means Validation?
Goal = find ‘true Utility values’ for alternatives?Fools’s Errand
![Page 8: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/8.jpg)
Condorcet’s Paradox of Majority Preference
1/3 prefer Mozart > Hayden > Bach 1/3 prefer Hayden > Bach > Mozart1/3 prefer Bach > Mozart > Hayden
THEN 2/3’s prefer
Bach > MozartMozart > HaydenHayden > Bach
![Page 9: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/9.jpg)
What can we do?
Random Utility Theory
Each (rational) stakeholder has a utility function over alternatives characterize population as distribution over utility functions
![Page 10: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/10.jpg)
Probabilistic Inversion
Domain: utility functionsOf stakeholders
Range: choices of stakeholders
G maps utilities into choices
Invert G at this distribution
Observe Stakeholders Preferences
![Page 11: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/11.jpg)
Used for stakeholder Preference Modeling:
• Risks of Nano enabled foods (Flari, WHO, CIS)• Valuing impaired health states (Flari, FDA)• Valuing fossil fuel policy options (RFF)• Prioritizing ecosystem threats (NCEAS)• Prioritizing zoonose threats (RIVM)• Modeling wiring failure (Mazzuchi)• Prioritizing vCVJ options (Aspinall Health Canada)• UK Research Council (Aspinall)• Aus. Univ. Fac Sci reviews (Aspinall).
![Page 12: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/12.jpg)
steps
1. Get discrete choice data from stakeholders for choice alternatives A1,…An– “Which of (A,B) do you prefer”– “Rank your top 3 of (A, B, C, D, E, F)”
2. Find dist’n over utilities on [0,1]n which reproduces stakeholders preferences
3. If utility is function of covariates, validate out of sample.
![Page 13: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/13.jpg)
Valuation of impaired Health statesFlari et al
17 health states6 criteriaEach criteria has 3 values, described in narrative19 Experts ranked 5 groups of 5 health states
![Page 14: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/14.jpg)
First, find dist’n over utilities for the 17 Health States which recover Observed Frequencies of rankings (i.e. wo criteria)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
f(x) = 0.912874736742624 x + 0.0143452928596942R² = 0.977197926615134
Predicted vs Observed Frequencies of 105 rankings, scale values for the 17 Health States, no MAUT model
RecoveredLinear (Recovered)
Observed
Reco
vere
d
![Page 15: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/15.jpg)
Build MAUT model for HS utilities, based on the 6 criteria
• Each stakeholder has a weight vector that determines his/her preference
• Population of stakeholders = population of weights
• Characterize population based on all rankings involving at least 7 (30%) experts (= 28 rankings).
• Validate on remaining rankings (= 77 rankings)
![Page 16: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/16.jpg)
Average weights
12%
29%
8%27%
10%
15%
Mean Weights
MobilitySelf CareUsual ActivitiesPain DiscomfortAnxiety DepressionCognitive Functioning
![Page 17: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/17.jpg)
Preference dependence emerges from fitting
![Page 18: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/18.jpg)
Average weights per group of 5 health states
![Page 19: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/19.jpg)
Predict out-of-sample rankings
0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.800
0.1
0.2
0.3
0.4
0.5
0.6
0.7
f(x) = 0.879511080111118 x + 0.0210766294292423R² = 0.5721455934275
Predicted vs Observed % rankings, 28 fitted, 77 Out-of-Sample
PredictedLinear (Predicted)
Observed Percentage Ranks
pred
icted
per
cent
ages
Ran
ks
First time in HISTORY that a multi attribute model has been WRONG!!!
![Page 20: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/20.jpg)
Average of predictions vs Out-of-Sample observed rankings, Not SOOO bad
0.00 0.05 0.10 0.15 0.20 0.25 0.30-0.05
0.00
0.05
0.10
0.15
0.20
0.25
0.30
Out-of-sample:average predicted versus observed
observed percentages for out-of-sample rankings (less than 30% of experts)
pred
icte
d pe
rcen
tage
s fo
r ou
t-of
-sam
-pl
e ra
nkin
gs
![Page 21: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/21.jpg)
Risks of Nano-Enabled Foods
Flari et al
![Page 22: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/22.jpg)
Nano enabled food risks (VLARI) rank top and bottom 5
![Page 23: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/23.jpg)
![Page 24: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/24.jpg)
![Page 25: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/25.jpg)
Correlation of criteria weights
![Page 26: Stakeholder Preference Modeling with Probabilistic Inversion](https://reader035.vdocuments.site/reader035/viewer/2022062814/568167ee550346895ddd5cae/html5/thumbnails/26.jpg)
Conclusion
• Stakeholder preference modeling is empirical science
• ‘preference for criteria’ inferred from data, not elicited
• (in) dependence in choices inferred from data, not assumed
THANK YOU