overlapping expert information: learning about...

Overlapping Expert Information: Learning about Dependencies in

Expert Judgment

Jason R. W. Merrick

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the

National Science Foundation.

1

Extending Winkler’s Multivariate Normal

Aggregation

2

P(Event | X1,…,Xn)

P(Event)

3

P(Event | X1,…,Xn)

Event = Collision between two vesselsX1 = Type of other vesselX2 = Proximity of other vesselX3 = Wind speedX4 = Wind directionX5 = Current speedX6 = Current directionX7 = Visibility

4

P(Event | X1,…,Xn)

Event = Incoming vessel contains RDDX1 = Last country dockedX2 = 2nd to last country dockedX3 = 3rd to last country dockedX4 = Frequency of US callsX5 = Vessel ownershipX6 = Type of vesselX7 = Type of crew

5

Issaquah class ferry On the Bremerton to Seattle route

Crossing situation within 15 minutesOther vessel is a navy vessel

No other vessels aroundGood visibility

Negligible wind

What is the probability of a collision?

6

Issaquah class ferry On the Bremerton to Seattle route

Crossing situation within 15 minutesOther vessel is a navy vessel

No other vessels aroundGood visibility

Negligible wind

Issaquah class ferry On the Bremerton to Seattle route

Crossing situation within 15 minutesOther vessel is a product tanker

No other vessels aroundGood visibility

Negligible wind

7

Issaquah Ferry Class -SEA-BRE(A) Ferry Route -

Navy 1st Interacting Vessel Product TankerCrossing Traffic Scenario 1st Vessel -

< 1 mile Traffic Proximity 1st Vessel -No Vessel 2nd Interacting Vessel -No Vessel Traffic Scenario 2nd Vessel -No Vessel Traffic Proximity 2nd Vessel -> 0.5 Miles Visibility -Along Ferry Wind Direction -

0 Wind Speed -Likelihood of Collision -

9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 99 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 99 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9

8

P(Event | X , p0 ,β ) = p0 exp X Tβ( )

P(Event | R,β )P(Event | L,β )

yi, j = ln(zi, j ) = XiTβ + ui, j

=

p0 exp(RTβ )p0 exp(LTβ )

= exp (R − L)T β( )

9

9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9Likelihood of Collision

yi, j = ln(zi, j ) = XiTβ + ui, j

10

u =u1up

⎛

⎝

⎜⎜⎜

⎞

⎠

⎟⎟⎟~ MVNormal 0,Σ( )

θ

ui = µi −θ

11

π (θ;µ,Σ)∝ exp − θ − µ*( )2 / 2σ *2( )

µ* = 1TΣ−1µ1TΣ−11

σ *2 = 11TΣ−11

12

u =u1up

⎛

⎝

⎜⎜⎜

⎞

⎠

⎟⎟⎟~ MVNormal 0,Σ( )

θ

ui = µi −θ

XiTβ

ui, j = yi, j − XiTβ

ui =ui,1ui,p

⎛

⎝

⎜⎜⎜

⎞

⎠

⎟⎟⎟~ MVNormal 0,Σ( )

13

y1,1 y1,p yN ,1 yN ,p

⎛

⎝

⎜⎜⎜

⎞

⎠

⎟⎟⎟=

x1,1 x1,q xN ,1 xN ,q

⎛

⎝

⎜⎜⎜

⎞

⎠

⎟⎟⎟

β1 β1 βq βq

⎛

⎝

⎜⎜⎜

⎞

⎠

⎟⎟⎟+

u1,1 u1,p uN ,1 uN ,p

⎛

⎝

⎜⎜⎜

⎞

⎠

⎟⎟⎟

Y = Xβ1T +U

14

p Y |X,β,Σ( )∝ Σ −N

2 exp − 12 tr VΣ

−1( ){ } exp − 12 tr

B− β1T( )T XTX

B− β1T( )Σ−1( ){ }

Σ*

β =X T X( )−1

1TΣ−11 µ*

β = B̂Σ−111TΣ−11

15

Σ( ) ~ Inv −Wishart G,m( )

β |Y,X,Σ( ) ~ MVNormal ϕ, A1TΣ−11

⎛⎝⎜

⎞⎠⎟

Σ |Y,X( ) ~ Inv −Wishart G +V,m + N( )

β |Y,X,Σ( ) ~ MVNormal A−1 +XTX( )−1 XTX B̂Σ−11

1TΣ−11+A−1ϕ

⎛⎝⎜

⎞⎠⎟,A−1 +XTX( )−11TΣ−11

⎛

⎝⎜⎜

⎞

⎠⎟⎟

V = Y −XB̂( )T Y −XB̂( )

16

Description Notation Values

Ferry route and class FR_FC 26

Type of 1st interacting vessel TT_1 13

Scenario of 1st interacting vessel TS_1 4

Proximity of 1st interacting vessel TP_1 Binary

Type of 2nd interacting vessel TT_2 5

Scenario of 2nd interacting vessel TS_2 4

Proximity of 2nd interacting vessel TP_2 Binary

Visibility VIS Binary

Wind direction WD Binary

Wind speed WS Continuous

17

Assume independence between the experts a priori

18

Comparing the two

scenarios we pictured earlier a priori

19

Analysis with dependenceDoesn’t dependence

between experts increase posterior variance?

Analysis with independence

20

Experts 1, 3 and 7 are correlatedExperts 2, 4 and 6 are correlatedExperts 5 and 8 are negatively or uncorrelated with other experts

Remember we assumed independence a priori, but we learnt

about Σ!

21

Comparing the two

scenarios we pictured earlier Text

Prior [1.88*10-35, 5.32*1034]Dependent [4.38,5.84] ½ width = 0.73Independent [4.43,7.04] ½ width = 1.3

Text

90% Credibility

Interval

22

Getting the Right Mix of Experts

23

z1,..., zp( )

r1,...,rp( )

24

25

26

27

28

29

30

31

32

33

34

35

36

37