1

Chap. 4 Decision Graphs

Statistical Genetics Forum

Bayesian Networks and Decision GraphsFinn V. Jensen

Presented byKen Chen

Genome Sequencing Center

2

Flu Fever Sleepy

T

A

Using probabilities provided by network to support decision-making•Test decisions

Look for more evidences•Action decisions

3

OH0 OH1

FC SC

OH2

BHMH

OH0 OH1

FC SC

OH2

BHMH

D U

One Action:Example:

Call Fold

€

EU (call) = U (BH,call)BH

∑ P(BH | evidence)

€

EU ( fold) = U (BH , fold)BH

∑ P(BH | evidence)

€

EU (call) > EU ( fold)?

Poker Game:

4

One action in general

D

€

EU (D | e) = U1(X

1)P(X

1| D,e) +

Xi

∑ +L + Un(X

n)P(X

n| D,e)

Xn

∑

Goal: find D=d that maximize EU(D|e)

5

Action

GA DSS

UGA UDSS

4.2 Utilities:Example: Management of effortDecision:•Gd: keep pace in GA, follow DSS superficially•SB: slow down in both courses•Dg: keep pace in DSS, follow GA superficially

€

EU (D = d) = P(m | d)mm∈GA

∑ + P(m | d)mm∈DSS

∑

Game 1: maximize the sum of the exp marks

General: maximize the sum of the exputilities

€

EU (D = d) = P(m | d)UGA

(m)m∈GA

∑ + P(m | d)UDSS

(m)m∈DSS

∑

6

4.3 Value of information

€

V (P(H )) = maxa∈A

U (a,h)P(h)h∈H

∑ ,

V (P(H | t)) = maxa∈A

U (a,h)P(h | t)h∈H

∑ ,

EV (T ) = V (P(H | t))P(t)t∈T

∑

EB(T ) = EV (T )−V (P(H ))

EP(T ) = EB(T )−CT

A

HU

T

7

Nonutility value functions

• When there is no proper model for actions and utilities, the reason for test is to decrease the uncertainty of the hypothesis

€

Entropy (P(H )) = − P(h)h∈H

∑ log2(P(h))

V (P(H )) = −Entropy (P(H ))

€

V (P(H )) = − (h − μh∈H

∑ )2 P(h)

8

Test1

Test2

Action

Action

Action

T2

Inf

99.94

-0.06

97.74

Inf

99.74

-0.26

T1

yes

pos

neg

yes

pour

pos

neg

discard

pour

clean

infectedclean

infected

Nonmyopic data request

9

Decision Tree• Nonleaf nodes are decision nodes or chance nodes, and the leaves are

utility nodes

• Complete: For a chance node there must be a link for each possible state, and from a decision node there must be a link for each possible decision option

D

action1

action2

actionn

…X

P(X=x1|o)

P(X=x2|o)

P(X=xn|o)

…U

10

A car start problem

• Possible Fault: – Spark Plug (SP), prob=0.3 – Ignition System (IS), prob=0.2, – Others, prob=0.5

• Actions:– SP, fixes SP, 4 min– IS, fixes IS with prob=0.5, 2 min– T, test OK iff. IS is OK, 0.5 min– RS, fixes everything, 15 min

• Goal:– Have car fixed asap

11

15

D

D

D

D

D

10.514.5 25.5

D

D

D

D

D

D

27.5

12.514.5

D

SP

28

26

OK

!OK

RSSP

OK!OK

0.38

0.62

0.8

0.2 IS !OK OK

RS

0.5

0.5RS

OK

!OK

0.7

0.3

RS

OK

!OK

0.1

0.9

P(SP fix|T=OK)=P(SP|T=OK)=P(SP| !IS )=P(SP)/(P(SP)+P(others))=0.3/0.8=0.38

P(IS fix)=P(IS)P(fix|IS)=0.2*0.5=0.1

Fault

T Fault-I

IS

…

…

…

…

RS

T

IS

12

15

D

D

D

D

D

10.514.5 25.5

D

D

D

D

D

D

27.5

12.514.5

D

RS

T

SP

IS

28

26

OK

!OK

RSSP

OK!OK

0.38

0.62

0.8

0.2 IS !OK OK

RS

0.5

0.5RS

OK

!OK

0.7

0.3

RS

OK

!OK

0.1

0.9

16.96

16.27

15.43 €

E[U (X )] = p(xi)U (x

i)

i

∑

€

E[U (D)] = maxiU (d

i)

…

…

…

…

Solving Decision Trees

13

Coalesced decision trees

• Grow exponentially with the number of decisions and chance variables

• When decision tree contains identical subtrees they can be collapsed.

14

4.5 Decision-Theoretic Troubleshooting

• A fault causing a device to malfunction is identified and eliminated through a sequence of troubleshooting steps.

• A troubleshooting problem can be represented and solved through a decision tree (actions and questions)

• As decision trees have a risk of becoming intractably large, we look for ways of pruning the decision tree.

15

Action sequences• Ai=yes, Ai=no

• Cost of action Ai, Ci(), evidences

• Action seq: s=<A1,…,An> repeatly performing the next action until problem gets fixed or the last action has been performed

• Expected cost of repair (ECR)

€

ECR(s) ≡ ECRi(s)

i

∑ ,

ECRi(s) = C

i(ε i−1)P(ε i−1)

16

Local optimality of the optimal sequence (Dynamic Programming)

Consider two neighboring actions Ai and Ai+1

€

s = (...,Ai,A

i+1,...), ′ s = (...,A

i+1,A

i,...)

ECR(s) ≤ ECR( ′ s )

Ci(ε i−1)P(ε i−1) +C

i+1(ε i )P(A

i= no,ε i−1) ≤

Ci+1

(ε i−1)P(ε i−1) +Ci(ε i−1,A

i+1= no)P(A

i+1= no,ε i−1)

Ci(ε i−1) +C

i+1(ε i )P(A

i= no |ε i−1) ≤ C

i+1(ε i−1) +C

i(ε i−1,A

i+1= no)P(A

i+1= no |ε i−1)

Assuming costs are independent of action

P(Ai= yes |ε i−1)

Ci

≥P(A

i+1= yes |ε i−1)

Ci+1

ef (Ai|ε i−1) ≥ ef (A

i+1|ε i−1) Proposition 4.1

Pruned tree has eightnon-RS links, comparedto 32 in a coalesced DTfor the same problem

17

The greedy approach• Always choose the action with the highest efficiency

Not necessarily optimal!• Proposition 4.2: Conditions under which the greedy approach is optimal:

– n faults F1…Fn, and n actions: A1 … An

– Exactly one of the faults is present– Each action has a specific probability of repair: pi=P(Ai=yes|Fi), P(Ai=yes|

Fj)=0 if i≠j– The cost Ci of an action does not depend on the performance of previous

actions

• Theorem 4.2: for action sequence s fulfilling the conditions in Proposition 4.2. Assume s is ordered according to decreasing initial efficiencies. Then s is an optimal action sequence and

F1

F2

F3

F4

A1

A2

A3

€

ECR(s) = Ci(1− p

j)

j=1

i−1

∑i=1

n

∑

18

Influence Diagram

• A compact representation of decision tree• Now seen more as a decision tool extending Bayesian networks• Syntax:

– There is a directed path comprising all decision nodes

– The utility nodes have no children

– The decision nodes and the chance nodes have a finite set of mutually exclusive states

– The utility nodes have no states

– To each chance node A is attached a conditional probability table P(A|pa(A))

– The each utility node U is attached a real-valued function over pa(U)

19

OH0 OH1

OFC OSC

OH

BHMH

D U

MH0 MH1

MFC MSC

BN

Influence Diagram

OH0 OH1

OFC OSC

OH

BHMH

D U

MH0 MH1

MFC MSC

No-forgetting:The decision makerremembers the pastobservations and decisions

20

Solution to influence diagrams

• Similar to decision-tree

• More efficiently by exploiting the structure of of the influence diagram (Chapter 7)

21

Information blockingV1

T1FV1

U1

V2

T2FV2

U2

V3

T3FV3

U3

V4

T4FV4

U4

V5

T5FV5

U5

FV5 has109 elements

V1

T1FV1

U1

V2

T2FV2

U2

V3

T3FV3

U3

V4

T4FV4

U4

V5

T5FV5

U5

Introduce variables/links which,when observed,d-separate most of the past from The present decision

Fishing Vol