aait-bn-jt

8/14/2019 aait-bn-jt

1/13

Graphical Models- Inference -

Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel

Albert-Ludwigs University Freiburg, Germany

PCWP CO

HRBP

HREKG HRSAT

ERRCAUTERHRHISTORY

CATECHOL

SAO2 EXPCO2

ARTCO2

VENTALV

VENTLUNG VENITUBE

DISCONNECT

MINVOLSET

VENTMACHKINKEDTUBEINTUBATIONPULMEMBOLUS

P AP S HU NT

ANAPHYLAXIS

MINOVL

PVSAT

FIO2

PRESS

INSUFFANESTHTPR

LVFAILURE

ERRBLOWOUTPUTSTROEVOLUMELVEDVOLUME

HYPOVOLEMIA

CVP

BP

Mainly based on F. V. Jensen, Bayesian Networks and Decision Graphs, Springer-Verlag New York, 2001.

AdvancedI WS 06/07

Junction Tree

BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07 Outline

Introduction

Reminder: Probability theory Basics of Bayesian Networks

Modeling Bayesian networks

Inference (VE, Junction tree)

Excourse: Markov Networks Learning Bayesian networks

Relational Models

BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07 Why Junction trees ?

Multiple inference, e.g.,

For each i do variable elimination?

No, instead reuse informationresp. computations

-Inference(JunctionTree)


BayesianNetworks

BayesianNetworks

AdvancedI WS 06/07

A potentialAover a set of variables A is a

function that maps each configuration into anon-negative real number.

domA=A (domain of A)

Potentials

Examples:Conditional probabilitydistribution and joint probability distributionsare special cases of potentials




2/13

BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07

Ex: A potential A,B,Cover the set of

variables {A,B,C}. Ahas four states,

andBand C has three states.

domA,B,C={A,B,C}

Potentials



BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07

Let be a set of potentials (e.g. CPDs), and let X

be a variable. X is eliminated from :

1. Remove all potentials in with X in thedomain. Let X that set.

2. Calculate

3. Add to

4. Iterate

VE using Potentials

Potential where X is not amember of the domain



BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07 Domain Graph

A1

A3

A4

A2

A5 A6

A1

A3

A4

A2

A5 A6

Bayesian Network

Moralizing

Domain Graph(Moral Graph)

moral link

Let ={1,...,n} be potentials over U={1,...,m} withdomi=Di. The domain graph for is the undirected graphwith variables of U as nodes and with a link between pairs ofvariables being members of the same Di.



BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07 Moralizing

A1

A3

A4

A2

A5 A6

A1

A3

A4

A2

A5 A6

Bayesian Network

Moralizing

Domain Graph

(Moral Graph)




3/13


4/13


5/13


6/13


7/13

BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07 Complexity of Elimination Sequence

A1

A3

A4

A2

A5 A6

A1

A3

A4

A2

A5

A1

A4

A2A4

A2

A4

A6 A5

A2

A1

A3

A4

A2

A3 A1 -Inference(JunctionTree)

-I

nference(JunctionTree)

{A3,A6} {A2,A3,A5} {A1,A2,A3}

{A1,A2} {A2,A4}BayesianNetworks

BayesianNetworks

AdvancedI WS 06/07

All perfect elimination sequences produce the samedomain set, namely the set of cliques of the domain

Any perfect elimination sequenceending

with A is optimalwith respect to computing P(A)

Complexity of Elimination Sequence


-I

nference(JunctionTree)

BayesianNetwork

s

BayesianNetworks

AdvancedI WS 06/07 Triangulated Graphs and Join Trees

An undirected graph with complete

elimination sequenceis called atriangulatedgraph

A1

A3

A4

A2

A5

A1

A3

A4

A2

A5

triangulated nontriangulated



BayesianNetwork

s

BayesianNetworks

Advanced

I WS 06/07 Triangulated Graphs and Join Trees

A1

A3

A4

A2

A5

Neighbours of A5

A1

A3

A4

A2

A5

Family of A5

Complete neighbour set = simplicial node

X is simplicial iff family of X is a clique

A1

A3

A4

A2

A5

Family of A3, i.e. A3 is simplicial




8/13

BayesianNetworks

BayesianNetworks

AdvancedI WS 06/07

Let Gbe a triangulatedgraph,

and letX be a simplicial node.

Let Gbe the graph resulting

from eliminating Xfrom G.

Then Gis a triangulated graph

A1

A3

A4

A2

A5

X

Elimi

natin

gX

A1

A3

A4

A2

A5

Triangulated Graphs and Join Trees



BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07 Triangulated Graphs and Join Trees

A triangulated graph with at least two

nodes has at least two simplicial nodes In a triangulated graph, each variable A

has a perfect elimination sequence endingwith A

Not all domain graphs are triangulated: An

undirected graph is triangulated iff allnodes can be eliminated by successivelyeliminating a simplicial node



BayesianNetwork

s

BayesianNetwork

s

AdvancedI WS 06/07 Join Trees

Let G be the set of cliques from

an undirected graph, and let

the cliques of G be organized ina tree.

T is ajoin treeif for anypair of nodes V, W allnodes on the pathbetween V an W contain

the intersection VW

BCDE

ABCD DEFI

BCDG

CHGJ

BCDE

ABCD DEFI

BCDG

CHGJ



BayesianNetwork

s

BayesianNetwork

s

AdvancedI WS 06/07 Join Trees

Let G be the set of cliques from

an undirected graph, and let

the cliques of G be organized ina tree.

T is ajoin treeif for anypair of nodes V, W allnodes on the pathbetween V an W contain

the intersection VW

BCDE

ABCD DEFI

BCDG

CHGJ

BCDE

ABCD DEFI

BCDG

CHGJ

join tree

not a join tree




9/13

BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07 Join Trees

If the cliques of an undirected graph G can

be organized into a join tree, then G istriangulated

If the undirected graph is triangulated, thenthe cliques of G can be organizes into a join

tree -Inference(JunctionTree)


BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07

Triangulated, undirected Graph-> Join trees

A E

F

I

D

J

GH

B

C

Simplicial node X

Family of X is a clique Eliminate nodes from

family of X which haveonly neighbours in thefamily of X

Give family of X is number

according to the numberof nodes eliminated

Denote the set ofremaining nodes Si



BayesianNetwork

s

BayesianNetwork

s

AdvancedI WS 06/07


A E

F

I

D

J

GH

B

C

X

ABCDV1

BCDS1 separator

Simplicial node X

Family of X is a clique

Eliminate nodes fromfamily of X which haveonly neighbours in thefamily of X

Give family of X a numberi according to the number

of nodes eliminated




BayesianNetwork

s

BayesianNetwork

s

AdvancedI WS 06/07

Simplicial node X



Give family of X a numberi according to the number

of nodes eliminated



A E

F

I

D

J

GH

B

C

X

ABCDV1

BCDS1 separator

A

{A,B,C,D}

{B,C,D}

1

S1




10/13

BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07


E

F

I

D

J

GH

B

C

ABCD

V1

BCD

S1

X

DEFIV3

DES3

X

Simplicial node X








BayesianNetworks

BayesianNetworks

AdvancedI WS 06/07

Simplicial node X







E

F

I

D

J

GH

B

C

ABCD

V1

BCD

S1

X

DEFIV3

DES3

X

{D,E,F,I}

E

{D,E}

3

S3



BayesianNetwork

s

BayesianNetwork

s

Advanced

I WS 06/07


ABCDV1

BCDS1

DEFIV3

DES3

CGHJV5

CGS5

BCDGV6

BCDS6

BCDEV10

Simplicial node X



Give family of X is numberaccording to the number

of nodes eliminated




BayesianNetwork

s

BayesianNetwork

s

Advanced

I WS 06/07


Connect eachseparator Sito a clique

Vj, j>i, such that Siis asubset of Vj

ABCDV1

BCDS1

DEFIV3

DES3

CGHJV5

CGS5

BCDGV6

BCDS6

BCDEV10




11/13

BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07 Join Tree

ABCD

V1

BCD

S1

DEFIV3

DES3

CGHJ

V5

CG

S5

BCDGV6

BCDS6

BCDEV10



The potential representation

of a join tree (aka cliquetree) is the product of the

clique potentials, dived by

the product of the separator

potentials.

BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07 Yet Another Example

A1

A3

A4

A2

A5 A6

A1

A3

A4

A2

A5 A6

A3,A6

V1

A3S1

A2,A3,A5

V2

A2,A3

S2

A2,A4V6

A2

S4

A1,A2,A3

V4



BayesianNetwork

s

BayesianNetwork

s

Advanced

I WS 06/07 Junction Trees

Let be a set ofpotentials with a

triangulated domaingraph. A junction tree foris join tree for G with

Each potential in isassociated to a cliquecontaining dom()

Each link has separatorattached containing twomailboxes, one for eachdirection

A1

A3

A4

A2

A5 A6



BayesianNetwork

s

BayesianNetwork

s

Advanced

I WS 06/07 Propagation on a Junction Tree

Node V can send exactly one message to aneighbour W, and it may only be sent when V

has received a message from all of its otherneighbours

Choose one clique (arbitrarily) as a root of thetree; collect message to this node and thendistribute messages away from it.

After collection and distribution phases, we

have all we need in each clique to computepotential for variables.




12/13

BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07 Junction Trees - Collect Evidence



P(A4) ?

Find clique containing A4

V6 temporary root

Send messages from

leaves to root

V4 assembles theincoming messages,potential

BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07

Propagation/message passing between twoadjacent cliques C1, C2(S0is their seperator)

Marginalize C1s potential to get new potential for S0

Update C2s potential

Update S0s potential to its new potential

Junction Tree - Messages

=01

1

\

*

SC

CSo

0

0

22

*

*

S

S

CC

=

BayesianNetwork

s

BayesianNetwork

s

Advanced




P(A4) ?

Find clique containing A4

V6 temporary root

Send messages from

leaves to root

V4 assembles theincoming messages,potential

BayesianNetwork

s

BayesianNetwork

s

Advanced




P(A4) ?

All marginals?


13/13

BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07 Junction Trees - Distribute Evidence



All marginals?

BayesianNetworks

BayesianNetworks

Advanced

I WS 06/07 Nontriangulated Domain Graphs

Embed domain graph in a traingulated graph

Use ist junction tree

Simple idea:

Eliminate variables in some order

If you wish to eliminate a node with non-completeneighbour set, make it complete by adding fill-ins

A

E

F

I

D

J

G

H

B C A

E

F

I

D

J

G

H

B C

A,C,H,I,J,B,G,D,E,F-Inference(JunctionTree)


BayesianNetwork

s

BayesianNetwork

s

Advanced

I WS 06/07 Summary JTA

Convert Bayesian network

into JT

Initialize potentials and

separators

Incorporate Evidence (set

potentials accordingly)

Collect and distribute

evidence

Obtain clique marginals by

marginalization/normalization



Image from

Cecil HuangBayesianNetwork

s

BayesianNetwork

s

Advanced

I WS 06/07 Inference Engines

(Commercial) HUGIN : http://www.hugin.com

(Commercial) NETICA: http://www.norsys.com

Bayesian Network Toolbox for Matlabwww.cs.ubc.ca/~murphyk/Software/BNT/bnt.html

GENIE/SMILE (JAVA) http://www2.sis.pitt.edu/~genie/

MSBNx, Microsoft, http://research.microsoft.com/adapt/MSBNx/

Profile Toolbox http://www.informatik.uni-freiburg.de/~kersting/profile/



aait-bn-jt

Documents