Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 1

University of WashingtonDepartment of Electrical Engineering

EE512 Spring, 2006 Graphical Models

Jeff A. Bilmes <bilmes@ee.washington.edu>Jeff A. Bilmes <bilmes@ee.washington.edu>

Lecture 2 Slides

March 30th, 2006

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 2

• d-separation, 3 canonical BNs, Bayes ball• Undirected Models• Start computing probabilities

Outline of Today’s Lecture

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 3

Books and Sources for Today

• Jordan: Chapters 1 and 2• Derin 1989 (Markov Random Fields)• Lauritzen, 1986.• Any graph theory text.

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 4

• L1: Tues, 3/28: Overview, GMs, Intro BNs.• L2: Thur, 3/30: semantics of BNs + UGMs• L3: Tues, 4/4• L4: Thur, 4/6• L5: Tue, 4/11• L6: Thur, 4/13• L7: Tues, 4/18• L8: Thur, 4/20• L9: Tue, 4/25• L10: Thur, 4/27

• L11: Tues, 5/2• L12: Thur, 5/4• L13: Tues, 5/9• L14: Thur, 5/11• L15: Tue, 5/16• L16: Thur, 5/18• L17: Tues, 5/23• L18: Thur, 5/25• L19: Tue, 5/30• L20: Thur, 6/1: final presentations

Class Road Map

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 5

Makeup Time Constraints

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 6

• READING: Chapter 1,2 in Jordan’s book (pick up book from basement of communications copy center).

• Syllabus handout• If you have not signed up before:

– List handout: name, department, and email– List handout: regular makeup slot, and discussion section

• Reminder: course web page: http://ssli.ee.washington.edu/ee512

• TA discussions and office hours:– Office hours: Thursdays 3:30-4:30, Sieg Ground Floor

Tutorial Center– Discussion Sections: Fridays 9:30-10:30, Sieg Ground Floor

Tutorial Center Lecture Room

Announcements

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 7

Bayesian Networks & CI

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 8

Bayesian Networks & CI

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 9

Bayesian Networks & CI

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 10

Pictorial d-separation: blocked/unblocked paths

Blocked Paths Unblocked Paths

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 11

• Three 3-node examples of BNs and their conditional independence statements.

V1

V2 V3

2 3 1|V V V

V1

V2 V3

2 3V V2 3V V

2 3 1|V V V

V1

V2 V3

2 3 1|V V V

2 3V V

Three Canonical Cases

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 12

Case 1

• Markov Chain

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 13

Case 2

• Still a Markov Chain

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 14

Case 3

• NOT a Markov Chain

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 15

SUVsGreenhouse

GassesGlobal

Warming

LungCancer

SmokingBad

Breath

Genetics Cancer Smoking

Examples of the three cases

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 16

Bayes Ball

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 17

Ex: What are some conditional independences?

D. PooleM. Jordan

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 18

Two Views of a Family

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 19

Undirected Models

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 20

Markov Random Fields

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 21

Happy Families

MRFs

BNs

Decomposable models

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 22

Can BNs represent C4?

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 23

Markov Random Fields->Graph Theory

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 24

Examples

•Subgraph

•Cycle

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 25

Markov Random Fields

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 26

Markov Random Fields

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 27

Markov Random Fields: interaction

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 28

Markov Random Fields: Image Processing

Derin, 1989

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 29

Summarizing

Derin, 1989

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 30

Computing Probabilities

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 31

Computing Probabilities

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 32

Computing Probabilities

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 33

Note on sums and evidence

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 34

Graph Algorithm Equivalent: Elimination

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 35

B

A

C

D

E F G

H I1 2

3 4

7

5

6

8

9

Order the Nodes

Eliminate the nodesin order

Elimination Example: different graph

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 36

Result of Elimination

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 37

Elimination

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 38

Elimination & Directed Graphs

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 39

Immoral Elimination

Lec 2: March 30th, 2006 EE512 - Graphical Models - J. Bilmes Page 40

Variable Elimination Algorithm