noisy or gate
DESCRIPTION
Javier Turek and Eyal Regev. Noisy Or Gate. Polytrees. …. U i. …. Like a tree, but with multiple parents. U 1. U n. e +. X. Y. Z. e -. Several causes sharing a common effect. Link matrix T x| u. T x| u contains the conditional probabilities: P(X=x | U 1 =u 1 ,…,U n =u n ) - PowerPoint PPT PresentationTRANSCRIPT
NOISY OR GATEJavier Turek and Eyal Regev
Several causes sharing a common effect.
Y Z
Ui
e+
e-
X
PolytreesU1 Un
… …
Like a tree, but with multiple parents.
Link matrix Tx|u
Tx|u contains the conditional probabilities:P(X=x | U1=u1,…,Un=un)
1. The table is huge (contains 2n entries).
2. Who can know such information anyway?You cannot expect to find the P(X | U1,…,Un) table.However, you may know how every Ui influences X
separately.
The OR-gateHaven’t done
homeworkCaught
cheating
Failed an exam More likely!
OR
Inhibitors
Failed an exam
Caught cheating
OR
AND
Paid the TA
Haven’t done
homework
AND
Re-doing the
course
InhibitorsU1
X
U2Un
I1
I2In
OR
AND
AND
AND
Inhibitors are independent
Associate probability to an inhibitor.
Noisy OR-GateU1
X
U2Un
I1
I2In
OR
AND
AND
AND
| , :i k iP x u u k i q
Noisy OR-GateU1
X
U2Un
I1
I2In
OR
AND
AND
AND
:u iT i U True
Tu
if
|1 if +
u
u
ii T
ii T
q xP x u
q x
Message Passing – updating X
Y Z
U
e+
e-
X
| , | |Belief x P x e e P e x P x e D x A x
Y ZD x U x U x |x u Xu
A x T C u
YU x
y
XC u
ZU x
u
Message Passing – updating X
Y Z
U
e+
W
e-
X
| , | | ABelief x P x e e P e x P x e D x x
YU x
y
XC u u
ZU x
XC w
Y ZD x U x U x | ,,
x u Xw
Xwu
T C CuA x w
| , | |Belief x P x e e P e x P x e D x A x
|x u Xu
A x T C u
Message Passing – updating U,Y,Z
Y Z
U
e+
e-
X
Y ZD x U x U x |x u Xu
A x T C u
XU u
YC x
ZC x
|xXx
uTU u D x ZY A xC x x U
| ,x u wX Xx w
T CU D x wu
Y Z
U
e+
W
e-
X
XU u
YC x
ZC x
Message Passing – updating W,U,Y,Z
XU w
Y ZD x U x U x | ,,
x u Xw
Xwu
T C CuA x w
ZY A xC x x U ZY A xC x x U
|x u Xu
A x T C u
|xXx
uTU u D x
Y Z
Ui
e+
e-
X
X iU u
YC x
ZC x
Message Passing – many parents
X nU u
|:k
i u ku k i k i
X Xx
xU D T uCu x
ZY A xC x x U
| X ku k
x uT CA x u Y ZD x U x U x
Link matrix
U1 Un
… …
1XU u
Belief Update – Noisy OR-Gate
if
1 if +
u
u
i X ku i T k
i X ku i T k
D x q C u x
Bel xD x q C u x
u u
i X i X ku i T k T
qC u C u
u u
i X i X ku i T k T
qC u C u
1u u
i X i X ku i T k T
qC u C u
Belief Update – Noisy OR-Gate
1 1 if
1 1 1 if +
i X ii
i X ii
D x q C u x
Bel xD x q C u x
1u u
i X i X ku i T k T
qC u C u
1i X i X ii
qC u C u 1 1 i X i
i
q C u
Update messages
Where
The message to the child is the same:
1 if +
1 if
i i i i i
X ii i i
D x q D x q uU u
D x D x u
1 1i k X kk i
q C u
kY
ii
kXC Ay U x x
Example
Windows Vista
No electricity Virus 2K bug
Does not start
Wrong date
Stolen Password Lost data
D1 D2 D3 D4
M1 M2 M3 M4
Example
D1 D2 D3 D4
M1 M2 M3 M4
0.01\0.99 0.1\0.9 0.2\0.8 0.2\0.8
ijq0.8
0.1 0.9
0.2 0.5
0.1 0.7
0.80.2
|W , forhere j iij kP m d d k iq
\ (1 )i ip p
Conjunction query Conjunction query q: find the belief that
several events happen simultaneously.
Applying the chain rule:
Product of m belief updates
Q
i ii I
q X x
1 2 2 1
1 2 1 1 2 1
1 2 2 1
, , , , ,
| | , ,
| , , , ,
m m m
m m
m m m
P q P x x x x x
P x P x x P x x x
P x x x x x
Answering a query In our example:
Computing P(q):
1 1 2 3 4q d m m m m
1 1 1 2 1 1
3 2 1 1
4 3 2 1 1
| | ,
| , ,
| , , ,
P q P d P m d P m m d
P m m m d
P m m m m d
Example Update the belief on M1
Update likelihoods and priors: 1 11 21 1 2| 1 1 1 1 1 1 0.272P m q qd p
1 1p 2p
1 2MU d
4 2MC d
1 2 11 21 111 ,1 0.92,0.2MU d q q q
4 1 12 2 2 2 2, 1 0.338,0.662M M MC d U d p U d p
D1 D2 D3 D4
M1 M2 M3 M4
Example Update the belief on M2
Update likelihoods and priors:
2 2 121 421 41| | 1 1 1 1 1 0.81,m d dP m P m q q p
1 1p
2 4 12 42 12, 0.45,0.9MU d q q q
4 2 24 4 4 4 4, 1 0.111,0.889M M MC d U d p U d p
2 4MU d
4p
4 4MC d
D1 D2 D3 D4
M1 M2 M3 M4
Example Update the belief on M3
Update likelihoods and priors:
3 32 1 1
13 33 3
1| |
1 1 1 1 1 1 0.836
, ,P m P m
q q p
m m d d
1 1p
3 3 13 33 131 ,1 0.98,0.8MU d q q q
4 3 33 3 3 3 3, 1 0.23,0.77M M MC d U d p U d p
3 3MU d
3p 4 3MC d
D1 D2 D3 D4
M1 M2 M3 M4
Example Update the belief on M4
4
3 2 1 14
42,3,4
|
1 1 0
,
.
,
7
,
81i M ii
m m m dP m
q C d
4 3MC d
4 4MC d
4 2MC d
D1 D2 D3 D4
M1 M2 M3 M4
Example And the final solution is…
Example And the final solution is…
1 1 1 2 1 1
3 2 1 1
4 3 2 1 1
3
| | ,
| , ,
| , , ,
0.01 0.272 0.81 0.836 0.781 1.439 10
P q P d P m d P m m d
P m m m d
P m m m m d
Thank You!