chapter 13€¦ · chapter 13 constraint optimization and counting, and enumeration 275 class ....
TRANSCRIPT
![Page 1: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/1.jpg)
Chapter 13
Constraint Optimization And counting, and enumeration
275 class
![Page 2: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/2.jpg)
Outline n Introduction
n Optimization tasks for graphical models n Solving optimization problems with inference and search
n Inference n Bucket elimination, dynamic programming n Mini-bucket elimination
n Search n Branch and bound and best-first n Lower-bounding heuristics n AND/OR search spaces
n Hybrids of search and inference n Cutset decomposition n Super-bucket scheme
![Page 3: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/3.jpg)
A B red green red yellow green red green yellow yellow green yellow red
Example: map coloring Variables - countries (A,B,C,etc.) Values - colors (e.g., red, green, yellow) Constraints: etc. ,ED D, AB,A ≠≠≠
C
A
B
D E
F G
Task: consistency? Find a solution, all solutions, counting
Constraint Satisfaction
![Page 4: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/4.jpg)
ϕ = {(¬C), (A v B v C), (¬A v B v E), (¬B v C v D)}.
Propositional Satisfiability
![Page 5: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/5.jpg)
Constraint Optimization Problems for Graphical Models
functionscost - },...,{ domains - },...,{ variables- },...,{
:where,, triplea is A
1
1
1
m
n
n
ffFDDDXXX
FDXRCOPfinite
=
=
=
=
A B D Cost 1 2 3 3 1 3 2 2 2 1 3 0 2 3 1 0 3 1 2 5 3 2 1 0
( )∑ ==
m
i i XfXF1
)(
FunctionCost Global
f(A,B,D) has scope {A,B,D}
![Page 6: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/6.jpg)
Constraint Optimization Problems for Graphical Models
functionscost - },...,{ domains - },...,{ variables- },...,{
:where,, triplea is A
1
1
1
m
n
n
ffFDDDXXX
FDXRCOPfinite
=
=
=
=
A B D Cost 1 2 3 3 1 3 2 2 2 1 3 0 2 3 1 0 3 1 2 5 3 2 1 0
G
A
B C
D F
( )∑ ==
m
i i XfXF1
)(
FunctionCost Global
Primal graph = Variables --> nodes Functions, Constraints -! arcs f1(A,B,D) f2(D,F,G) f3(B,C,F)
f(A,B,D) has scope {A,B,D}
F(a,b,c,d,f,g)= f1(a,b,d)+f2(d,f,g)+f3(b,c,f)
![Page 7: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/7.jpg)
Constrained Optimization
Example: power plant scheduling
)X,...,ost(XTotalFuelC minimize :
)(Power : demandpower time,down-min and up-min ,, :sConstraint
. domain ,Variables
N1
4321
1
Objective
DemandXXXXX
{ON,OFF}},...,X{X
i
n
≥∨¬∨
==
∑
![Page 8: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/8.jpg)
8
n A graphical model (X,D,F): n X = {X1,…Xn} variables n D = {D1, … Dn} domains n F = {f1,…,fm} functions
n Operators: n combination n elimination (projection)
n Tasks: n Belief updating: ΣX-y ∏j Pi
n MPE: maxX ∏j Pj
n CSP: ∏X ×j Cj
n Max-CSP: minX Σj fj
Graphical Models
)( : CAFfi +==
A
D
B C
E
F
n All these tasks are NP-‐hard n exploit problem structure n iden=fy special cases n approximate
A C F P(F|A,C) 0 0 0 0.14 0 0 1 0.96 0 1 0 0.40 0 1 1 0.60 1 0 0 0.35 1 0 1 0.65 1 1 0 0.72 1 1 1 0.68
Primal graph (interac=on graph)
A C F red green blue blue red red blue blue green
green red blue
Rela=on
![Page 9: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/9.jpg)
Combination of Cost Functions
A B f(A,B)
b b 6
b g 0
g b 0
g g 6
B C f(B,C)
b b 6
b g 0
g b 0
g g 6 A B C f(A,B,C)
b b b 12
b b g 6
b g b 0
b g g 6
g b b 6
g b g 0
g g b 6
g g g 12
+
= 0 + 6
11
![Page 10: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/10.jpg)
Elimination in a Cost Function
A B f(A,B)
b b 4
b g 6
b r 1
g b 2
g g 6
g r 3
r b 1
r g 1
r r 6
Elim(f,B) A g(A)
b
g
r
1
1
2
Elim(g,A) h∅
1
Min
12
![Page 11: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/11.jpg)
Conditioning a Cost Function
A B f(A,B) b b 6 b g 0 b r 3 g b 0 g g 6 g r 0 r b 0 r g 0 r r 6
Assign(fAB,A,b)
B b 6 g 0 r 3 g(B)
Assign(g,B,r)
0
3 h∅
13
![Page 12: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/12.jpg)
Outline n Introduction
n Optimization tasks for graphical models n Solving by inference and search
n Inference n Bucket elimination, dynamic programming, tree-
clustering, bucket-elimination n Mini-bucket elimination, belief propagation
n Search n Branch and bound and best-first n Lower-bounding heuristics n AND/OR search spaces
n Hybrids of search and inference n Cutset decomposition n Super-bucket scheme
![Page 13: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/13.jpg)
15
Computing the Optimal Cost Solution The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
0=emin
Constraint graph
A
D E
C B B C
E D
Variable Elimination
bcde ,,,min0=
f(a,b)+f(a,c)+f(a,d)+f(b,c)+f(b,d)+f(b,e)+f(c,e) OPT =
f(a,c)+f(c,e) + cmin
),,,( ecdahB
f(a,b)+f(b,c)+f(b,d)+f(b,e) bmin
dmin f(a,d) +
Combination
![Page 14: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/14.jpg)
16
Elimination operator
OPT
bucket B:
f(c,a) f(c,e)
f(a,b) f(b,c) f(b,d) f(b,e)
bucket C:
bucket D:
bucket E:
bucket A:
e=0
(a)hE
e)c,d,(a,hB
e)d,(a,hC
Finding
∑bmin
∑==
r
jjXXXfOPT
n 11
)(min,...,
Algorithm elim-opt (Dechter, 1996) Non-serial Dynamic Programming (Bertele and Briochi, 1973)
B
C
D
E
A
f(a,d)
e)(a,hD
),(),(),(),(),(),(),(min,,,,
ecFebFdbFcbFdaFcaFbaFOPTbcdea
++++++=
![Page 15: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/15.jpg)
17
Generating the Optimal Assignment
C:
E:
f(a,b) f(b,c) f(b,d) f(b,e) B:
D:
A:
f(c,a) f(c,e)
e=0 e)(a,hD
(a)hE
e)c,d,(a,hB
e)d,(a,hC
(a)hEa
min arga' 1. =
0e' 2. =
Cf(a',d) h (a',d,e')= +d
3. d' arg min
( , ')
( ', ', , ')B
f(c,a') f c e
h a d c e
= + +
+
c4. c' arg min
f(a',b) f(b,c')
f(b,d') f(b,e')
= + +
+ +
b5. b' arg min
)e',d',c',b',(a' Return
f(a,d)
![Page 16: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/16.jpg)
18
exp(w*=4) ”induced width” (max clique size)
Complexity
Elimination operator
OPT
bucket B:
f(c,a) f(c,e)
f(a,b) f(b,c) f(b,d) f(b,e)
bucket C:
bucket D:
bucket E:
bucket A:
e=0
(a)hE
e)c,d,(a,hB
e)d,(a,hC
∑bmin
B
C
D
E
A
f(a,d)
e)(a,hD
Algorithm elim-opt (Dechter, 1996) Non-serial Dynamic Programming (Bertele and Briochi, 1973)
),(),(),(),(),(),(),(min,,,,
ecFebFdbFcbFdaFcaFbaFOPTbcdea
++++++=
![Page 17: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/17.jpg)
Induced Width
))((exp ( * dwrOddw ordering alonggraph primal theof w idth induced the)(* −
The effect of the ordering:
4)( 1* =dw 2)( 2
* =dwconstraint graph
A
D E
C B
B
C
D
E
A
E
D
C
B
A
Finding smallest induced-width is hard
r = number of functions
Bucket-elimination is time and space
![Page 18: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/18.jpg)
Directional i-consistency
DCBR
A
E
C D
B ≠
≠ ≠
≠ ≠≠≠ D
C B
E
D C
B
E
D C
B
E
:AB A:BBC :C
AD C,D :DBE C,E D,E :E
≠
≠
≠≠
≠≠≠Adaptive d-arc d-path
DBDC RR ,CBR
DRCRDR
![Page 19: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/19.jpg)
Mini-bucket approximation
Split a bucket into mini-buckets =>bound complexity
XX gh ≤)()()O(e :decrease complexity lExponentia n rnr eOeO −+→
![Page 20: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/20.jpg)
Mini-Bucket Elimination
A
B C
D
E
P(A)
P(B|A) P(C|A)
P(E|B,C)
P(D|A,B)
Bucket B
Bucket C
Bucket D
Bucket E
Bucket A
P(B|A) P(D|A,B) P(E|B,C)
P(C|A)
E = 0
P(A)
maxB∏
hB (A,D)
MPE* is an upper bound on MPE --U Generating a solution yields a lower bound--L
maxB∏
hD (A)
hC (A,E)
hB (C,E)
hE (A)
![Page 21: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/21.jpg)
MBE-MPE(i) Algorithm Approx-MPE (Dechter&Rish 1997)
n Input: i – max number of variables allowed in a mini-bucket n Output: [lower bound (cost of a sub-optimal solution), upper bound]
Example: approx-mpe(3) versus elim-mpe
2* =w 4* =w
![Page 22: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/22.jpg)
Properties of MBE(i)
n Complexity: O(r exp(i)) time and O(exp(i)) space. n Yields an upper-bound and a lower-bound. n Accuracy: determined by upper/lower (U/L) bound. n As i increases, both accuracy and complexity increase.
n Possible use of mini-bucket approximations: n As anytime algorithms n As heuristics in search
n Other tasks: similar mini-bucket approximations for: belief updating, MAP and MEU (Dechter and Rish, 1997)
![Page 23: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/23.jpg)
Outline n Introduction
n Optimization tasks for graphical models n Solving by inference and search
n Inference n Bucket elimination, dynamic programming n Mini-bucket elimination
n Search n Branch and bound and best-first n Lower-bounding heuristics n AND/OR search spaces
n Hybrids of search and inference n Cutset decomposition n Super-bucket scheme
![Page 24: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/24.jpg)
Branch and Bound
![Page 25: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/25.jpg)
A Combinatorial Auction Example
![Page 26: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/26.jpg)
Search tree for first-choice heuristic
![Page 27: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/27.jpg)
Bucket-Elimination for the Auction Example
![Page 28: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/28.jpg)
Elim-OPT for auction problem
![Page 29: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/29.jpg)
Example MBE-opt
![Page 30: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/30.jpg)
BnB with first-cut heuristic
![Page 31: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/31.jpg)
BnB with first-cut (b) and mini-bucket (a) heuristics
![Page 32: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/32.jpg)
Search with MB heuristic
![Page 33: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/33.jpg)
Bucket-elimination for counting
![Page 34: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/34.jpg)
The Search Space
( ) ( )∑=
Χ=Χ9
1
mini
iX ff
A
E
C
B
F
D
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
C
D
F
E
B
A 0 1
Objective function:
A B f1 0 0 2 0 1 0 1 0 1 1 1 4
A C f2 0 0 3 0 1 0 1 0 0 1 1 1
A E f3 0 0 0 0 1 3 1 0 2 1 1 0
A F f4 0 0 2 0 1 0 1 0 0 1 1 2
B C f5 0 0 0 0 1 1 1 0 2 1 1 4
B D f6 0 0 4 0 1 2 1 0 1 1 1 0
B E f7 0 0 3 0 1 2 1 0 1 1 1 0
C D f8 0 0 1 0 1 4 1 0 0 1 1 0
E F f9 0 0 1 0 1 0 1 0 0 1 1 2
![Page 35: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/35.jpg)
The Search Space
Arc-cost is calculated based on cost components.
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
C
D
F
E
B
A 0 1
A B f1 0 0 2 0 1 0 1 0 1 1 1 4
A
E
C
B
F
D
A C f2 0 0 3 0 1 0 1 0 0 1 1 1
A E f3 0 0 0 0 1 3 1 0 2 1 1 0
A F f4 0 0 2 0 1 0 1 0 0 1 1 2
B C f5 0 0 0 0 1 1 1 0 2 1 1 4
B D f6 0 0 4 0 1 2 1 0 1 1 1 0
B E f7 0 0 3 0 1 2 1 0 1 1 1 0
C D f8 0 0 1 0 1 4 1 0 0 1 1 0
E F f9 0 0 1 0 1 0 1 0 0 1 1 2
3 0 2 2 3 0 2 2 3 0 2 2 3 0 2 2 3 0 2 2 3 0 2 2 3 0 2 2 3 0 2 2
0 0
3 5 3 5 3 5 3 5 1 3 1 3 1 3 1 3
5 6 4 2 2 4 1 0
3 1
2
5 4
0
1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4
5 2 5 2 5 2 5 2 3 0 3 0 3 0 3 0
5 6 4 2 2 4 1 0
0 2 2 5
1 4
( ) ( )∑=
Χ=Χ9
1
mini
iX ff
![Page 36: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/36.jpg)
The Value Function
Value of node = minimal cost solution below it
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
C
D
F
E
B
A 0 1
A B f1 0 0 2 0 1 0 1 0 1 1 1 4
A
E
C
B
F
D
A C f2 0 0 3 0 1 0 1 0 0 1 1 1
A E f3 0 0 0 0 1 3 1 0 2 1 1 0
A F f4 0 0 2 0 1 0 1 0 0 1 1 2
B C f5 0 0 0 0 1 1 1 0 2 1 1 4
B D f6 0 0 4 0 1 2 1 0 1 1 1 0
B E f7 0 0 3 0 1 2 1 0 1 1 1 0
C D f8 0 0 1 0 1 4 1 0 0 1 1 0
E F f9 0 0 1 0 1 0 1 0 0 1 1 2
3 0 0
2 2
6
2
3
3 0 2 2 3 0 2 2 3 0 2 2 3 0 2 2 3 0 2 2 3 0 2 2 3 0 2 2 0 0 0 2 2 2 0 2 0 0 0 2 2 2
3 3 3 1 1 1 1
8 5 3 1
5
5
1 0 1 1 1 0 0 0 1 0 1 1 1 0 0 0
2 2 2 2 0 0 0 0
7 4 2 0
7 4
7
5 0 0
3 5 3 5 3 5 3 5 1 3 1 3 1 3 1 3
5 6 4 2 2 4 1 0
3 1
2
5 4
0
1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4
5 2 5 2 5 2 5 2 3 0 3 0 3 0 3 0
5 6 4 2 2 4 1 0
0 2 2 5
0 4
( ) ( )∑=
Χ=Χ9
1
mini
iX ff
![Page 37: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/37.jpg)
An Optimal Solution
Value of node = minimal cost solution below it
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
C
D
F
E
B
A 0 1
A B f1 0 0 2 0 1 0 1 0 1 1 1 4
A
E
C
B
F
D
A C f2 0 0 3 0 1 0 1 0 0 1 1 1
A E f3 0 0 0 0 1 3 1 0 2 1 1 0
A F f4 0 0 2 0 1 0 1 0 0 1 1 2
B C f5 0 0 0 0 1 1 1 0 2 1 1 4
B D f6 0 0 4 0 1 2 1 0 1 1 1 0
B E f7 0 0 3 0 1 2 1 0 1 1 1 0
C D f8 0 0 1 0 1 4 1 0 0 1 1 0
E F f9 0 0 1 0 1 0 1 0 0 1 1 2
3 0 0
2 2
6
2
3
3 0 2 2 3 0 2 2 3 0 2 2 3 0 2 2 3 0 2 2 3 0 2 2 3 0 2 2 0 0 0 2 2 2 0 2 0 0 0 2 2 2
3 3 3 1 1 1 1
8 5 3 1
5
5
1 0 1 1 1 0 0 0 1 0 1 1 1 0 0 0
2 2 2 2 0 0 0 0
7 4 2 0
7 4
7
5 0 0
3 5 3 5 3 5 3 5 1 3 1 3 1 3 1 3
5 6 4 2 2 4 1 0
3 1
2
5 4
0
1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4 1 2 0 4
5 2 5 2 5 2 5 2 3 0 3 0 3 0 3 0
5 6 4 2 2 4 1 0
0 2 2 5
0 4
![Page 38: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/38.jpg)
Basic Heuristic Search Schemes
Heuristic function f(x) computes a lower bound on the best extension of x and can be used to guide a heuristic search algorithm. We focus on
1.Branch and Bound Use heuristic function f(xp) to prune the depth-first search tree. Linear space
2.Best-First Search Always expand the node with the highest heuristic value f(xp). Needs lots of memory
f ≤ L
L
![Page 39: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/39.jpg)
Classic Branch-and-Bound
n
g(n)
h(n)
LB(n) = g(n) + h(n)
Lower Bound LB
OR Search Tree
Prune if LB(n) ≥ UB
Upper Bound UB
![Page 40: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/40.jpg)
How to Generate Heuristics
n The principle of relaxed models
n Linear optimization for integer programs n Mini-bucket elimination n Bounded directional consistency ideas
![Page 41: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/41.jpg)
Static MBE Heuristics
n
B: P(E|B,C) P(D|A,B) P(B|A)
A:
E:
D:
C: P(C|A) hB(E,C)
hB(D,A)
hC(E,A)
P(A) hE(A) hD(A)
f(a,e,D) = P(a) + hB(D,a) + hC(e,a)
g h – is admissible
A
B C
D E
Belief Network
E
E
D A
D
B D
B 0
1
1
0
1
0
f(a,e,D))=g(a,e) + H(a,e,D )
![Page 42: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/42.jpg)
Heuristics Properties
n MB Heuristic is monotone, admissible n Retrieved in linear time n IMPORTANT:
n Heuristic strength can vary by MB(i). n Higher i-bound ⇒ more pre-processing ⇒ stronger
heuristic ⇒ less search.
n Allows controlled trade-off between preprocessing and search
![Page 43: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/43.jpg)
Experimental Methodology
Algorithms n BBMB(i) – Branch and Bound with MB(i) n BBFB(i) - Best-First with MB(i) n MBE(i)
Test networks: n Random Coding (Bayesian) n CPCS (Bayesian) n Random (CSP)
Measures of performance n Compare accuracy given a fixed amount of time - how close is the
cost found to the optimal solution n Compare trade-off performance as a function of time
![Page 44: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/44.jpg)
Empirical Evaluation of mini-bucket heuristics, Bayesian networks, coding
Time [sec]
0 10 20 30
% S
olve
d E
xact
ly
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
BBMB i=2 BFMB i=2 BBMB i=6 BFMB i=6 BBMB i=10 BFMB i=10 BBMB i=14 BFMB i=14
Random Coding, K=100, noise=0.28 Random Coding, K=100, noise 0.32
Time [sec]
0 10 20 30
% S
olve
d E
xact
ly
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
BBMB i=6 BFMB i=6 BBMB i=10 BFMB i=10 BBMB i=14 BFMB i=14
Random Coding, K=100, noise=0.32
![Page 45: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/45.jpg)
51
Max-CSP experiments (Kask and Dechter, 2000)
![Page 46: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/46.jpg)
Dynamic MB Heuristics
n Rather than pre-compiling, the mini-bucket heuristics can be generated during search
n Dynamic mini-bucket heuristics use the Mini-Bucket algorithm to produce a bound for any node in the search space
(a partial assignment, along the given variable ordering)
![Page 47: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/47.jpg)
Branch and Bound w/ Mini-Buckets
n BB with static Mini-Bucket Heuristics (s-BBMB) n Heuristic information is pre-compiled before search. Static
variable ordering, prunes current variable
n BB with dynamic Mini-Bucket Heuristics (d-BBMB) n Heuristic information is assembled during search. Static
variable ordering, prunes current variable
n BB with dynamic Mini-Bucket-Tree Heuristics (BBBT) n Heuristic information is assembled during search. Dynamic
variable ordering, prunes all future variables
![Page 48: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/48.jpg)
Empirical Evaluation n Algorithms:
n Complete n BBBT n BBMB
n Incomplete n DLM n GLS n SLS n IJGP n IBP (coding)
n Measures: n Time n Accuracy (% exact) n #Backtracks n Bit Error Rate (coding)
n Benchmarks: n Coding networks n Bayesian Network Repository n Grid networks (N-by-N) n Random noisy-OR networks n Random networks
![Page 49: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/49.jpg)
Real World Benchmarks
Average Accuracy and Time. 30 samples, 10 observations, 30 seconds
![Page 50: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/50.jpg)
Empirical Results: Max-CSP n Random Binary Problems: <N, K, C, T>
n N: number of variables n K: domain size n C: number of constraints n T: Tightness
n Task: Max-CSP
![Page 51: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/51.jpg)
BBBT(i) vs. BBMB(i).
BBBT(i) vs BBMB(i), N=100
i=2 i=3 i=4 i=5 i=6 i=7 i=2
![Page 52: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/52.jpg)
Outline n Introduction
n Optimization tasks for graphical models n Solving by inference and search
n Inference n Bucket elimination, dynamic programming n Mini-bucket elimination, belief propagation
n Search n Branch and bound and best-first n Lower-bounding heuristics n AND/OR search spaces
n Hybrids of search and inference n Cutset decomposition n Super-bucket scheme
![Page 53: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/53.jpg)
Classic OR Search Space
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
E
C
F
D
B
A 0 1
Ordering: A B E C D F
A
D
B C
E
F
![Page 54: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/54.jpg)
AND/OR Search Space
A OR
0 AND 1
B OR B
0 AND 1 0 1
E OR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
A
D
B C
E
F
A
D
B
C E
F
Primal graph DFS tree
A
D
B C
E
F
A
D
B C
E
F
![Page 55: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/55.jpg)
AND/OR vs. OR
E 0 1 0 1 0 1 0 1
0 C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0 1
A 0 1
A OR
0 AND 1
B OR B
0 AND 1 0 1
E OR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
E 0 1 0 1 0 1 0 1
0 C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0 C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
F A
D
B
C E
F
AND/OR size: exp(4), OR size exp(6)
![Page 56: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/56.jpg)
OR space vs. AND/OR space
width
height
OR space AND/OR space
Time (sec.) Nodes Backtracks Time (sec.) AND nodes OR nodes
5 10 3.154 2,097,150 1,048,575 0.03 10,494 5,247
4 9 3.135 2,097,150 1,048,575 0.01 5,102 2,551
5 10 3.124 2,097,150 1,048,575 0.03 8,926 4,463
4 10 3.125 2,097,150 1,048,575 0.02 7,806 3,903
5 13 3.104 2,097,150 1,048,575 0.1 36,510 18,255
Random graphs with 20 nodes, 20 edges and 2 values per node.
![Page 57: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/57.jpg)
AND/OR vs. OR
F
AND/OR
A
D
B C
E
F A
D
B
C E
F
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0
A 0 1
A OR
0 AND 1
B OR B
0 AND 1 0
E OR C E C E C
OR D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 1 0 1 0 1 0 1 1 0 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0
A 0 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0
A 0 1
OR
(A=1,B=1) (B=0,C=0)
![Page 58: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/58.jpg)
AND/OR vs. OR
F
AND/OR
A
D
B C
E
F A
D
B
C E
F
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0
A 0 1
A OR
0 AND 1
B OR B
0 AND 1 0
E OR C E C E C
OR D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 1 0 1 0 1 0 1 1 0 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0
A 0 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 B 1 0
A 0 1
OR
(A=1,B=1) (B=0,C=0)
Space: linear Time: O(exp(m)) O(w* log n)
Linear space, Time: O(exp(n))
![Page 59: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/59.jpg)
#CSP – AND/OR Search Tree
A
E
C
B
F
D
A
D
B
E C
F
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
A OR
0 AND
B OR
0 AND
OR E
OR F F
AND 0 1 0 1
AND 0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
B
0
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
![Page 60: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/60.jpg)
#CSP – AND/OR Search Tree
A
E
C
B
F
D
A
D
B
E C
F
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
A OR
0 AND
B OR
0 AND
OR E
OR F F
AND 0 1 0 1
AND 0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
B
0
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
![Page 61: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/61.jpg)
#CSP – AND/OR Tree DFS
A
E
C
B
F
D
A
D
B
E C
F
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
A OR
0 AND
B OR
0 AND
OR E
OR F
AND 0 1
AND 0 1
C
D D
0 1 0 1
0 1
1 1 1 0 0 1
2 1 1
2 1 1 0
3 1
3
9
9
1
E
F F
0 1 0 1
0 1
C
D
0 1
0 1
1
B
0
E
F
0 1
0 1
C
D D
0 1 0 1
0 1
1
E
F
0 1
0 1
C
D
0 1
0 1
1 1 0 1 1 1
2 1 2
2 1 2 0
2 3
6
1 1 1 0 1 0 1 0 1 1
2 1 1 1 2
3 1
2 1 1 0 0 1 0 2
1 2
3 2
5
5
14
OR node: Marginalization operator (summation)
AND node: Combination operator (product)
![Page 62: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/62.jpg)
AND/OR Tree Search for COP
A
0
B
0
E
F F
0 1 0 1
OR
AND
OR
AND
OR
OR
AND
AND 0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
A
E
C
B
F
D
A B f1 0 0 2 0 1 0 1 0 1 1 1 4
A C f2 0 0 3 0 1 0 1 0 0 1 1 1
A E f3 0 0 0 0 1 3 1 0 2 1 1 0
A F f4 0 0 2 0 1 0 1 0 0 1 1 2
B C f5 0 0 0 0 1 1 1 0 2 1 1 4
B D f6 0 0 4 0 1 2 1 0 1 1 1 0
B E f7 0 0 3 0 1 2 1 0 1 1 1 0
C D f8 0 0 1 0 1 4 1 0 0 1 1 0
E F f9 0 0 1 0 1 0 1 0 0 1 1 2
5 6 4 2 3 0 2 2
5 2 0 2
5 2 0 2
3 3
6
5
5
5
3 1 3 5
2
2 4 1 0 3 0 2 2
2 0 0 2
2 0 0 2
4 1
5
5 4 1 3
0
0
1
B
0
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
5 6 4 2 1 2 0 4 2 4 1 0 1 2 0 4
6 4
7
7
5 2 1 0 2 0 1 0
5 2 1 0 2 0 1 0
4 2 4 0
0
0 2 5 2 2 5 3 0
1 4
A
D
B
E C
F
AND node = Combination operator (summation)
( )∑ =
9
1min :Goal
i iX Xf
OR node = Marginalization operator (minimization)
![Page 63: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/63.jpg)
Summary of AND/OR Search Trees
n Based on a backbone pseudo-tree
n A solution is a subtree
n Each node has a value – cost of the optimal solution to the subproblem (computed recursively based on the values of the descendants)
n Solving a task = finding the value of the root node
n AND/OR search tree and algorithms are ([Freuder & Quinn85], [Collin, Dechter & Katz91], [Bayardo & Miranker95])
n Space: O(n) n Time: O(exp(m)), where m is the depth of the pseudo-tree n Time: O(exp(w* log n)) n BFS is time and space O(exp(w* log n)
![Page 64: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/64.jpg)
Caching A
D
B C
E
F
A
D
B
C E
F
G H
J
K
G
H
J
K A OR
0 AND 1
B OR B
0 AND 1 0 1
E OR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
context(B) = {A, B} context(C) = {A,B,C} context(D) = {D} context(F) = {F}
![Page 65: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/65.jpg)
An AND/OR Graph: Caching Goods
A
D
B C
E
F
A
D
B
C E
F
G H
J
K
G
H
J
K A OR
0 AND 1
B OR B
0 AND 1 0 1
E OR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
![Page 66: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/66.jpg)
Context-based Caching n Caching is possible when context is the same
n context = parent-separator set in induced pseudo-graph
= current variable + parents connected to subtree below
A
D
B C
E
F
A
D
B
C E
F
G H
J
K
G
H
J
K
context(B) = {A, B}
context(c) = {A,B,C}
context(D) = {D}
context(F) = {F}
![Page 67: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/67.jpg)
Complexity of AND/OR Graph
n Theorem: Traversing the AND/OR search graph is time and space exponential in the induced width/tree-width.
n If applied to the OR graph complexity is time and space exponential in the path-width.
![Page 68: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/68.jpg)
#CSP – AND/OR Search Tree
A
E
C
B
F
D
A
D
B
E C
F
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
A OR
0 AND
B OR
0 AND
OR E
OR F F
AND 0 1 0 1
AND 0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
B
0
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
![Page 69: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/69.jpg)
#CSP – AND/OR Tree DFS
A
E
C
B
F
D
A
D
B
E C
F
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
A OR
0 AND
B OR
0 AND
OR E
OR F
AND 0 1
AND 0 1
C
D D
0 1 0 1
0 1
1 1 1 0 0 1
2 1 1
2 1 1 0
3 1
3
9
9
1
E
F F
0 1 0 1
0 1
C
D
0 1
0 1
1
B
0
E
F
0 1
0 1
C
D D
0 1 0 1
0 1
1
E
F
0 1
0 1
C
D
0 1
0 1
1 1 0 1 1 1
2 1 2
2 1 2 0
2 3
6
1 1 1 0 1 0 1 0 1 1
2 1 1 1 2
3 1
2 1 1 0 0 1 0 2
1 2
3 2
5
5
14
![Page 70: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/70.jpg)
#CSP – AND/OR Search Graph (Caching Goods)
A
E
C
B
F
D
A
D
B
E C
F
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
A OR
0 AND
B OR
0 AND
OR E
OR F F
AND 0 1
AND 0 1
C
D D
0 1
0 1
1
E C
D D
0 1
1
B
0
E
F F
0 1
C
1
E C
[EA] [CB]
![Page 71: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/71.jpg)
#CSP – AND/OR Search Graph (Caching Goods)
A
E
C
B
F
D
A
D
B
E C
F
A B C RABC 0 0 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
A B E RABE 0 0 0 1 0 0 1 0 0 1 0 1 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 0
A E F RAEF 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0
B C D RBCD 0 0 0 1 0 0 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1
A OR
0 AND
B OR
0 AND
OR E
OR F F
AND 0 1
AND 0 1
C
D D
0 1
0 1
1
E C
D D
0 1
1
B
0
E
F F
0 1
C
1
E C
Time and Space O(exp(w*))
Space O(exp(sep-w*))
![Page 72: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/72.jpg)
All Four Search Spaces
Full OR search tree
126 nodes
Full AND/OR search tree
54 AND nodes
Context minimal OR search graph
28 nodes
Context minimal AND/OR search graph
18 AND nodes
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
C D
F E
B A 0 1
A OR 0 AND B OR
0 AND
OR E
OR F F
AND 0 1 0 1
AND 0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
B
0
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1
0 1
0 1 0 1
0 1 0 1
C D
F E
B A 0 1
A OR 0 AND B OR
0 AND OR E
OR F F AND 0 1
AND 0 1
C
D D 0 1
0 1
1
E C
D D
0 1
1
B
0
E
F F
0 1
C 1
E C
![Page 73: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/73.jpg)
AND/OR vs. OR DFS algorithms
n AND/OR tree n Space: O(n) n Time: O(n km)
O(n kw* log n) (Freuder85; Bayardo95; Darwiche01)
n AND/OR graph n Space: O(n kw*) n Time: O(n kw*)
k = domain size m = pseudo-tree depth n = number of variables w*= induced width pw*= path width n OR tree
n Space: O(n) n Time: O(kn)
n OR graph n Space: O(n kpw*) n Time: O(n kpw*)
![Page 74: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/74.jpg)
Searching AND/OR Graphs
n AO(i): searches depth-first, cache i-context n i = the max size of a cache table (i.e.
number of variables in a context)
i=0 i=w*
Space: O(n)
Time: O(exp(w* log n))
Space: O(exp w*)
Time: O(exp w*)
AO(i) time complexity?
![Page 75: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/75.jpg)
AND/OR Branch-and-Bound (AOBB)
n Associate each node n with a static heuristic estimate h(n) of v(n) n h(n) is a lower bound on the value v(n)
n For every node n in the search tree: n ub(n) – current best solution cost rooted at n n lb(n) – lower bound on the minimal cost at n
![Page 76: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/76.jpg)
Lower/Upper Bounds
1 0
X
A B
0
C h(C) v(A)
h(B,0) LB(B,0) = h(B,0)
LB(X,1) = l(X,1) + v(A) + h(C) + LB(B)
Prune below AND node (B,0) if LB(X) ≥ UB(X)
LB(X) = LB(X,1)
LB(B) = LB(B,0)
UB(X) LB(X) UB(X) = best cost below X (i.e. v(X,0))
v(X,0)
![Page 77: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/77.jpg)
Shallow/Deep Cutoffs
Deep cutoff
0
X
1
UB(X)
LB(X) = h(X,1)
1 0
X
A B
0
E D
0
C
UB(X)
LB(X)
Shallow cutoff
Reminiscent of Minimax shallow/deep cutoffs
Prune if LB(X) ≥ UB(X)
![Page 78: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/78.jpg)
Summary of AOBB
n Traverses the AND/OR search tree in a depth-first manner
n Lower bounds computed based on heuristic estimates of nodes at the frontier of search, as well as the values of nodes already explored
n Prunes the search space as soon as an upper-lower bound violation occurs
![Page 79: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/79.jpg)
Heuristics for AND/OR
n In the AND/OR search space h(n) can be computed using any heuristic. We used:
n Static Mini-Bucket heuristics
n Dynamic Mini-Bucket heuristics
n Maintaining FDAC [Larrosa & Schiex03]
(full directional soft arc-consistency)
![Page 80: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/80.jpg)
Empirical Evaluation n Tasks
n Solving WCSPs n Finding the MPE in belief networks
n Benchmarks (WCSP) n Random binary WCSPs n RLFAP networks (CELAR6) n Bayesian Networks Repository
n Algorithms n s-AOMB(i), d-AOMB(i), AOMFDAC n s-BBMB(i), d-BBMB(i), BBMFDAC n Static variable ordering (dfs traversal of the pseudo-tree)
![Page 81: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/81.jpg)
Random Binary WCSPs (Marinescu and Dechter, 2005)
Random networks with n=20 (number of variables), d=5 (domain size), c=100 (number of constraints), t=70% (tightness). Time limit 180 seconds. AO search is superior to OR search
S-AOMB vs S-BBMB D-AOMB vs D-BBMB
![Page 82: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/82.jpg)
Random Binary WCSPs (contd.)
n=20 (variables), d=5 (domain size), c=100 (constraints), t=70% (tightness)
dense sparse
n=50 (variables), d=5 (domain size), c=80 (constraints), t=70% (tightness)
AOMB for large i is competitive with AOMFDAC
![Page 83: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/83.jpg)
Resource Allocation
Instance BBMFDAC AOMFDAC time (sec) nodes time (sec) nodes
CELAR6-SUB0 2.78 1,871 1.98 435 CELAR6-SUB1 2,420.93 364,986 981.98 180,784 CELAR6-SUB2 8,801.12 19,544,182 1,138.87 175,377 CELAR6-SUB3 38,889.20 91,168,896 4,028.59 846,986 CELAR6-SUB4 84,478.40 6,955,039 47,115.40 4,643,229
CELAR6 sub-instances
Radio Link Frequency Assignment Problem (RLFAP)
AOMFDAC is superior to ORMFDAC
![Page 84: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/84.jpg)
Bayesian Networks Repository
Network Algorithm i=2 i=3 i=4 i=5 (n,d,w*,h) time nodes time nodes time nodes time nodes
s-AOMB(i) - 8.5M - 7.6M 46.22 807K 0.563 9.6K Barley s-BBMB(i) - 16M - 18M - 17M - 14M
(48,67,7,17) d-AOMB(i) - 79K 136.0 23K 12.55 667 45.95 567 d-BBMB(i) - 2.2M - 1M 346.1 76K - 86K s-AOMB(i) 57.36 1.2M 12.08 260K 7.203 172K 1.657 43K
Munin1 s-BBMB(i) - 8.5M - 9M - 10M - 8M (189,21,11,24) d-AOMB(i) 66.56 185K 12.47 8.1K 10.30 1.6K 11.99 523
d-BBMB(i) - 405K - 430K - 235K 14.63 917 s-AOMB(i) - 5.9M - 4.9M 1.313 17K 0.453 6K
Munin3 s-BBMB(i) - 1.4M - 1.2M - 316K - 1.5M (1044,21,7,25) d-AOMB(i) - 2.3M 68.64 58K 3.594 5.9K 2.844 3.8K
d-BBMB(i) - 33K - 125K - 52K - 31K
Time limit 600 seconds
available at http://www.cs.huji.ac.il/labs/compbio/Repository
Static AO is better with accurate heuristic (large i)
![Page 85: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/85.jpg)
Outline n Introduction
n Optimization tasks for graphical models n Solving by inference and search
n Inference n Bucket elimination, dynamic programming n Mini-bucket elimination, belief propagation
n Search n Branch and bound and best-first n Lower-bounding heuristics n AND/OR search spaces
n Searching trees n Searching graphs
n Hybrids of search and inference n Cutset decomposition n Super-bucket scheme
![Page 86: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/86.jpg)
From Searching Trees to Searching Graphs
n Any two nodes that root identical subtrees/subgraphs can be merged
n Minimal AND/OR search graph: closure under merge of the AND/OR search tree
n Inconsistent sub-trees can be pruned too. n Some portions can be collapsed or reduced.
![Page 87: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/87.jpg)
AND/OR Search Graph
A
E
C
B
F
D
A
D
B
E C
F Primal graph Pseudo-tree
context(A) = {A} context(B) = {B,A} context(C) = {C,B} context(D) = {D} context(E) = {E,A} context(F) = {F}
A OR
0 AND
B OR
0 AND
OR E
OR F F
AND 0 1 0 1
AND 0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
B
0
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
1
E
F F
0 1 0 1
0 1
C
D D
0 1 0 1
0 1
![Page 88: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/88.jpg)
AND/OR Search Graph
OR
AND
OR
AND
OR E
OR F F
AND 0 1
AND 0 1
D D
0 1
1
1
E C
D D
0 1
E
F F
0 1
A
0
B
0
C
0
1
B
0
C
1
E C
A
E
C
B
F
D
A
D
B
E C
F Primal graph Pseudo-tree
context(A) = {A} context(B) = {B,A} context(C) = {C,B} context(D) = {D} context(E) = {E,A} context(F) = {F}
![Page 89: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/89.jpg)
Context-based caching A
D
B
E C
F
context(A) = {A} context(B) = {B,A} context(C) = {C,B} context(D) = {D} context(E) = {E,A} context(F) = {F}
Cache Table (C)
B C Value 0 0 5 0 1 2 1 0 2 1 1 0
A
0
B
0
E C
D D
0 1
1
E C
D D
0 1
1
B
0
E C
1
E C
5
5
5 2
3
6
2 0
4
5
4 4
4 6
7
7
A
E
C
B
F
D
Primal graph
Space: O(exp(2))
![Page 90: Chapter 13€¦ · Chapter 13 Constraint Optimization And counting, and enumeration 275 class . Outline ! Introduction ! Optimization tasks for graphical models ! Solving optimization](https://reader034.vdocuments.site/reader034/viewer/2022042912/5f4575384403e55cbc2c0252/html5/thumbnails/90.jpg)
Searching AND/OR Graphs
n AO(j): searches depth-first, cache j-context n j = the max size of a cache table (i.e.
number of variables in a context)
j=0 j=w*
Space: O(n)
Time: O(exp(w* log n))
Space: O(exp w*)
Time: O(exp w*)
AO(j) time complexity?