problem solving with constraints csce421/821, spring 2011 cse.unl/~ choueiry/f11-496-896
DESCRIPTION
Arc Consistency. Problem Solving With Constraints CSCE421/821, Spring 2011 www.cse.unl.edu/~ choueiry/F11-496-896 All questions: [email protected] Berthe Y. Choueiry (Shu-we-ri) Avery Hall, Room 360 Tel: +1(402)472-5444. Lecture Sources. Required reading - PowerPoint PPT PresentationTRANSCRIPT
Problem Solving with Constraints
Arc Consistency 1
Problem Solving With ConstraintsCSCE421821 Spring 2011
wwwcseunledu~choueiryF11-496-896All questions cse496cpcseunledu
Berthe Y Choueiry (Shu-we-ri)Avery Hall Room 360Tel +1(402)472-5444
Arc Consistency
Problem Solving with Constraints
Arc Consistency 2
Lecture SourcesRequired reading1 Algorithms for Constraint Satisfaction Problems Mackworth and
Freuder AIJ85 (focus on Node consistency arc consistency)2 Sections 31 amp 32 Chapter 3 Constraint Processing Dechter
Recommended reading1 Bartak Consistency Techniques (link)2 Constraint Propagation with Interval Labels Davis AIJ87
Problem Solving with Constraints
Arc Consistency 3
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 4
Basic Consistency Notation
Examining finite binary CSPs
Notation
Given variables ijk with values xyz the predicate
Pijk(xyz) is true iff the 3-tuple (xyz) Rijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Problem Solving with Constraints
Arc Consistency 5
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower bound f(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Problem Solving with Constraints
Arc Consistency 6
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experiments1 Random CSPs (Model B)
2 Statistical Analysis
Problem Solving with Constraints
Arc Consistency 7
Node consistency (NC)Procedure NC(V1V2hellipVn)
For i 1 until n do DVi DVi x | Pi(x)
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation
If Dvi (Di= ) empty Then BREAK
Problem Solving with Constraints
Arc Consistency 8
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 9
Arc Consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj
bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Problem Solving with Constraints
Arc Consistency
Main Functionsbull Variable-value pair
ndash (Via) ⟨Via variable value from its domain⟩
bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise
bull SUPPORTED(⟨Via ⟩ Vj)
ndash Verifies that ⟨Via has at least one support in ⟩ CViVj
ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo
bull REVISE(ViVj)
ndash Updates DVi given RViVj
ndash BREAK when domain wipe-out
10
Problem Solving with Constraints
Arc Consistency
Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms
ndash Update all domains given all constraints
ndash Call REVISE
ndash Differ in how they manage updates (queue)
bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of
graph deg
bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation
is accessed
ndash Is a cost measure to compare algorithmsrsquo performance in practive
bull Assumptionndash Domains are sorted alphabetically (lexicographic)
11
Problem Solving with Constraints
Arc Consistency
CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation
ndash Verifies whether a constraint exists between ViVj
ndash If a constraint existsbull Increases CC
bull Accesses the constraint between ViVj
bull Verifies if (ab)isinRViVj
ndash Returns true if (ab) is consistent
ndash Returns false if (ab) is not consistent
ndash If no constraints existbull Returns true (universal constraint)
bull Pseudo code necessarybull Complexity cost in practice
12
Problem Solving with Constraints
Arc Consistency
SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)
1 support nil
2 b DVj
3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩
4 Then Begin
5 support true
6 RETURN support
7 End
8 RETURN support
bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation
13
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 2
Lecture SourcesRequired reading1 Algorithms for Constraint Satisfaction Problems Mackworth and
Freuder AIJ85 (focus on Node consistency arc consistency)2 Sections 31 amp 32 Chapter 3 Constraint Processing Dechter
Recommended reading1 Bartak Consistency Techniques (link)2 Constraint Propagation with Interval Labels Davis AIJ87
Problem Solving with Constraints
Arc Consistency 3
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 4
Basic Consistency Notation
Examining finite binary CSPs
Notation
Given variables ijk with values xyz the predicate
Pijk(xyz) is true iff the 3-tuple (xyz) Rijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Problem Solving with Constraints
Arc Consistency 5
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower bound f(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Problem Solving with Constraints
Arc Consistency 6
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experiments1 Random CSPs (Model B)
2 Statistical Analysis
Problem Solving with Constraints
Arc Consistency 7
Node consistency (NC)Procedure NC(V1V2hellipVn)
For i 1 until n do DVi DVi x | Pi(x)
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation
If Dvi (Di= ) empty Then BREAK
Problem Solving with Constraints
Arc Consistency 8
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 9
Arc Consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj
bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Problem Solving with Constraints
Arc Consistency
Main Functionsbull Variable-value pair
ndash (Via) ⟨Via variable value from its domain⟩
bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise
bull SUPPORTED(⟨Via ⟩ Vj)
ndash Verifies that ⟨Via has at least one support in ⟩ CViVj
ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo
bull REVISE(ViVj)
ndash Updates DVi given RViVj
ndash BREAK when domain wipe-out
10
Problem Solving with Constraints
Arc Consistency
Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms
ndash Update all domains given all constraints
ndash Call REVISE
ndash Differ in how they manage updates (queue)
bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of
graph deg
bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation
is accessed
ndash Is a cost measure to compare algorithmsrsquo performance in practive
bull Assumptionndash Domains are sorted alphabetically (lexicographic)
11
Problem Solving with Constraints
Arc Consistency
CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation
ndash Verifies whether a constraint exists between ViVj
ndash If a constraint existsbull Increases CC
bull Accesses the constraint between ViVj
bull Verifies if (ab)isinRViVj
ndash Returns true if (ab) is consistent
ndash Returns false if (ab) is not consistent
ndash If no constraints existbull Returns true (universal constraint)
bull Pseudo code necessarybull Complexity cost in practice
12
Problem Solving with Constraints
Arc Consistency
SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)
1 support nil
2 b DVj
3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩
4 Then Begin
5 support true
6 RETURN support
7 End
8 RETURN support
bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation
13
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 3
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 4
Basic Consistency Notation
Examining finite binary CSPs
Notation
Given variables ijk with values xyz the predicate
Pijk(xyz) is true iff the 3-tuple (xyz) Rijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Problem Solving with Constraints
Arc Consistency 5
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower bound f(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Problem Solving with Constraints
Arc Consistency 6
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experiments1 Random CSPs (Model B)
2 Statistical Analysis
Problem Solving with Constraints
Arc Consistency 7
Node consistency (NC)Procedure NC(V1V2hellipVn)
For i 1 until n do DVi DVi x | Pi(x)
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation
If Dvi (Di= ) empty Then BREAK
Problem Solving with Constraints
Arc Consistency 8
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 9
Arc Consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj
bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Problem Solving with Constraints
Arc Consistency
Main Functionsbull Variable-value pair
ndash (Via) ⟨Via variable value from its domain⟩
bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise
bull SUPPORTED(⟨Via ⟩ Vj)
ndash Verifies that ⟨Via has at least one support in ⟩ CViVj
ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo
bull REVISE(ViVj)
ndash Updates DVi given RViVj
ndash BREAK when domain wipe-out
10
Problem Solving with Constraints
Arc Consistency
Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms
ndash Update all domains given all constraints
ndash Call REVISE
ndash Differ in how they manage updates (queue)
bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of
graph deg
bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation
is accessed
ndash Is a cost measure to compare algorithmsrsquo performance in practive
bull Assumptionndash Domains are sorted alphabetically (lexicographic)
11
Problem Solving with Constraints
Arc Consistency
CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation
ndash Verifies whether a constraint exists between ViVj
ndash If a constraint existsbull Increases CC
bull Accesses the constraint between ViVj
bull Verifies if (ab)isinRViVj
ndash Returns true if (ab) is consistent
ndash Returns false if (ab) is not consistent
ndash If no constraints existbull Returns true (universal constraint)
bull Pseudo code necessarybull Complexity cost in practice
12
Problem Solving with Constraints
Arc Consistency
SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)
1 support nil
2 b DVj
3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩
4 Then Begin
5 support true
6 RETURN support
7 End
8 RETURN support
bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation
13
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 4
Basic Consistency Notation
Examining finite binary CSPs
Notation
Given variables ijk with values xyz the predicate
Pijk(xyz) is true iff the 3-tuple (xyz) Rijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Problem Solving with Constraints
Arc Consistency 5
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower bound f(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Problem Solving with Constraints
Arc Consistency 6
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experiments1 Random CSPs (Model B)
2 Statistical Analysis
Problem Solving with Constraints
Arc Consistency 7
Node consistency (NC)Procedure NC(V1V2hellipVn)
For i 1 until n do DVi DVi x | Pi(x)
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation
If Dvi (Di= ) empty Then BREAK
Problem Solving with Constraints
Arc Consistency 8
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 9
Arc Consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj
bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Problem Solving with Constraints
Arc Consistency
Main Functionsbull Variable-value pair
ndash (Via) ⟨Via variable value from its domain⟩
bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise
bull SUPPORTED(⟨Via ⟩ Vj)
ndash Verifies that ⟨Via has at least one support in ⟩ CViVj
ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo
bull REVISE(ViVj)
ndash Updates DVi given RViVj
ndash BREAK when domain wipe-out
10
Problem Solving with Constraints
Arc Consistency
Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms
ndash Update all domains given all constraints
ndash Call REVISE
ndash Differ in how they manage updates (queue)
bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of
graph deg
bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation
is accessed
ndash Is a cost measure to compare algorithmsrsquo performance in practive
bull Assumptionndash Domains are sorted alphabetically (lexicographic)
11
Problem Solving with Constraints
Arc Consistency
CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation
ndash Verifies whether a constraint exists between ViVj
ndash If a constraint existsbull Increases CC
bull Accesses the constraint between ViVj
bull Verifies if (ab)isinRViVj
ndash Returns true if (ab) is consistent
ndash Returns false if (ab) is not consistent
ndash If no constraints existbull Returns true (universal constraint)
bull Pseudo code necessarybull Complexity cost in practice
12
Problem Solving with Constraints
Arc Consistency
SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)
1 support nil
2 b DVj
3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩
4 Then Begin
5 support true
6 RETURN support
7 End
8 RETURN support
bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation
13
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 5
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower bound f(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Problem Solving with Constraints
Arc Consistency 6
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experiments1 Random CSPs (Model B)
2 Statistical Analysis
Problem Solving with Constraints
Arc Consistency 7
Node consistency (NC)Procedure NC(V1V2hellipVn)
For i 1 until n do DVi DVi x | Pi(x)
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation
If Dvi (Di= ) empty Then BREAK
Problem Solving with Constraints
Arc Consistency 8
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 9
Arc Consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj
bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Problem Solving with Constraints
Arc Consistency
Main Functionsbull Variable-value pair
ndash (Via) ⟨Via variable value from its domain⟩
bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise
bull SUPPORTED(⟨Via ⟩ Vj)
ndash Verifies that ⟨Via has at least one support in ⟩ CViVj
ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo
bull REVISE(ViVj)
ndash Updates DVi given RViVj
ndash BREAK when domain wipe-out
10
Problem Solving with Constraints
Arc Consistency
Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms
ndash Update all domains given all constraints
ndash Call REVISE
ndash Differ in how they manage updates (queue)
bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of
graph deg
bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation
is accessed
ndash Is a cost measure to compare algorithmsrsquo performance in practive
bull Assumptionndash Domains are sorted alphabetically (lexicographic)
11
Problem Solving with Constraints
Arc Consistency
CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation
ndash Verifies whether a constraint exists between ViVj
ndash If a constraint existsbull Increases CC
bull Accesses the constraint between ViVj
bull Verifies if (ab)isinRViVj
ndash Returns true if (ab) is consistent
ndash Returns false if (ab) is not consistent
ndash If no constraints existbull Returns true (universal constraint)
bull Pseudo code necessarybull Complexity cost in practice
12
Problem Solving with Constraints
Arc Consistency
SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)
1 support nil
2 b DVj
3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩
4 Then Begin
5 support true
6 RETURN support
7 End
8 RETURN support
bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation
13
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 6
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experiments1 Random CSPs (Model B)
2 Statistical Analysis
Problem Solving with Constraints
Arc Consistency 7
Node consistency (NC)Procedure NC(V1V2hellipVn)
For i 1 until n do DVi DVi x | Pi(x)
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation
If Dvi (Di= ) empty Then BREAK
Problem Solving with Constraints
Arc Consistency 8
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 9
Arc Consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj
bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Problem Solving with Constraints
Arc Consistency
Main Functionsbull Variable-value pair
ndash (Via) ⟨Via variable value from its domain⟩
bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise
bull SUPPORTED(⟨Via ⟩ Vj)
ndash Verifies that ⟨Via has at least one support in ⟩ CViVj
ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo
bull REVISE(ViVj)
ndash Updates DVi given RViVj
ndash BREAK when domain wipe-out
10
Problem Solving with Constraints
Arc Consistency
Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms
ndash Update all domains given all constraints
ndash Call REVISE
ndash Differ in how they manage updates (queue)
bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of
graph deg
bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation
is accessed
ndash Is a cost measure to compare algorithmsrsquo performance in practive
bull Assumptionndash Domains are sorted alphabetically (lexicographic)
11
Problem Solving with Constraints
Arc Consistency
CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation
ndash Verifies whether a constraint exists between ViVj
ndash If a constraint existsbull Increases CC
bull Accesses the constraint between ViVj
bull Verifies if (ab)isinRViVj
ndash Returns true if (ab) is consistent
ndash Returns false if (ab) is not consistent
ndash If no constraints existbull Returns true (universal constraint)
bull Pseudo code necessarybull Complexity cost in practice
12
Problem Solving with Constraints
Arc Consistency
SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)
1 support nil
2 b DVj
3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩
4 Then Begin
5 support true
6 RETURN support
7 End
8 RETURN support
bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation
13
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 7
Node consistency (NC)Procedure NC(V1V2hellipVn)
For i 1 until n do DVi DVi x | Pi(x)
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation
If Dvi (Di= ) empty Then BREAK
Problem Solving with Constraints
Arc Consistency 8
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 9
Arc Consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj
bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Problem Solving with Constraints
Arc Consistency
Main Functionsbull Variable-value pair
ndash (Via) ⟨Via variable value from its domain⟩
bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise
bull SUPPORTED(⟨Via ⟩ Vj)
ndash Verifies that ⟨Via has at least one support in ⟩ CViVj
ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo
bull REVISE(ViVj)
ndash Updates DVi given RViVj
ndash BREAK when domain wipe-out
10
Problem Solving with Constraints
Arc Consistency
Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms
ndash Update all domains given all constraints
ndash Call REVISE
ndash Differ in how they manage updates (queue)
bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of
graph deg
bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation
is accessed
ndash Is a cost measure to compare algorithmsrsquo performance in practive
bull Assumptionndash Domains are sorted alphabetically (lexicographic)
11
Problem Solving with Constraints
Arc Consistency
CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation
ndash Verifies whether a constraint exists between ViVj
ndash If a constraint existsbull Increases CC
bull Accesses the constraint between ViVj
bull Verifies if (ab)isinRViVj
ndash Returns true if (ab) is consistent
ndash Returns false if (ab) is not consistent
ndash If no constraints existbull Returns true (universal constraint)
bull Pseudo code necessarybull Complexity cost in practice
12
Problem Solving with Constraints
Arc Consistency
SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)
1 support nil
2 b DVj
3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩
4 Then Begin
5 support true
6 RETURN support
7 End
8 RETURN support
bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation
13
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 8
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 9
Arc Consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj
bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Problem Solving with Constraints
Arc Consistency
Main Functionsbull Variable-value pair
ndash (Via) ⟨Via variable value from its domain⟩
bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise
bull SUPPORTED(⟨Via ⟩ Vj)
ndash Verifies that ⟨Via has at least one support in ⟩ CViVj
ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo
bull REVISE(ViVj)
ndash Updates DVi given RViVj
ndash BREAK when domain wipe-out
10
Problem Solving with Constraints
Arc Consistency
Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms
ndash Update all domains given all constraints
ndash Call REVISE
ndash Differ in how they manage updates (queue)
bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of
graph deg
bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation
is accessed
ndash Is a cost measure to compare algorithmsrsquo performance in practive
bull Assumptionndash Domains are sorted alphabetically (lexicographic)
11
Problem Solving with Constraints
Arc Consistency
CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation
ndash Verifies whether a constraint exists between ViVj
ndash If a constraint existsbull Increases CC
bull Accesses the constraint between ViVj
bull Verifies if (ab)isinRViVj
ndash Returns true if (ab) is consistent
ndash Returns false if (ab) is not consistent
ndash If no constraints existbull Returns true (universal constraint)
bull Pseudo code necessarybull Complexity cost in practice
12
Problem Solving with Constraints
Arc Consistency
SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)
1 support nil
2 b DVj
3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩
4 Then Begin
5 support true
6 RETURN support
7 End
8 RETURN support
bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation
13
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 9
Arc Consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj
bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Problem Solving with Constraints
Arc Consistency
Main Functionsbull Variable-value pair
ndash (Via) ⟨Via variable value from its domain⟩
bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise
bull SUPPORTED(⟨Via ⟩ Vj)
ndash Verifies that ⟨Via has at least one support in ⟩ CViVj
ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo
bull REVISE(ViVj)
ndash Updates DVi given RViVj
ndash BREAK when domain wipe-out
10
Problem Solving with Constraints
Arc Consistency
Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms
ndash Update all domains given all constraints
ndash Call REVISE
ndash Differ in how they manage updates (queue)
bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of
graph deg
bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation
is accessed
ndash Is a cost measure to compare algorithmsrsquo performance in practive
bull Assumptionndash Domains are sorted alphabetically (lexicographic)
11
Problem Solving with Constraints
Arc Consistency
CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation
ndash Verifies whether a constraint exists between ViVj
ndash If a constraint existsbull Increases CC
bull Accesses the constraint between ViVj
bull Verifies if (ab)isinRViVj
ndash Returns true if (ab) is consistent
ndash Returns false if (ab) is not consistent
ndash If no constraints existbull Returns true (universal constraint)
bull Pseudo code necessarybull Complexity cost in practice
12
Problem Solving with Constraints
Arc Consistency
SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)
1 support nil
2 b DVj
3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩
4 Then Begin
5 support true
6 RETURN support
7 End
8 RETURN support
bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation
13
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
Main Functionsbull Variable-value pair
ndash (Via) ⟨Via variable value from its domain⟩
bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise
bull SUPPORTED(⟨Via ⟩ Vj)
ndash Verifies that ⟨Via has at least one support in ⟩ CViVj
ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo
bull REVISE(ViVj)
ndash Updates DVi given RViVj
ndash BREAK when domain wipe-out
10
Problem Solving with Constraints
Arc Consistency
Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms
ndash Update all domains given all constraints
ndash Call REVISE
ndash Differ in how they manage updates (queue)
bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of
graph deg
bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation
is accessed
ndash Is a cost measure to compare algorithmsrsquo performance in practive
bull Assumptionndash Domains are sorted alphabetically (lexicographic)
11
Problem Solving with Constraints
Arc Consistency
CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation
ndash Verifies whether a constraint exists between ViVj
ndash If a constraint existsbull Increases CC
bull Accesses the constraint between ViVj
bull Verifies if (ab)isinRViVj
ndash Returns true if (ab) is consistent
ndash Returns false if (ab) is not consistent
ndash If no constraints existbull Returns true (universal constraint)
bull Pseudo code necessarybull Complexity cost in practice
12
Problem Solving with Constraints
Arc Consistency
SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)
1 support nil
2 b DVj
3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩
4 Then Begin
5 support true
6 RETURN support
7 End
8 RETURN support
bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation
13
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms
ndash Update all domains given all constraints
ndash Call REVISE
ndash Differ in how they manage updates (queue)
bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of
graph deg
bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation
is accessed
ndash Is a cost measure to compare algorithmsrsquo performance in practive
bull Assumptionndash Domains are sorted alphabetically (lexicographic)
11
Problem Solving with Constraints
Arc Consistency
CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation
ndash Verifies whether a constraint exists between ViVj
ndash If a constraint existsbull Increases CC
bull Accesses the constraint between ViVj
bull Verifies if (ab)isinRViVj
ndash Returns true if (ab) is consistent
ndash Returns false if (ab) is not consistent
ndash If no constraints existbull Returns true (universal constraint)
bull Pseudo code necessarybull Complexity cost in practice
12
Problem Solving with Constraints
Arc Consistency
SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)
1 support nil
2 b DVj
3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩
4 Then Begin
5 support true
6 RETURN support
7 End
8 RETURN support
bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation
13
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation
ndash Verifies whether a constraint exists between ViVj
ndash If a constraint existsbull Increases CC
bull Accesses the constraint between ViVj
bull Verifies if (ab)isinRViVj
ndash Returns true if (ab) is consistent
ndash Returns false if (ab) is not consistent
ndash If no constraints existbull Returns true (universal constraint)
bull Pseudo code necessarybull Complexity cost in practice
12
Problem Solving with Constraints
Arc Consistency
SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)
1 support nil
2 b DVj
3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩
4 Then Begin
5 support true
6 RETURN support
7 End
8 RETURN support
bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation
13
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)
1 support nil
2 b DVj
3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩
4 Then Begin
5 support true
6 RETURN support
7 End
8 RETURN support
bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation
13
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Description
bull Function
bull Updates DVi given the constraint CViVj
bull Is directional does not update DVj
bull Calls SUPPORTED(⟨Vi ⟩ Vj)
bull If DVi is modified
ndash Returns true false otherwise
14
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)
1 revised false
2 x DVi
3 found SUPPORTED(⟨Vix ⟩ Vj)
4 If found = false
5 Then Begin
6 revised true
7 DVi DVi x
8 End
9 RETURN revised
1 Complexity
15
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 16
REVISE example R Dechter
bull REVISE(ViVj)
bull REVISE(ViVj)
1
2 2
3
1
2
3
1
2
3
Vi Vj Vi Vj
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 17
Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
Warningbull Most papers do not check for domain wipe-out
bull In your code have AC1
ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs
bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles
18
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 19
Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)
Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)
2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 20
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 21
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 22
NC amp AC Example courtesy of M Fromherz
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 23
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 24
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 25
AC-3AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 26
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (Ij) arcs(G) i j
4 While Q is not empty do
5 begin
bull select and delete any arc (km) Q
7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 27
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 28
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 29
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 30
AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 31
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
MM
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 32
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 33
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
AC2001
bull When checking (Via) against Vj
ndash Keep track of the Vj value that supports (Via)
ndash Last((Via)Vi)
bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj
bull Big savings
34
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 35
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 36
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Performance comparisonndash Criteria and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 37
CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 38
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes
2 Counting CC and NV (theoretical empirical)
3 Measuring CPU time (empirical)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 39
Performance comparison Courtesy of Lin XU
AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time
AC-4
AC-7
AC-3
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 40
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
Performance comparison Courtesy of Shant
41
AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
AC-what
Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]
42
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 43
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 44
WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz
Algorithm does not solve the problem
A=2 B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types
ndash domain types
1 2 3 2 3 4
BA BA
2 3 2 3
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 45
Outline1 Motivation (done) amp background
2 Node consistency and its complexity
3 Arc consistency and its complexity
4 Criteria for performance comparison and CSP parameters
5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 46
bull Various models exist (use Model B)ndash Models A B C E F etc
bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |
ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax
bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)
Empirical evaluations random problems
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 47
1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of
2 nodes4 Choose e elements from above list as constraints to
between the n nodes5 If the graph is not connected throw away go back to
step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2
values7 For each constraint choose randomly a number of
tuples from the list to guarantee tightness t for the constraint
Model B
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 48
Cos
t of
sol
ving
Mostly solvable problems
Mostly un-solvable problems
Order parameterCritical value of order parameter
bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition
Phase transition [Cheeseman et al lsquo91]
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 49
bull Fix n a p1 and ndash Vary t in 01 02 hellip09
bull Fix n a t and ndash Vary p1 in 01 02 hellip09
bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances
ndash Store all instances
bull Make measurementsndash CC CPU time (in other contexts NV messages
etc)
Tests
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
Comparing two algorithms A1 and A2
bull Store all measurements in Excelbull Use Excel R SAS etc for statistical
measurementsbull Use the t-test paired test
bull Comparing measurementsndash A1 A2 a significantly different
bull Comparing ln measurementsndash A1is significantly better than A2
For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip
CC ln(CC)
A1 A2 A1 A2
i1 100 200 hellip hellip
i2 hellip
i3
hellip
i50
50
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
t-test in Excel
bull Using ln values
ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)
ndash t tinv(pdf)bull degree of freedom = instances ndash 2
51
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
t-test with 95 confidencebull One-tailed test
ndash Interested in direction of change
ndash When t gt 1645 A1 is larger than A2
ndash When t -1645 A2 is larger than A1
ndash When -1645 t 1645 A1 and A2 do not differ significantly
ndash |t|=1645 corresponds to p=005 for a one-tailed test
bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test
ndash When t gt 196 A1 is larger than A2
ndash When t -196 A2 is larger than A1
ndash When -196 t 196 A1 and A2 do not differ significantly
ndash |t|=196 corresponds to p=005 for a two-tailed test
bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court
52
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
Computing the 95 confidence interval
bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance
bull We usually use the t-testbull Assumptions
Normal distribution of data
Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)
Equality of variances
Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population
53
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
Alternatives to the t testbull To relax the normality assumption a non-parametric
alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the
Wilcoxon signed-rank test
bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed
bull To test the equality of the means of two normal populations with known variance a Z-test can be performed
54
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency
Alertsbull For choosing the value of t in general check
httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis
ndash consult the Help Desk of the Department of Statistics at UNL
ndash held at least twice a week at Avery Hall
bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine
55
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)
Problem Solving with Constraints
Arc Consistency 56
Summary1 Alert
ndash Do not confuse a consistency property with the algorithms for reinforcing it
ndash For each property many algorithms may exist
2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)