problem solving with constraints csce421/821, spring 2011 cse.unl/~ choueiry/f11-496-896

56
Problem Solving with Constraints Arc Consistency 1 Problem Solving With Constraints CSCE421/821, Spring 2011 www.cse.unl.edu/~ choueiry/F11-496-896 All questions: [email protected] Berthe Y. Choueiry (Shu-we-ri) Avery Hall, Room 360 Tel: +1(402)472-5444 Arc Consistency

Upload: august-elliott

Post on 30-Dec-2015

27 views

Category:

Documents


0 download

DESCRIPTION

Arc Consistency. Problem Solving With Constraints CSCE421/821, Spring 2011 www.cse.unl.edu/~ choueiry/F11-496-896 All questions: [email protected] Berthe Y. Choueiry (Shu-we-ri) Avery Hall, Room 360 Tel: +1(402)472-5444. Lecture Sources. Required reading - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 1

Problem Solving With ConstraintsCSCE421821 Spring 2011

wwwcseunledu~choueiryF11-496-896All questions cse496cpcseunledu

Berthe Y Choueiry (Shu-we-ri)Avery Hall Room 360Tel +1(402)472-5444

Arc Consistency

Problem Solving with Constraints

Arc Consistency 2

Lecture SourcesRequired reading1 Algorithms for Constraint Satisfaction Problems Mackworth and

Freuder AIJ85 (focus on Node consistency arc consistency)2 Sections 31 amp 32 Chapter 3 Constraint Processing Dechter

Recommended reading1 Bartak Consistency Techniques (link)2 Constraint Propagation with Interval Labels Davis AIJ87

Problem Solving with Constraints

Arc Consistency 3

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 4

Basic Consistency Notation

Examining finite binary CSPs

Notation

Given variables ijk with values xyz the predicate

Pijk(xyz) is true iff the 3-tuple (xyz) Rijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Problem Solving with Constraints

Arc Consistency 5

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Problem Solving with Constraints

Arc Consistency 6

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experiments1 Random CSPs (Model B)

2 Statistical Analysis

Problem Solving with Constraints

Arc Consistency 7

Node consistency (NC)Procedure NC(V1V2hellipVn)

For i 1 until n do DVi DVi x | Pi(x)

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation

If Dvi (Di= ) empty Then BREAK

Problem Solving with Constraints

Arc Consistency 8

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 9

Arc Consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj

bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Problem Solving with Constraints

Arc Consistency

Main Functionsbull Variable-value pair

ndash (Via) ⟨Via variable value from its domain⟩

bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise

bull SUPPORTED(⟨Via ⟩ Vj)

ndash Verifies that ⟨Via has at least one support in ⟩ CViVj

ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo

bull REVISE(ViVj)

ndash Updates DVi given RViVj

ndash BREAK when domain wipe-out

10

Problem Solving with Constraints

Arc Consistency

Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms

ndash Update all domains given all constraints

ndash Call REVISE

ndash Differ in how they manage updates (queue)

bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of

graph deg

bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation

is accessed

ndash Is a cost measure to compare algorithmsrsquo performance in practive

bull Assumptionndash Domains are sorted alphabetically (lexicographic)

11

Problem Solving with Constraints

Arc Consistency

CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation

ndash Verifies whether a constraint exists between ViVj

ndash If a constraint existsbull Increases CC

bull Accesses the constraint between ViVj

bull Verifies if (ab)isinRViVj

ndash Returns true if (ab) is consistent

ndash Returns false if (ab) is not consistent

ndash If no constraints existbull Returns true (universal constraint)

bull Pseudo code necessarybull Complexity cost in practice

12

Problem Solving with Constraints

Arc Consistency

SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)

1 support nil

2 b DVj

3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩

4 Then Begin

5 support true

6 RETURN support

7 End

8 RETURN support

bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation

13

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 2: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 2

Lecture SourcesRequired reading1 Algorithms for Constraint Satisfaction Problems Mackworth and

Freuder AIJ85 (focus on Node consistency arc consistency)2 Sections 31 amp 32 Chapter 3 Constraint Processing Dechter

Recommended reading1 Bartak Consistency Techniques (link)2 Constraint Propagation with Interval Labels Davis AIJ87

Problem Solving with Constraints

Arc Consistency 3

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 4

Basic Consistency Notation

Examining finite binary CSPs

Notation

Given variables ijk with values xyz the predicate

Pijk(xyz) is true iff the 3-tuple (xyz) Rijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Problem Solving with Constraints

Arc Consistency 5

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Problem Solving with Constraints

Arc Consistency 6

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experiments1 Random CSPs (Model B)

2 Statistical Analysis

Problem Solving with Constraints

Arc Consistency 7

Node consistency (NC)Procedure NC(V1V2hellipVn)

For i 1 until n do DVi DVi x | Pi(x)

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation

If Dvi (Di= ) empty Then BREAK

Problem Solving with Constraints

Arc Consistency 8

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 9

Arc Consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj

bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Problem Solving with Constraints

Arc Consistency

Main Functionsbull Variable-value pair

ndash (Via) ⟨Via variable value from its domain⟩

bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise

bull SUPPORTED(⟨Via ⟩ Vj)

ndash Verifies that ⟨Via has at least one support in ⟩ CViVj

ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo

bull REVISE(ViVj)

ndash Updates DVi given RViVj

ndash BREAK when domain wipe-out

10

Problem Solving with Constraints

Arc Consistency

Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms

ndash Update all domains given all constraints

ndash Call REVISE

ndash Differ in how they manage updates (queue)

bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of

graph deg

bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation

is accessed

ndash Is a cost measure to compare algorithmsrsquo performance in practive

bull Assumptionndash Domains are sorted alphabetically (lexicographic)

11

Problem Solving with Constraints

Arc Consistency

CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation

ndash Verifies whether a constraint exists between ViVj

ndash If a constraint existsbull Increases CC

bull Accesses the constraint between ViVj

bull Verifies if (ab)isinRViVj

ndash Returns true if (ab) is consistent

ndash Returns false if (ab) is not consistent

ndash If no constraints existbull Returns true (universal constraint)

bull Pseudo code necessarybull Complexity cost in practice

12

Problem Solving with Constraints

Arc Consistency

SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)

1 support nil

2 b DVj

3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩

4 Then Begin

5 support true

6 RETURN support

7 End

8 RETURN support

bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation

13

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 3: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 3

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 4

Basic Consistency Notation

Examining finite binary CSPs

Notation

Given variables ijk with values xyz the predicate

Pijk(xyz) is true iff the 3-tuple (xyz) Rijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Problem Solving with Constraints

Arc Consistency 5

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Problem Solving with Constraints

Arc Consistency 6

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experiments1 Random CSPs (Model B)

2 Statistical Analysis

Problem Solving with Constraints

Arc Consistency 7

Node consistency (NC)Procedure NC(V1V2hellipVn)

For i 1 until n do DVi DVi x | Pi(x)

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation

If Dvi (Di= ) empty Then BREAK

Problem Solving with Constraints

Arc Consistency 8

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 9

Arc Consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj

bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Problem Solving with Constraints

Arc Consistency

Main Functionsbull Variable-value pair

ndash (Via) ⟨Via variable value from its domain⟩

bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise

bull SUPPORTED(⟨Via ⟩ Vj)

ndash Verifies that ⟨Via has at least one support in ⟩ CViVj

ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo

bull REVISE(ViVj)

ndash Updates DVi given RViVj

ndash BREAK when domain wipe-out

10

Problem Solving with Constraints

Arc Consistency

Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms

ndash Update all domains given all constraints

ndash Call REVISE

ndash Differ in how they manage updates (queue)

bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of

graph deg

bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation

is accessed

ndash Is a cost measure to compare algorithmsrsquo performance in practive

bull Assumptionndash Domains are sorted alphabetically (lexicographic)

11

Problem Solving with Constraints

Arc Consistency

CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation

ndash Verifies whether a constraint exists between ViVj

ndash If a constraint existsbull Increases CC

bull Accesses the constraint between ViVj

bull Verifies if (ab)isinRViVj

ndash Returns true if (ab) is consistent

ndash Returns false if (ab) is not consistent

ndash If no constraints existbull Returns true (universal constraint)

bull Pseudo code necessarybull Complexity cost in practice

12

Problem Solving with Constraints

Arc Consistency

SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)

1 support nil

2 b DVj

3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩

4 Then Begin

5 support true

6 RETURN support

7 End

8 RETURN support

bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation

13

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 4: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 4

Basic Consistency Notation

Examining finite binary CSPs

Notation

Given variables ijk with values xyz the predicate

Pijk(xyz) is true iff the 3-tuple (xyz) Rijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Problem Solving with Constraints

Arc Consistency 5

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Problem Solving with Constraints

Arc Consistency 6

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experiments1 Random CSPs (Model B)

2 Statistical Analysis

Problem Solving with Constraints

Arc Consistency 7

Node consistency (NC)Procedure NC(V1V2hellipVn)

For i 1 until n do DVi DVi x | Pi(x)

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation

If Dvi (Di= ) empty Then BREAK

Problem Solving with Constraints

Arc Consistency 8

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 9

Arc Consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj

bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Problem Solving with Constraints

Arc Consistency

Main Functionsbull Variable-value pair

ndash (Via) ⟨Via variable value from its domain⟩

bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise

bull SUPPORTED(⟨Via ⟩ Vj)

ndash Verifies that ⟨Via has at least one support in ⟩ CViVj

ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo

bull REVISE(ViVj)

ndash Updates DVi given RViVj

ndash BREAK when domain wipe-out

10

Problem Solving with Constraints

Arc Consistency

Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms

ndash Update all domains given all constraints

ndash Call REVISE

ndash Differ in how they manage updates (queue)

bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of

graph deg

bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation

is accessed

ndash Is a cost measure to compare algorithmsrsquo performance in practive

bull Assumptionndash Domains are sorted alphabetically (lexicographic)

11

Problem Solving with Constraints

Arc Consistency

CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation

ndash Verifies whether a constraint exists between ViVj

ndash If a constraint existsbull Increases CC

bull Accesses the constraint between ViVj

bull Verifies if (ab)isinRViVj

ndash Returns true if (ab) is consistent

ndash Returns false if (ab) is not consistent

ndash If no constraints existbull Returns true (universal constraint)

bull Pseudo code necessarybull Complexity cost in practice

12

Problem Solving with Constraints

Arc Consistency

SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)

1 support nil

2 b DVj

3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩

4 Then Begin

5 support true

6 RETURN support

7 End

8 RETURN support

bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation

13

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 5: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 5

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Problem Solving with Constraints

Arc Consistency 6

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experiments1 Random CSPs (Model B)

2 Statistical Analysis

Problem Solving with Constraints

Arc Consistency 7

Node consistency (NC)Procedure NC(V1V2hellipVn)

For i 1 until n do DVi DVi x | Pi(x)

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation

If Dvi (Di= ) empty Then BREAK

Problem Solving with Constraints

Arc Consistency 8

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 9

Arc Consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj

bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Problem Solving with Constraints

Arc Consistency

Main Functionsbull Variable-value pair

ndash (Via) ⟨Via variable value from its domain⟩

bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise

bull SUPPORTED(⟨Via ⟩ Vj)

ndash Verifies that ⟨Via has at least one support in ⟩ CViVj

ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo

bull REVISE(ViVj)

ndash Updates DVi given RViVj

ndash BREAK when domain wipe-out

10

Problem Solving with Constraints

Arc Consistency

Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms

ndash Update all domains given all constraints

ndash Call REVISE

ndash Differ in how they manage updates (queue)

bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of

graph deg

bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation

is accessed

ndash Is a cost measure to compare algorithmsrsquo performance in practive

bull Assumptionndash Domains are sorted alphabetically (lexicographic)

11

Problem Solving with Constraints

Arc Consistency

CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation

ndash Verifies whether a constraint exists between ViVj

ndash If a constraint existsbull Increases CC

bull Accesses the constraint between ViVj

bull Verifies if (ab)isinRViVj

ndash Returns true if (ab) is consistent

ndash Returns false if (ab) is not consistent

ndash If no constraints existbull Returns true (universal constraint)

bull Pseudo code necessarybull Complexity cost in practice

12

Problem Solving with Constraints

Arc Consistency

SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)

1 support nil

2 b DVj

3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩

4 Then Begin

5 support true

6 RETURN support

7 End

8 RETURN support

bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation

13

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 6: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 6

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experiments1 Random CSPs (Model B)

2 Statistical Analysis

Problem Solving with Constraints

Arc Consistency 7

Node consistency (NC)Procedure NC(V1V2hellipVn)

For i 1 until n do DVi DVi x | Pi(x)

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation

If Dvi (Di= ) empty Then BREAK

Problem Solving with Constraints

Arc Consistency 8

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 9

Arc Consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj

bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Problem Solving with Constraints

Arc Consistency

Main Functionsbull Variable-value pair

ndash (Via) ⟨Via variable value from its domain⟩

bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise

bull SUPPORTED(⟨Via ⟩ Vj)

ndash Verifies that ⟨Via has at least one support in ⟩ CViVj

ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo

bull REVISE(ViVj)

ndash Updates DVi given RViVj

ndash BREAK when domain wipe-out

10

Problem Solving with Constraints

Arc Consistency

Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms

ndash Update all domains given all constraints

ndash Call REVISE

ndash Differ in how they manage updates (queue)

bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of

graph deg

bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation

is accessed

ndash Is a cost measure to compare algorithmsrsquo performance in practive

bull Assumptionndash Domains are sorted alphabetically (lexicographic)

11

Problem Solving with Constraints

Arc Consistency

CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation

ndash Verifies whether a constraint exists between ViVj

ndash If a constraint existsbull Increases CC

bull Accesses the constraint between ViVj

bull Verifies if (ab)isinRViVj

ndash Returns true if (ab) is consistent

ndash Returns false if (ab) is not consistent

ndash If no constraints existbull Returns true (universal constraint)

bull Pseudo code necessarybull Complexity cost in practice

12

Problem Solving with Constraints

Arc Consistency

SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)

1 support nil

2 b DVj

3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩

4 Then Begin

5 support true

6 RETURN support

7 End

8 RETURN support

bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation

13

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 7: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 7

Node consistency (NC)Procedure NC(V1V2hellipVn)

For i 1 until n do DVi DVi x | Pi(x)

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)bull Alert check for domain wipe-out domain annihilation

If Dvi (Di= ) empty Then BREAK

Problem Solving with Constraints

Arc Consistency 8

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 9

Arc Consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj

bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Problem Solving with Constraints

Arc Consistency

Main Functionsbull Variable-value pair

ndash (Via) ⟨Via variable value from its domain⟩

bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise

bull SUPPORTED(⟨Via ⟩ Vj)

ndash Verifies that ⟨Via has at least one support in ⟩ CViVj

ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo

bull REVISE(ViVj)

ndash Updates DVi given RViVj

ndash BREAK when domain wipe-out

10

Problem Solving with Constraints

Arc Consistency

Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms

ndash Update all domains given all constraints

ndash Call REVISE

ndash Differ in how they manage updates (queue)

bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of

graph deg

bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation

is accessed

ndash Is a cost measure to compare algorithmsrsquo performance in practive

bull Assumptionndash Domains are sorted alphabetically (lexicographic)

11

Problem Solving with Constraints

Arc Consistency

CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation

ndash Verifies whether a constraint exists between ViVj

ndash If a constraint existsbull Increases CC

bull Accesses the constraint between ViVj

bull Verifies if (ab)isinRViVj

ndash Returns true if (ab) is consistent

ndash Returns false if (ab) is not consistent

ndash If no constraints existbull Returns true (universal constraint)

bull Pseudo code necessarybull Complexity cost in practice

12

Problem Solving with Constraints

Arc Consistency

SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)

1 support nil

2 b DVj

3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩

4 Then Begin

5 support true

6 RETURN support

7 End

8 RETURN support

bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation

13

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 8: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 8

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexityndash Algorithms AC1 AC3 AC4 AC2001

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 9

Arc Consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj

bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Problem Solving with Constraints

Arc Consistency

Main Functionsbull Variable-value pair

ndash (Via) ⟨Via variable value from its domain⟩

bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise

bull SUPPORTED(⟨Via ⟩ Vj)

ndash Verifies that ⟨Via has at least one support in ⟩ CViVj

ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo

bull REVISE(ViVj)

ndash Updates DVi given RViVj

ndash BREAK when domain wipe-out

10

Problem Solving with Constraints

Arc Consistency

Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms

ndash Update all domains given all constraints

ndash Call REVISE

ndash Differ in how they manage updates (queue)

bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of

graph deg

bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation

is accessed

ndash Is a cost measure to compare algorithmsrsquo performance in practive

bull Assumptionndash Domains are sorted alphabetically (lexicographic)

11

Problem Solving with Constraints

Arc Consistency

CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation

ndash Verifies whether a constraint exists between ViVj

ndash If a constraint existsbull Increases CC

bull Accesses the constraint between ViVj

bull Verifies if (ab)isinRViVj

ndash Returns true if (ab) is consistent

ndash Returns false if (ab) is not consistent

ndash If no constraints existbull Returns true (universal constraint)

bull Pseudo code necessarybull Complexity cost in practice

12

Problem Solving with Constraints

Arc Consistency

SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)

1 support nil

2 b DVj

3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩

4 Then Begin

5 support true

6 RETURN support

7 End

8 RETURN support

bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation

13

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 9: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 9

Arc Consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc consistent relative to Vj iff for every value a DVi there exists b DVj | (ab) RViVj

bull The constraint CViVj is arc consistent iff ndash Vi is arc consistent relative to Vj and ndash Vj is arc consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Problem Solving with Constraints

Arc Consistency

Main Functionsbull Variable-value pair

ndash (Via) ⟨Via variable value from its domain⟩

bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise

bull SUPPORTED(⟨Via ⟩ Vj)

ndash Verifies that ⟨Via has at least one support in ⟩ CViVj

ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo

bull REVISE(ViVj)

ndash Updates DVi given RViVj

ndash BREAK when domain wipe-out

10

Problem Solving with Constraints

Arc Consistency

Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms

ndash Update all domains given all constraints

ndash Call REVISE

ndash Differ in how they manage updates (queue)

bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of

graph deg

bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation

is accessed

ndash Is a cost measure to compare algorithmsrsquo performance in practive

bull Assumptionndash Domains are sorted alphabetically (lexicographic)

11

Problem Solving with Constraints

Arc Consistency

CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation

ndash Verifies whether a constraint exists between ViVj

ndash If a constraint existsbull Increases CC

bull Accesses the constraint between ViVj

bull Verifies if (ab)isinRViVj

ndash Returns true if (ab) is consistent

ndash Returns false if (ab) is not consistent

ndash If no constraints existbull Returns true (universal constraint)

bull Pseudo code necessarybull Complexity cost in practice

12

Problem Solving with Constraints

Arc Consistency

SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)

1 support nil

2 b DVj

3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩

4 Then Begin

5 support true

6 RETURN support

7 End

8 RETURN support

bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation

13

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 10: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

Main Functionsbull Variable-value pair

ndash (Via) ⟨Via variable value from its domain⟩

bull CHECK(⟨Via ⟩ ⟨Vjb )⟩ndash Returns true if (ab)isinRViVj false otherwise

bull SUPPORTED(⟨Via ⟩ Vj)

ndash Verifies that ⟨Via has at least one support in ⟩ CViVj

ndash Is not standard lsquoterminologyrsquo but my lsquoconventionrsquo

bull REVISE(ViVj)

ndash Updates DVi given RViVj

ndash BREAK when domain wipe-out

10

Problem Solving with Constraints

Arc Consistency

Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms

ndash Update all domains given all constraints

ndash Call REVISE

ndash Differ in how they manage updates (queue)

bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of

graph deg

bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation

is accessed

ndash Is a cost measure to compare algorithmsrsquo performance in practive

bull Assumptionndash Domains are sorted alphabetically (lexicographic)

11

Problem Solving with Constraints

Arc Consistency

CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation

ndash Verifies whether a constraint exists between ViVj

ndash If a constraint existsbull Increases CC

bull Accesses the constraint between ViVj

bull Verifies if (ab)isinRViVj

ndash Returns true if (ab) is consistent

ndash Returns false if (ab) is not consistent

ndash If no constraints existbull Returns true (universal constraint)

bull Pseudo code necessarybull Complexity cost in practice

12

Problem Solving with Constraints

Arc Consistency

SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)

1 support nil

2 b DVj

3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩

4 Then Begin

5 support true

6 RETURN support

7 End

8 RETURN support

bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation

13

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 11: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

Arc Consistency Algorithmsbull AC-1 AC-3 hellip AC2001 arc consistency algorithms

ndash Update all domains given all constraints

ndash Call REVISE

ndash Differ in how they manage updates (queue)

bull Complexityndash Nbr of variables n Domain size d Nbr of constraints e degree of

graph deg

bull CC is a global variable ndash Keeps track of the number of times that the definition of a binary relation

is accessed

ndash Is a cost measure to compare algorithmsrsquo performance in practive

bull Assumptionndash Domains are sorted alphabetically (lexicographic)

11

Problem Solving with Constraints

Arc Consistency

CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation

ndash Verifies whether a constraint exists between ViVj

ndash If a constraint existsbull Increases CC

bull Accesses the constraint between ViVj

bull Verifies if (ab)isinRViVj

ndash Returns true if (ab) is consistent

ndash Returns false if (ab) is not consistent

ndash If no constraints existbull Returns true (universal constraint)

bull Pseudo code necessarybull Complexity cost in practice

12

Problem Solving with Constraints

Arc Consistency

SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)

1 support nil

2 b DVj

3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩

4 Then Begin

5 support true

6 RETURN support

7 End

8 RETURN support

bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation

13

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 12: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

CHECK(⟨Via ⟩ ⟨Vjb )⟩bull Operation

ndash Verifies whether a constraint exists between ViVj

ndash If a constraint existsbull Increases CC

bull Accesses the constraint between ViVj

bull Verifies if (ab)isinRViVj

ndash Returns true if (ab) is consistent

ndash Returns false if (ab) is not consistent

ndash If no constraints existbull Returns true (universal constraint)

bull Pseudo code necessarybull Complexity cost in practice

12

Problem Solving with Constraints

Arc Consistency

SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)

1 support nil

2 b DVj

3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩

4 Then Begin

5 support true

6 RETURN support

7 End

8 RETURN support

bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation

13

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 13: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

SUPPORTED(⟨Via ⟩ Vj)SUPPORTED(⟨Via ⟩ Vj)

1 support nil

2 b DVj

3 If CHECK(⟨Via ⟩ ⟨Vib ) ⟩

4 Then Begin

5 support true

6 RETURN support

7 End

8 RETURN support

bull Complexitybull Once you find a support stop lookingbull Cost in practice depends on implementation

13

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 14: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Description

bull Function

bull Updates DVi given the constraint CViVj

bull Is directional does not update DVj

bull Calls SUPPORTED(⟨Vi ⟩ Vj)

bull If DVi is modified

ndash Returns true false otherwise

14

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 15: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

REVISE(Vi Vj) Pseudocode REVISE(Vi Vj)

1 revised false

2 x DVi

3 found SUPPORTED(⟨Vix ⟩ Vj)

4 If found = false

5 Then Begin

6 revised true

7 DVi DVi x

8 End

9 RETURN revised

1 Complexity

15

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 16: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 16

REVISE example R Dechter

bull REVISE(ViVj)

bull REVISE(ViVj)

1

2 2

3

1

2

3

1

2

3

Vi Vj Vi Vj

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 17: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 17

Arc Consistency (AC-1)Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j 3 Repeat4 change false5 Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

6 AC-1 does not update Q the queue of arcs7 No algorithm can have time complexity below O(ea2)8 AC1 should test for empty domains (does not in the paper)

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 18: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

Warningbull Most papers do not check for domain wipe-out

bull In your code have AC1

ndash Always check for domain wipe out andndash terminateinterrupt the algorithm when that occurs

bull Complexity versus performancendash Does not affect worst-case complexityndash In practice saves lots of CC and cycles

18

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 19: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 19

Arc Consistency (AC-1)bull If a domain is wiped out AC1 returns nilbull Otherwise returns the problem given as input (alternatively change)

Procedure AC-11 NC(Problem)2 Q (i j) | (ij) directed arcs in constraint network of Problem i j bull Repeatbull change falsebull Foreach (i j) Q do 6 Begin for each1 updated REVISE(i j)

2 If DVi = Then Return false Else change (updated or change)3 End for each4 Until change = false5 Return Problem

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 20: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 20

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 21: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 21

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 22: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 22

NC amp AC Example courtesy of M Fromherz

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

AC propagates boundUnary constraint xgt3 is imposedAC propagates bounds again

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 23: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 23

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 24: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 24

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 25: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 25

AC-3AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-3bull NC(Problem)bull for i 1 until n do NC(i)bull Q (i j) | (i j) directed arcs in constraint network of Problem i j bull While Q is not empty dobull Beginbull select and delete any arc (km) Qbull If REVISE(km) Then Q Q (Ik) | (Ik) arcs(G) ik im bull End(Donrsquot forget to make AC-3 check for domain wipe out)

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 26: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 26

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (Ij) arcs(G) i j

4 While Q is not empty do

5 begin

bull select and delete any arc (km) Q

7 If Revise(k m) then Q Q (ik) | (Ik) arcs(G) ik im

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 27: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 27

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 28: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 28

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 29: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 29

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 30: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 30

AC-4Mohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceedsndash Generates data structuresndash Prepares data structuresndash Iterates over constraints while updating support in data structures

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 31: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 31

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

MM

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 32: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 32

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 33: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 33

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 34: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

AC2001

bull When checking (Via) against Vj

ndash Keep track of the Vj value that supports (Via)

ndash Last((Via)Vi)

bull Next time when checking again (Via) against Vj we start from Last((Via)Vi) and donrsquot have to traverse again the domain of Vj again or list of tuples in CViVj

bull Big savings

34

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 35: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 35

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001equivAC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]

ndash all-different (dedicated) [Reacutegin 94]

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 36: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 36

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Performance comparisonndash Criteria and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 37: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 37

CSP parameters ⟨na t d⟩bull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 38: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 38

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-case alwaysndash average-case neverndash Best-case sometimes

2 Counting CC and NV (theoretical empirical)

3 Measuring CPU time (empirical)

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 39: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 39

Performance comparison Courtesy of Lin XU

AC-3 AC-7 AC-4 on n=10a=10td=06 displaying CC and CPU time

AC-4

AC-7

AC-3

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 40: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 40

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 41: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

Performance comparison Courtesy of Shant

41

AC3 AC31 AC4 on on n=40a=16td=015 displaying CC and CPU time

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 42: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

AC-what

Instructorrsquos personal opinionbull Used to recommend using AC-3bull Now recommend using AC2001bull Do the project on AC- if you are curious [Reacutegin 03]

42

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 43: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 43

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 44: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 44

WARNINGrarr Completeness (ie for solving the CSP) Running the Waltz

Algorithm does not solve the problem

A=2 B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis characterizes the completeness and quiescence of the Waltz algorithm (see Table 3) in terms ofndash constraint types

ndash domain types

1 2 3 2 3 4

BA BA

2 3 2 3

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 45: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 45

Outline1 Motivation (done) amp background

2 Node consistency and its complexity

3 Arc consistency and its complexity

4 Criteria for performance comparison and CSP parameters

5 Experimentsndash Random CSPs (Model B)ndash Statistical Analysis

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 46: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 46

bull Various models exist (use Model B)ndash Models A B C E F etc

bull Vary parameters ltn a t pgtndash Number of variables nndash Domain size a dndash Constraint tightness t = |forbidden tuples| | all tuples |

ndash Proportion of constraints (aka constraint density constraint probability) p1 = e emax

bull Issues ndash Uniformityndash Difficulty (phase transition)ndash Solvability of instances (for incomplete search techniques)

Empirical evaluations random problems

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 47: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 47

1 Input n a t p12 Generate n nodes3 Generate a list of n(n-1)2 tuples of all combinations of

2 nodes4 Choose e elements from above list as constraints to

between the n nodes5 If the graph is not connected throw away go back to

step 4 else proceed6 Generate a list of a2 tuples of all combinations of 2

values7 For each constraint choose randomly a number of

tuples from the list to guarantee tightness t for the constraint

Model B

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 48: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 48

Cos

t of

sol

ving

Mostly solvable problems

Mostly un-solvable problems

Order parameterCritical value of order parameter

bull Significant increase of cost around critical valuebull In CSPs order parameter is constraint tightness amp ratiobull Algorithms compared around phase transition

Phase transition [Cheeseman et al lsquo91]

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 49: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 49

bull Fix n a p1 and ndash Vary t in 01 02 hellip09

bull Fix n a t and ndash Vary p1 in 01 02 hellip09

bull For each data point (for each value of tp1)ndash Generate (at least) 50 instances

ndash Store all instances

bull Make measurementsndash CC CPU time (in other contexts NV messages

etc)

Tests

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 50: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

Comparing two algorithms A1 and A2

bull Store all measurements in Excelbull Use Excel R SAS etc for statistical

measurementsbull Use the t-test paired test

bull Comparing measurementsndash A1 A2 a significantly different

bull Comparing ln measurementsndash A1is significantly better than A2

For Excel Microsoft button Excel Options Adds in Analysis ToolPak Go check the box for Analysis ToolPak Go Intallhellip

CC ln(CC)

A1 A2 A1 A2

i1 100 200 hellip hellip

i2 hellip

i3

hellip

i50

50

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 51: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

t-test in Excel

bull Using ln values

ndash p ttest(array1array2tailstype)bull tails=1 or 2 bull type1 (paired)

ndash t tinv(pdf)bull degree of freedom = instances ndash 2

51

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 52: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

t-test with 95 confidencebull One-tailed test

ndash Interested in direction of change

ndash When t gt 1645 A1 is larger than A2

ndash When t -1645 A2 is larger than A1

ndash When -1645 t 1645 A1 and A2 do not differ significantly

ndash |t|=1645 corresponds to p=005 for a one-tailed test

bull Two-tailed testndash Although it tells direction not as accurate as the one-tailed test

ndash When t gt 196 A1 is larger than A2

ndash When t -196 A2 is larger than A1

ndash When -196 t 196 A1 and A2 do not differ significantly

ndash |t|=196 corresponds to p=005 for a two-tailed test

bull p=005 is a US Supreme Court ruling any statistical analysis needs to be significant at the 005 level to be admitted in court

52

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 53: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

Computing the 95 confidence interval

bull The t test can be used to test the equality of the means of two normal populations with unknown but equal variance

bull We usually use the t-testbull Assumptions

Normal distribution of data

Sampling distributions of the mean approaches a uniform distribution (holds when instances 30)

Equality of variances

Sampling distribution distribution calculated from all possible samples of a given size drawn from a given population

53

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 54: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

Alternatives to the t testbull To relax the normality assumption a non-parametric

alternative to the t test can be used and the usual choices are ndash for independent samples the Mann-Whitney U testndash for related samples either the binomial test or the

Wilcoxon signed-rank test

bull To test the equality of the means of more than two normal populations an Analysis of Variance can be performed

bull To test the equality of the means of two normal populations with known variance a Z-test can be performed

54

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 55: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency

Alertsbull For choosing the value of t in general check

httpwwwsocruclaeduAppletsdirT-tablehtml bull For a sound statistical analysis

ndash consult the Help Desk of the Department of Statistics at UNL

ndash held at least twice a week at Avery Hall

bull Acknowledgments Dr Makram Geha Department of Statistics UNL All errors are mine

55

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)

Page 56: Problem Solving With Constraints CSCE421/821, Spring 2011 cse.unl/~ choueiry/F11-496-896

Problem Solving with Constraints

Arc Consistency 56

Summary1 Alert

ndash Do not confuse a consistency property with the algorithms for reinforcing it

ndash For each property many algorithms may exist

2 Local consistency methodsndash Remove inconsistent values (node arc consistency)ndash Remove Inconsistent tuples (path consistency)ndash Get us closer to the solutionndash Reduce the lsquosizersquo of the problem amp thrashing during search ndash Are lsquocheaprsquo (ie polynomial time)