Principles of Intelligent SystemsConstraint Satisfaction + Local Search
Written by Dr John ThorntonSchool of IT, Griffith University Gold Coast
Outline
The Constraint Satisfaction Paradigm Variables, Domains and Constraints Example Problem Domains Constraint Satisfaction Algorithms
Local Search Algorithms Summary
The Constraint Satisfaction Paradigm
Knowledge represented by constraints What is allowed or disallowed.
Transforms broad range of problems into a computable form
Standard representation means general purpose algorithms can be used solvers ‘know’ how to solve problems issue becomes how to model a problem
Variables, Domains and Constraints
A Constraint Satisfaction Problem (CSP) is: a set of variables each with a set of domain of values, and a set of constraints that define which
combinations of variable values are allowed the task is to find a domain value for each
variable such that all constraints are simultaneously satisfied
Problem Domains
N-Queens Satisfiability Problems Binary Constraint Satisfaction Nurse Rostering University Timetabling
Local Search Algorithms
Getting ‘stuck’ Variable and Constraint-based Search Random Walk Tabu Search
NOVELTY and RNOVELTY Constraint Weighting
Satisfiability Problems
Conjunctive Normal Form: Example {x y z} {x y z} {x y} {y z}
To Solve: All clauses evaluate true e.g.: x = false, y = true, z = true
As a CSP: Each clause is a constraint, each literal is a
variable with a {true, false} domain
Predicate to Conjunctive Form
Predicate: (X) (dog (X) animal (X)) Clause: ( dog(X) animal(X)) Clause is false only if dog(X) is false and
animal(X) is false, i.e. if X is a dog and X is not an animal - in other words if X is a dog then X is an animal
Binary Constraint Satisfaction
Two variable constraint representation Non-binary to binary transformation Phase transition:
p2crit = 1 - m-2/p1(n - 1)
where p1 = constraint density
p2 = constraint tightness
m = uniform domain size
n = number of variables
Nurse Scheduling
Binary Constraint Representation
0 1 0 0 1
1 1 1 0 1
1 0 0 1 0
0 0 1 1 0
Variable 1
Variable 2
0 1 2 3 4
0
1
2
3
Random Walk Local Search
if randomly generated probability < p: Take best cost local move
else: Take randomly selected local move
Tabu Search
if cost(best local move) < best cost yet: Take best local move
else Take best cost local move from the set of
domain values that have not been used in the last n moves
The Eight Queens Problem
Variable
Constraint
Domain
Constraint Satisfaction Algorithms
Backtracking Algorithm 8-Queens Example 8-Queens Performance
Local Search Algorithm 8-Queens Example 8-Queens Performance
Summary
Constraint Satisfaction practical knowledge representation allows general purpose approaches
Algorithms Backtracking Local Search
Backtracking Algorithm
V = list of all variables vi (i = 1 to n)U = emptyTryVariable()
select next vi from V and move to Ufor each domain value di of vi
if di satisfies all constraints with UValue(vi) = di if(V is empty) SaveSolution()else TryVariable()
move vi from U back to V
Basic Local Search Algorithm
assign a domain value di to each variable vi
while no solution and not stuck and not timed outbestCost ; bestList ;for each variable vi | Cost(Value(vi)) > 0
for each domain value di of vi
if Cost(di) < bestCost bestCost Cost(di); bestList di;
else if Cost(di) = bestCost bestList bestList di
Take a randomly selected move from bestList
Backtracking Performance
0
1000
2000
3000
4000
5000
0 4 8 12 16 20 24 28 32
Number of Queens
Tim
e in
sec
onds
Local Search Performance
0
500
1000
1500
2000
2500
0 5000 10000 15000 20000
Number of Queens
Tim
e in
sec
onds
Local Search Diagram
Eight Queens using Backtracking
Try Queen 1Try Queen 2Try Queen 3Try Queen 4Try Queen 5
Stuck!
Undo move
for Queen 5Try next value
for Queen 5
Still Stuck
Undo move
for Queen 5
no move left
Backtrack and
undo last move
for Queen 4
Try next value
for Queen 4
Try Queen 5Try Queen 6Try Queen 7
Stuck AgainUndo move
for Queen 7
and so on...
Place 8 Queens
randomly on
the board
Eight Queens using Local Search
Pick a Queen:
Calculate cost
of each move3 1 054 111
Take least cost
move then try
another Queen
0 4 4 41 1 1 1
4 3 41 1 1 1 31
3 3 3 21 1 1 12
3 4 41 1 1 1 32
2 2 3 42 2 2 12
3 2 32 1 1 2 31
2 0 4 21 2 2 31
2 3 22 1 3 2 31
2 3 3 21 2 2 21
2 3 2 32 2 1 31
2 2 32 1 3 2 11
3 2 23 3 3 3 01
Answer Found
Variable/Constraint-based Search
Variable-based search: Randomly select variable in constraint violation Try all domain values for that variable
Constraint-based search: Randomly select a violated constraint Try all domain values for all variables in the
constraint that improve the constraint
NOVELTY
select best and second best cost moves that improve a violated constraint
if best cost move not most recent or p >= random value select best cost move
else select second best cost move
RNOVELTY
Same as NOVELTY except when best cost move is also most recent, then :
if second best cost - best cost <= c or p < random value select second best cost move
else select best cost move
Constraint Weighting
Add weight to all constraints violated at a local minimum: graph colouring example
Use these weights to calculate cost of subsequent moves
Adding weights changes shape of cost surface: “fills in” the minimum
Constraint Weighting Example
Graph Colouring
a green
b green
c red
d green
cab = 2w
cac = 0 ccd = 0
cbd = w
cbc = 0 a green
b red
c red
d green
cab = 0
cac = 0 ccd = 0
cbd = 0
cbc = wa green
b green
c red
d red
cab = w
cac = 0 ccd = w
cbd = 0
cbc = 0
(a) Local minimum, cost 2w (b) After Weighting, cost 3w (c) Final solution, cost w