Principles of Intelligent SystemsConstraint Satisfaction + Local Search

Written by Dr John ThorntonSchool of IT, Griffith University Gold Coast

Outline

The Constraint Satisfaction Paradigm Variables, Domains and Constraints Example Problem Domains Constraint Satisfaction Algorithms

Local Search Algorithms Summary

The Constraint Satisfaction Paradigm

Knowledge represented by constraints What is allowed or disallowed.

Transforms broad range of problems into a computable form

Standard representation means general purpose algorithms can be used solvers ‘know’ how to solve problems issue becomes how to model a problem

Variables, Domains and Constraints

A Constraint Satisfaction Problem (CSP) is: a set of variables each with a set of domain of values, and a set of constraints that define which

combinations of variable values are allowed the task is to find a domain value for each

variable such that all constraints are simultaneously satisfied

Problem Domains

N-Queens Satisfiability Problems Binary Constraint Satisfaction Nurse Rostering University Timetabling

Local Search Algorithms

Getting ‘stuck’ Variable and Constraint-based Search Random Walk Tabu Search

NOVELTY and RNOVELTY Constraint Weighting

Satisfiability Problems

Conjunctive Normal Form: Example {x y z} {x y z} {x y} {y z}

To Solve: All clauses evaluate true e.g.: x = false, y = true, z = true

As a CSP: Each clause is a constraint, each literal is a

variable with a {true, false} domain

Predicate to Conjunctive Form

Predicate: (X) (dog (X) animal (X)) Clause: ( dog(X) animal(X)) Clause is false only if dog(X) is false and

animal(X) is false, i.e. if X is a dog and X is not an animal - in other words if X is a dog then X is an animal

Binary Constraint Satisfaction

Two variable constraint representation Non-binary to binary transformation Phase transition:

p2crit = 1 - m-2/p1(n - 1)

where p1 = constraint density

p2 = constraint tightness

m = uniform domain size

n = number of variables

Nurse Scheduling

Binary Constraint Representation

0 1 0 0 1

1 1 1 0 1

1 0 0 1 0

0 0 1 1 0

Variable 1

Variable 2

0 1 2 3 4

0

1

2

3

Random Walk Local Search

if randomly generated probability < p: Take best cost local move

else: Take randomly selected local move

Tabu Search

if cost(best local move) < best cost yet: Take best local move

else Take best cost local move from the set of

domain values that have not been used in the last n moves

The Eight Queens Problem

Variable

Constraint

Domain

Constraint Satisfaction Algorithms

Backtracking Algorithm 8-Queens Example 8-Queens Performance

Local Search Algorithm 8-Queens Example 8-Queens Performance

Summary

Constraint Satisfaction practical knowledge representation allows general purpose approaches

Algorithms Backtracking Local Search

Backtracking Algorithm

V = list of all variables vi (i = 1 to n)U = emptyTryVariable()

select next vi from V and move to Ufor each domain value di of vi

if di satisfies all constraints with UValue(vi) = di if(V is empty) SaveSolution()else TryVariable()

move vi from U back to V

Basic Local Search Algorithm

assign a domain value di to each variable vi

while no solution and not stuck and not timed outbestCost ; bestList ;for each variable vi | Cost(Value(vi)) > 0

for each domain value di of vi

if Cost(di) < bestCost bestCost Cost(di); bestList di;

else if Cost(di) = bestCost bestList bestList di

Take a randomly selected move from bestList

Backtracking Performance

0

1000

2000

3000

4000

5000

0 4 8 12 16 20 24 28 32

Number of Queens

Tim

e in

sec

onds

Local Search Performance

0

500

1000

1500

2000

2500

0 5000 10000 15000 20000

Number of Queens

Tim

e in

sec

onds

Local Search Diagram

Eight Queens using Backtracking

Try Queen 1Try Queen 2Try Queen 3Try Queen 4Try Queen 5

Stuck!

Undo move

for Queen 5Try next value

for Queen 5

Still Stuck

Undo move

for Queen 5

no move left

Backtrack and

undo last move

for Queen 4

Try next value

for Queen 4

Try Queen 5Try Queen 6Try Queen 7

Stuck AgainUndo move

for Queen 7

and so on...

Place 8 Queens

randomly on

the board

Eight Queens using Local Search

Pick a Queen:

Calculate cost

of each move3 1 054 111

Take least cost

move then try

another Queen

0 4 4 41 1 1 1

4 3 41 1 1 1 31

3 3 3 21 1 1 12

3 4 41 1 1 1 32

2 2 3 42 2 2 12

3 2 32 1 1 2 31

2 0 4 21 2 2 31

2 3 22 1 3 2 31

2 3 3 21 2 2 21

2 3 2 32 2 1 31

2 2 32 1 3 2 11

3 2 23 3 3 3 01

Answer Found

Variable/Constraint-based Search

Variable-based search: Randomly select variable in constraint violation Try all domain values for that variable

Constraint-based search: Randomly select a violated constraint Try all domain values for all variables in the

constraint that improve the constraint

NOVELTY

select best and second best cost moves that improve a violated constraint

if best cost move not most recent or p >= random value select best cost move

else select second best cost move

RNOVELTY

Same as NOVELTY except when best cost move is also most recent, then :

if second best cost - best cost <= c or p < random value select second best cost move

else select best cost move

Constraint Weighting

Add weight to all constraints violated at a local minimum: graph colouring example

Use these weights to calculate cost of subsequent moves

Adding weights changes shape of cost surface: “fills in” the minimum

Constraint Weighting Example

Graph Colouring

a green

b green

c red

d green

cab = 2w

cac = 0 ccd = 0

cbd = w

cbc = 0 a green

b red

c red

d green

cab = 0

cac = 0 ccd = 0

cbd = 0

cbc = wa green

b green

c red

d red

cab = w

cac = 0 ccd = w

cbd = 0

cbc = 0

(a) Local minimum, cost 2w (b) After Weighting, cost 3w (c) Final solution, cost w