04 search heuristic

Heuristic SearchHeuristic Search

(Where we try to be smarter in how we choose among alternatives)(Where we try to be smarter in how we choose among alternatives)

Dr. Ali Abdallah Dr. Ali Abdallah

([email protected])([email protected])

Search Algorithm Search Algorithm (Modified)(Modified)

1.1. INSERT(initial-node,FRINGE)INSERT(initial-node,FRINGE)

2.2. Repeat:Repeat:

a.a. If empty(FRINGE) then return If empty(FRINGE) then return failurefailure

b.b. nn REMOVE(FRINGE) REMOVE(FRINGE)

c.c. ss STATE( STATE(nn))

d.d. If GOAL?(If GOAL?(ss) then return ) then return path or goal statepath or goal state

e.e. For every state For every state s’s’ in SUCCESSORS( in SUCCESSORS(ss))

i.i. Create a node Create a node n’n’ as a successor of as a successor of nn

ii.ii. INSERT(INSERT(n’n’,FRINGE),FRINGE)

Best-First SearchBest-First Search It exploits state description to estimate It exploits state description to estimate

how promising each search node ishow promising each search node is An An evaluation functionevaluation function f maps each f maps each

search node N to positive real number search node N to positive real number f(N)f(N)

Traditionally, the smaller f(N), the Traditionally, the smaller f(N), the more promising Nmore promising N

Best-first searchBest-first search sorts the fringe in sorts the fringe in increasing fincreasing f [random order is assumed [random order is assumed among nodes with equal values of f]among nodes with equal values of f]

Best-First SearchBest-First Search

“Best” only refers to the value of f,not to the quality of the actual path.Best-first search does not generate optimal paths in general

“Best” only refers to the value of f,not to the quality of the actual path.Best-first search does not generate optimal paths in general

There are no limitations on f. Any There are no limitations on f. Any function of your choice is acceptable. function of your choice is acceptable. But will it help the search algorithm?But will it help the search algorithm?

The classical approach is to construct The classical approach is to construct f(N) as an estimator of the cost of a f(N) as an estimator of the cost of a solution path through Nsolution path through N

How to construct How to construct an evaluation function?an evaluation function?

The The heuristic functionheuristic function h(N) estimates the h(N) estimates the distance of distance of STATESTATE(N) to a goal state (N) to a goal state

Its value is Its value is independent of the current independent of the current search treesearch tree; it depends only on ; it depends only on STATE(N) and the goal testSTATE(N) and the goal test

Example:Example:

hh11(N) = number of misplaced tiles = 6(N) = number of misplaced tiles = 6

Heuristic FunctionHeuristic Function

14

7

5

2

63

8

STATE(N)

64

7

1

5

2

8

3

Goal state

hh11(N) = number of misplaced tiles = 6(N) = number of misplaced tiles = 6

hh22(N) = sum of the (Manhattan) distances of (N) = sum of the (Manhattan) distances of every tile to its goal every tile to its goal

positionposition = 2 + 3 + 0 + 1 + 3 + 0 + 3 + 1 = 13 = 2 + 3 + 0 + 1 + 3 + 0 + 3 + 1 = 13

Other ExamplesOther Examples

14

7

5

2

63

8

STATE(N)

64

7

1

5

2

8

3

Goal state

Other Examples Other Examples (Robot Navigation)(Robot Navigation)

xN

yN

N

xg

yg

2 2g g1 N Nh (N) = (x -x ) +(y -y ) (Euclidean distance)

h2(N) = |xN-xg| + |yN-yg| (Manhattan distance)

Classical Evaluation Classical Evaluation FunctionsFunctions h(N): heuristic functionh(N): heuristic function

[Independent of search tree][Independent of search tree] g(N): cost of the best path found so g(N): cost of the best path found so

far between the initial node and Nfar between the initial node and N[Dependent on search tree][Dependent on search tree]

f(N) = h(N)f(N) = h(N) greedygreedy best-first best-first searchsearch

f(N) = g(N) + h(N)f(N) = g(N) + h(N)

8-Puzzle8-Puzzle

empty

5

5

3

3

4

3 4

4

2 1

2

3

4

3

f(N) = h(N) = number of misplaced tiles

0

4

0+4

1+5

1+5

1+3

3+3

3+4

3+4

3+2 4+1

5+2

5+0

2+3

2+4

2+3

8-Puzzle8-Puzzlef(N) = g(N) + h(N) with h(N) = number of misplaced tiles

5

6

6

4

4

2 1

2

0

5

5

3

8-Puzzle8-Puzzlef(N) = h(N) = distances of tiles to goal

Best-First Best-First Efficiency Efficiency

f(N) = h(N) = straight distance to the goal

Local-minimum problem

Can we Prove Anything?Can we Prove Anything? If the state space is finite and we discard If the state space is finite and we discard

nodes that revisit states, the search is nodes that revisit states, the search is complete, but in general is not optimalcomplete, but in general is not optimal

If the state space is finite and we do not If the state space is finite and we do not discard nodes that revisit states, in discard nodes that revisit states, in general the search is not completegeneral the search is not complete

If the state space is infinite, in general the If the state space is infinite, in general the search is not completesearch is not complete

Admissible HeuristicAdmissible Heuristic

Let h*(N) be the cost of the optimal Let h*(N) be the cost of the optimal path from N to a goal nodepath from N to a goal node

The heuristic function h(N) is The heuristic function h(N) is admissibleadmissible if: if: 0 0 h(N) h(N) h*(N) h*(N)

An admissible heuristic function is An admissible heuristic function is always always optimistic optimistic !!

•Note: G is a goal node h(G) = 0

hh11(N) = number of misplaced tiles = 6(N) = number of misplaced tiles = 6is admissibleis admissible

hh22(N) = sum of the (Manhattan) distances of (N) = sum of the (Manhattan) distances of every tile to its goal position every tile to its goal position = 2 + 3 + 0 + 1 + 3 + 0 + 3 + 1 = = 2 + 3 + 0 + 1 + 3 + 0 + 3 + 1 = 1313is admissibleis admissible

8-Puzzle Heuristics8-Puzzle Heuristics

14

7

5

2

63

8

STATE(N)

64

7

1

5

2

8

3

Goal state

Robot Navigation HeuristicsRobot Navigation Heuristics

Cost of one horizontal/vertical step = 1Cost of one diagonal step = 2

2 2g g1 N Nh (N) = (x -x ) +(y -y )

h2(N) = |xN-xg| + |yN-yg|is admissible

is admissible if moving along diagonals is not allowed, and not admissible otherwise

A* SearchA* Search(most popular algorithm in AI)(most popular algorithm in AI)

f(N) = g(N) + h(N), where:f(N) = g(N) + h(N), where:•g(N) = cost of best path found so far to Ng(N) = cost of best path found so far to N

•h(N) = admissible heuristic functionh(N) = admissible heuristic function

for all arcs: for all arcs: 00 << cc(N,N’)(N,N’)

““modified” search algorithm is usedmodified” search algorithm is used

Best-first search is then called Best-first search is then called A* A* searchsearch

Result #1Result #1

A* is A* is completecomplete and and optimaloptimal

[This result holds if nodes revisiting [This result holds if nodes revisiting states are not discarded]states are not discarded]

Proof (1/2)Proof (1/2)1)1) If a solution exists, A* terminates and If a solution exists, A* terminates and

returns a solutionreturns a solution

For each node N on the fringe, f(N)For each node N on the fringe, f(N)d(N)d(N), , where d(N) is the depth of N in the treewhere d(N) is the depth of N in the tree

As long as A* hasn’t terminated, a node K on As long as A* hasn’t terminated, a node K on the fringe lies on a solution paththe fringe lies on a solution path

Since each node expansion increases the Since each node expansion increases the length of one path, K will eventually be length of one path, K will eventually be selected for expansionselected for expansion

Proof (2/2)Proof (2/2)2)2) Whenever A* chooses to expand a goal Whenever A* chooses to expand a goal

node, the path to this node is optimalnode, the path to this node is optimal

C*= h*(initial-node)C*= h*(initial-node)

G’: non-optimal goal node in the fringeG’: non-optimal goal node in the fringef(G’) = g(G’) + h(G’) = g(G’) f(G’) = g(G’) + h(G’) = g(G’) C* C*

A node K in the fringe lies on an optimal A node K in the fringe lies on an optimal path:path:f(K) = g(K) + h(K) f(K) = g(K) + h(K) C* C*

So, G’ is not selected for expansionSo, G’ is not selected for expansion

Time Limit IssueTime Limit Issue When a problem has no solution, A* runs for ever When a problem has no solution, A* runs for ever

if the state space is infinite or states can be if the state space is infinite or states can be revisited an arbitrary number of times (the search revisited an arbitrary number of times (the search tree can grow arbitrarily large). In other cases, it tree can grow arbitrarily large). In other cases, it may take a huge amount of time to terminate may take a huge amount of time to terminate

So, in practice, A* must be given a time limit. If it So, in practice, A* must be given a time limit. If it has not found a solution within this limit, it stops. has not found a solution within this limit, it stops. Then there is no way to know if the problem has Then there is no way to know if the problem has no solution or A* needed more time to find itno solution or A* needed more time to find it

In the past, when AI systems were “small” and In the past, when AI systems were “small” and solving a single search problem at a time, this solving a single search problem at a time, this was not too much of a concern. As AI systems was not too much of a concern. As AI systems become larger, they have to solve a multitude of become larger, they have to solve a multitude of search problems concurrently. Then, a question search problems concurrently. Then, a question arises: arises: What should be the time limit for What should be the time limit for each of them?each of them?

Time Limit IssueTime Limit Issue

Hence, the usefulness of a simple test, like in the (2n-1)-puzzle, that determines if the goal is reachable

Unfortunately, such a test rarely exists

Hence, the usefulness of a simple test, like in the (2n-1)-puzzle, that determines if the goal is reachable

Unfortunately, such a test rarely exists

8-Puzzle8-Puzzle

0+4

1+5

1+5

1+3

3+3

3+4

3+4

3+2 4+1

5+2

5+0

2+3

2+4

2+3

f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

Robot NavigationRobot Navigation


0 211

58 7

7

3

4

7

6

7

6 3 2

8

6

45

23 3

36 5 24 43 5

54 6

5

6

4

5

f(N) = h(N), with h(N) = Manhattan distance to the goal(not A*)


0 211

58 7

7

3

4

7

6

7

6 3 2

8

6

45

23 3

36 5 24 43 5

54 6

5

6

4

5

f(N) = h(N), with h(N) = Manhattan distance to the goal(not A*)

7

0


f(N) = g(N)+h(N), with h(N) = Manhattan distance to goal(A*)

0 211

58 7

7

3

4

7

6

7

6 3 2

8

6

45

23 3

36 5 24 43 5

54 6

5

6

4

57+0

6+1

6+1

8+1

7+0

7+2

6+1

7+2

6+1

8+1

7+2

8+3

7+26+36+35+45+44+54+53+63+62+7

8+37+47+46+5

5+6

6+55+6

2+73+8

4+7

5+64+7

3+8

4+73+83+82+92+93+10

2+9

3+8

2+91+101+100+110+11

How to create an How to create an admissible h?admissible h?

An admissible heuristic can usually be seen An admissible heuristic can usually be seen as the cost of an optimal solution to a as the cost of an optimal solution to a relaxedrelaxed problem (one obtained by removing problem (one obtained by removing constraints)constraints)

In robot navigation on a grid:In robot navigation on a grid:• The Manhattan distance corresponds to removing The Manhattan distance corresponds to removing

the obstacles the obstacles • The Euclidean distance corresponds to removing The Euclidean distance corresponds to removing

both the obstacles and the constraint that the both the obstacles and the constraint that the robot moves on a gridrobot moves on a grid

More on this topic later More on this topic later

What to do with revisited What to do with revisited states?states?

c = 1

100

21

2

h = 100

0

90

1 The heuristic h is The heuristic h is clearly admissibleclearly admissible


c = 1

100

21

2

h = 100

0

90

1

104

4+90

f = 1+100 2+1

?If we discard this new node, then the searchalgorithm expands the goal node next andreturns a non-optimal solution

1

100

21

2

100

0

90

1

104

4+90

1+100 2+1

2+90

102

Instead, if we do not discard nodes revisiting states, the search terminates with an optimal solution


But ...But ...If we do not discard nodes revisiting If we do not discard nodes revisiting states, the size of the search tree can states, the size of the search tree can be exponential in the number of visited be exponential in the number of visited statesstates

1

2

11

1

2

1

1

1+1 1+1

2+1 2+1 2+1 2+1

4 4 4 4 4 4 4 4

It is not harmful to discard a node It is not harmful to discard a node revisiting a state revisiting a state if the new path to this if the new path to this state has higher cost than the previous state has higher cost than the previous oneone

A* remains optimalA* remains optimal

Fortunately, for a large family of Fortunately, for a large family of admissible heuristics – admissible heuristics – consistentconsistent heuristics – there is a much easier way of heuristics – there is a much easier way of dealing with revisited statesdealing with revisited states

Consistent HeuristicConsistent HeuristicA heuristic h is A heuristic h is consistentconsistent if if

1) for each node N and each child N’ of 1) for each node N and each child N’ of N:N:

h(N) h(N) c(N,N’) + h(N’) c(N,N’) + h(N’)

2) for each goal node G:2) for each goal node G:

h(G) = 0h(G) = 0

The heuristic is also said to beThe heuristic is also said to be monotonemonotone

(triangle inequality)

N

N’ h(N)

h(N’)

c(N,N’)

Consistency ViolationConsistency Violation

N

N’ h(N)=100

h(N’)=10

c(N,N’)=10

(triangle inequality)

If h tells that N is 100 units from the goal, then moving from N along an arc costing 10 units should not lead to a node N’ that h estimates to be 10 units away from the goal

If h tells that N is 100 units from the goal, then moving from N along an arc costing 10 units should not lead to a node N’ that h estimates to be 10 units away from the goal

A consistent heuristic is also admissibleA consistent heuristic is also admissible

An admissible heuristic may not be An admissible heuristic may not be consistent, but many admissible consistent, but many admissible heuristics are consistentheuristics are consistent

Admissibility and Admissibility and ConsistencyConsistency

8-Puzzle8-Puzzle

1 2 3

4 5 6

7 8

12

3

4

5

67

8

STATE(N) Goal

h1(N) = number of misplaced tiles h2(N) = sum of the (Manhattan) distances of every tile to its goal positionare both consistent


Cost of one horizontal/vertical step = 1Cost of one diagonal step = 2

2 2g g1 N Nh (N) = (x -x ) +(y -y )

h2(N) = |xN-xg| + |yN-yg|is consistent

is consistent if moving along diagonals is not allowed, and not consistent otherwise

If h is consistent, then whenever If h is consistent, then whenever A* expands a node, it has already A* expands a node, it has already found an optimal path to this found an optimal path to this node’s statenode’s state

Result #2Result #2

ProofProof1)1) Consider a node N and its child N’ Consider a node N and its child N’

Since h is consistent: h(N) Since h is consistent: h(N) c(N,N’)+h(N’) c(N,N’)+h(N’)

f(N) = g(N)+h(N) f(N) = g(N)+h(N) g(N)+c(N,N’)+h(N’) = g(N)+c(N,N’)+h(N’) = f(N’)f(N’)So, f is So, f is non-decreasingnon-decreasing along any path along any path

2)2) If K is selected for expansion, then any other If K is selected for expansion, then any other node K’ in the fringe verifies f(K’) node K’ in the fringe verifies f(K’) f(K) f(K)

So, if one node K’ lies on another path to the So, if one node K’ lies on another path to the state of K, the cost of this other path is no state of K, the cost of this other path is no smaller than the path to Ksmaller than the path to K

N

N’

If h is consistent, then whenever If h is consistent, then whenever A* expands a node, it has already A* expands a node, it has already found an optimal path to this found an optimal path to this node’s statenode’s state

Result #2Result #2

N N1S S1

The path to N is the optimal path to S

N2

N2 can be discarded

Revisited States with Revisited States with Consistent HeuristicConsistent Heuristic When a node is expanded, store its When a node is expanded, store its

state into state into CLOSED CLOSED

When a new node N is generated:When a new node N is generated:• If If STATESTATE(N) is in (N) is in CLOSEDCLOSED, discard N, discard N• If there exists a node N’ in the fringe If there exists a node N’ in the fringe

such that such that STATESTATE(N’) = (N’) = STATESTATE(N), (N), discard the node – N or N’ – with the discard the node – N or N’ – with the largest flargest f

Is A* with some consistent Is A* with some consistent heuristic all that we need?heuristic all that we need?

No !No ! There are There are very dumbvery dumb consistent consistent

heuristicsheuristics

h h 0 0

It is consistent (hence, admissible) !It is consistent (hence, admissible) ! A* with hA* with h0 is uniform-cost search0 is uniform-cost search Breadth-first and uniform-cost are Breadth-first and uniform-cost are

particular cases of A*particular cases of A*

Heuristic AccuracyHeuristic AccuracyLet hLet h11 and h and h22 be two consistent heuristics such be two consistent heuristics such that for all nodes N: that for all nodes N:

hh11(N) (N) h h22(N)(N)

hh22 is said to be is said to be more accuratemore accurate (or (or more more informedinformed)) than hthan h11

hh11(N) = number of misplaced tiles (N) = number of misplaced tiles hh22(N) = sum of distances of every (N) = sum of distances of every

tile to its goal positiontile to its goal position

hh22 is more accurate than h is more accurate than h11

14

7

5

2

63

8

STATE(N)

64

7

1

5

2

8

3

Goal state

Result #3Result #3

Let hLet h22 be more accurate than h be more accurate than h11

Let ALet A11* be A* using h* be A* using h11 and A and A22* be A* * be A* using husing h22

Whenever a solution exists, all the Whenever a solution exists, all the nodes expanded by Anodes expanded by A22* are also * are also expanded by Aexpanded by A11* *

ProofProof C* = h*(initial-node)C* = h*(initial-node) Every node N such that f(N) Every node N such that f(N) C* is C* is

eventually expandedeventually expanded Every node N such that h(N) Every node N such that h(N) C* C*g(N) is g(N) is

eventually expanded eventually expanded Since hSince h11(N) (N) h h22(N), every node (N), every node

expanded by Aexpanded by A22* is also expanded by * is also expanded by AA11**

……except for nodes N such that f(N) = C*except for nodes N such that f(N) = C*

Effective Branching FactorEffective Branching Factor

It is used as a measure the It is used as a measure the effectiveness of a heuristiceffectiveness of a heuristic

Let n be the total number of nodes Let n be the total number of nodes expanded by A* for a particular expanded by A* for a particular problem and d the depth of the problem and d the depth of the solutionsolution

The The effective branchingeffective branching factor b* is factor b* is defined by n = 1defined by n = 1 ++ b*b* ++ (b*)(b*)22 +...++...+ (b*)(b*)dd

Experimental ResultsExperimental Results 8-puzzle with:8-puzzle with:

hh11 = number of misplaced tiles = number of misplaced tiles

hh22 = sum of distances of tiles to their goal positions = sum of distances of tiles to their goal positions

Random generation of many problem instancesRandom generation of many problem instances Average effective branching factors Average effective branching factors (number of (number of

expanded nodes)expanded nodes)::dd IDSIDS AA11** AA22**

22 2.452.45 1.791.79 1.791.79

66 2.732.73 1.341.34 1.301.30

1212 2.78 2.78 (3,644,035)(3,644,035)

1.42 1.42 (227)(227) 1.24 1.24 (73)(73)

1616 ---- 1.451.45 1.251.25

2020 ---- 1.471.47 1.271.27

2424 ---- 1.48 1.48 (39,135)(39,135)

1.26 1.26 (1,641)(1,641)

By solving By solving relaxedrelaxed problems at each node problems at each node In the 8-puzzle, the sum of the distances of In the 8-puzzle, the sum of the distances of

each tile to its goal position (heach tile to its goal position (h22) corresponds ) corresponds to solving 8 simple problems:to solving 8 simple problems:

It ignores negative interactions among tiles It ignores negative interactions among tiles

How to create good How to create good heuristics?heuristics?

14

7

5

2

63

8

64

7

1

5

2

8

3

5

5

For example, we could consider two more complex relaxed problems:For example, we could consider two more complex relaxed problems:

h = dh = d12341234 + d + d5678 5678 [disjoint pattern heuristic][disjoint pattern heuristic]

These distances could have been precomputed in a databaseThese distances could have been precomputed in a database

Can we do better?Can we do better?

14

7

5

2

63

8

64

7

1

5

2

8

3

3

2 14 4

1 2 3

6

7

5

87

5

6

8

Can we do better?Can we do better?

Several order-of-magnitude speedups have been obtained this way for the

15- and 24-puzzle (see R&N)

Several order-of-magnitude speedups have been obtained this way for the

15- and 24-puzzle (see R&N)

On Completeness and On Completeness and OptimalityOptimality A* with a consistent heuristic has nice A* with a consistent heuristic has nice

properties: completeness, optimality, no properties: completeness, optimality, no need to revisit statesneed to revisit states

Theoretical completeness does not mean Theoretical completeness does not mean “practical” completeness if you must wait “practical” completeness if you must wait too long to get a solution (remember the too long to get a solution (remember the time limit issue)time limit issue)

So, if one can’t design an accurate So, if one can’t design an accurate consistent heuristic, it may be better to consistent heuristic, it may be better to settle for a non-admissible heuristic that settle for a non-admissible heuristic that “works well in practice”, even through “works well in practice”, even through completeness and optimality are no longer completeness and optimality are no longer guaranteed guaranteed

Iterative Deepening A* Iterative Deepening A* (IDA*)(IDA*) Idea: Reduce memory requirement of Idea: Reduce memory requirement of

A* by applying cutoff on values of fA* by applying cutoff on values of f Consistent heuristic hConsistent heuristic h Algorithm IDA*:Algorithm IDA*:

1.1. Initialize cutoff to f(initial-node)Initialize cutoff to f(initial-node)

2.2. Repeat:Repeat:a.a.Perform depth-first search by expanding Perform depth-first search by expanding

all nodes N such that f(N) all nodes N such that f(N) cutoff cutoff

b.b.Reset cutoff to smallest value f of non-Reset cutoff to smallest value f of non-expanded (leaf) nodesexpanded (leaf) nodes

8-Puzzle8-Puzzle

4

6


Cutoff=4

8-Puzzle8-Puzzle

4

4

6

Cutoff=4

6


8-Puzzle8-Puzzle

4

4

6

Cutoff=4

6

5


8-Puzzle8-Puzzle

4

4

6

Cutoff=4

6

5

5


4

8-Puzzle8-Puzzle

4

6

Cutoff=4

6

5

56


8-Puzzle8-Puzzle

4

6

Cutoff=5


8-Puzzle8-Puzzle

4

4

6

Cutoff=5

6


8-Puzzle8-Puzzle

4

4

6

Cutoff=5

6

5


8-Puzzle8-Puzzle

4

4

6

Cutoff=5

6

5

7


8-Puzzle8-Puzzle

4

4

6

Cutoff=5

6

5

7

5


8-Puzzle8-Puzzle

4

4

6

Cutoff=5

6

5

7

5 5


8-Puzzle8-Puzzle

4

4

6

Cutoff=5

6

5

7

5 5


5

Advantages/Drawbacks of Advantages/Drawbacks of IDA*IDA*

Advantages:Advantages:• Still complete and optimalStill complete and optimal• Requires less memory than A*Requires less memory than A*• Avoid the overhead to sort the fringeAvoid the overhead to sort the fringe

Drawbacks:Drawbacks:• Can’t avoid revisiting states not on the Can’t avoid revisiting states not on the

current pathcurrent path• Available memory is poorly usedAvailable memory is poorly used

SMA*SMA*(Simplified Memory-bounded A*)(Simplified Memory-bounded A*) Works like A* until memory is fullWorks like A* until memory is full Then SMA* drops the node in the fringe with the Then SMA* drops the node in the fringe with the

largest f value and “backs up” this value to its largest f value and “backs up” this value to its parentparent

When all children of a node N have been dropped, When all children of a node N have been dropped, the smallest backed up value replaces f(N)the smallest backed up value replaces f(N)

In this way, the root of an erased subtree In this way, the root of an erased subtree remembers the best path in that subtreeremembers the best path in that subtree

SMA* will regenerate this subtree only if all other SMA* will regenerate this subtree only if all other nodes in the fringe have greater f values nodes in the fringe have greater f values

SMA* can’t completely avoid revisiting states, but SMA* can’t completely avoid revisiting states, but it can do a better job at this that IDA*it can do a better job at this that IDA*

Local SearchLocal Search

Light-memory search method Light-memory search method No search tree; only the current state No search tree; only the current state

is represented!is represented! Only applicable to problems where the Only applicable to problems where the

path is irrelevant (e.g., 8-queen), path is irrelevant (e.g., 8-queen), unless the path is encoded in the stateunless the path is encoded in the state

Many similarities with optimization Many similarities with optimization techniquestechniques

Steepest DescentSteepest Descent1)1) S S initial state initial state

2)2) Repeat:Repeat:a)a) S’ S’ arg min arg minS’S’SUCCESSORS(S)SUCCESSORS(S){h(S’)} {h(S’)}

b)b) if GOAL?(S’) return S’ if GOAL?(S’) return S’

c)c) if h(S’) if h(S’) h(S) then S h(S) then S S’ else return failure S’ else return failure

Similar to:Similar to:

- hill climbing with –h- hill climbing with –h

- gradient descent over continuous space- gradient descent over continuous space

Application: 8-QueenApplication: 8-QueenRepeat n times:Repeat n times:

1)1) Pick an initial state S at random with one queen in each Pick an initial state S at random with one queen in each columncolumn

2)2) Repeat k times:Repeat k times:

a)a) If GOAL?(S) then return SIf GOAL?(S) then return S

b)b) Pick an attacked queen Pick an attacked queen QQ at random at random

c)c) Move Move QQ it in its column to minimize the number of it in its column to minimize the number of attacking queens is minimum attacking queens is minimum new S new S [min-conflicts [min-conflicts heuristic]heuristic]

3)3) Return failureReturn failure12

33223

22

22

202

• Why does it work ???• There are many goal states that are

well-distributed over the state space• If no solution has been found after a few

steps, it’s better to start it all over again. Building a search tree would be much less efficient because of the high branching factor

• Running time almost independent of the number of queens

• Why does it work ???• There are many goal states that are

well-distributed over the state space• If no solution has been found after a few

steps, it’s better to start it all over again. Building a search tree would be much less efficient because of the high branching factor

• Running time almost independent of the number of queens

Steepest DescentSteepest Descentmay easily get stuck in local minimamay easily get stuck in local minima Random restart (as in n-queen Random restart (as in n-queen

example)example)

Monte Carlo descent: chooses Monte Carlo descent: chooses probabilistically among the probabilistically among the successorssuccessors

Monte Carlo DescentMonte Carlo Descent1)1) S S initial state initial state

2)2) Repeat k times:Repeat k times:a)a) If GOAL?(S) then return SIf GOAL?(S) then return S

b)b) S’ S’ successor of S picked at random successor of S picked at random c)c) if h(S’) if h(S’) h(S) then S h(S) then S S’ S’

d)d) else else - h = h(S’)-h(S)h = h(S’)-h(S)

- with probability ~ exp(with probability ~ exp(h/T), where T is called the h/T), where T is called the “temperature” S “temperature” S S’ S’ [Metropolis criterion][Metropolis criterion]

3)3) Return failureReturn failure

Simulated annealingSimulated annealing lowers T over the k iterations. lowers T over the k iterations.

It starts with a large T and slowly decreases TIt starts with a large T and slowly decreases T

Search problems

Blind search

Heuristic search: best-first and A*

Construction of heuristics Local searchVariants of A*

When to Use Search When to Use Search Techniques?Techniques?

1)1)The search space is small, andThe search space is small, and• No other technique is available, orNo other technique is available, or• Developing a more efficient technique is Developing a more efficient technique is

not worth the effort not worth the effort

2)2)The search space is large, andThe search space is large, and• No other available technique is available, No other available technique is available,

andand• There exist “good” heuristicsThere exist “good” heuristics

04 search heuristic

Education

search node

search fn

n modified search algorithm

search iscomplete

generalthe search

multitude of search

search tree fn

searchnode n