informed search r & n chapter 4, except 4.4. heuristics the word heuristic comes from the greek...

Informed Search

R & N Chapter 4, except 4.4

Heuristics

The word heuristic comes from the Greek word (heuriskein), meaning “to discover”, which is also the origin of eureka, derived from Archimedes’ reputed exclamation, heurika (“I have found”), uttered when he had discovered a method for determining the purity of gold.

We typically use the term heuristic to refer to a technique that is likely, but not guaranteed, to produce a good (but not necessarily best) answer or the best answer with a good but not necessarily best expenditure of effort.

Heuristics and Human BehaviorThe availability heuristic is an oversimplified rule of thumb, or heuristic, which occurs when people estimate the probability of an outcome based on how easy that outcome is to imagine. As such, vividly described, emotionally-charged possibilities will be perceived as being more likely than those that are harder to picture or are difficult to understand, resulting in a corresponding cognitive bias.

For example, most people think that dying from a shark attack is more likely than dying from injuries sustained from falling airplane parts, yet the opposite is true by a factor of 30. Perhaps this is because sharks are inherently terrifying or because shark attacks receive more media coverage. Many people seem to fear plane crashes, yet a dark fate is much more likely to befall you in an automobile accident on the way to the airport.

One important corollary finding to this heuristic is that people asked to imagine an outcome immediately view it as more likely than those that were not. And, that which was vividly described is viewed as more likely than that which was wrought a pallid description. This tendency is often seen in politics. For example, when justifying an invasion of Iraq in the 2003 State of the Union Address, President George W. Bush said: "Imagine those 19 hijackers with other weapons and other plans, this time armed by Saddam Hussein. It would take one vial, one canister, one crate slipped into this country to bring a day of horror like none we have ever known."

An opposite effect of this bias, called denial, occurs when an outcome is so upsetting that the very act of thinking about it leads to an increased refusal to believe it might occur. In this case, being asked to imagine the outcome actually made subjects view it as less likely.

This phenomenon was first reported by psychologists Amos Tversky and Daniel Kahneman.

http://en.wikipedia.org/wiki/Availability_heuristic

http://en.wikipedia.org/wiki/Heuristic

http://en.wikipedia.org/wiki/Cognitive_bias

Heuristic Functions

The job of a heuristic function f(n) is to evaluate a node n in a search tree so that the “best” node can be selected for expansion.

Two approaches to defining f:

•f measures the value of the state contained in the node.

•f measures the cost of the state (and possibly the path to it). Often, in this case, we decompose f:

f(n) = g(n) + h(n), whereg(n) = the cost to get to n, andh(n) = an estimate of the cost to get from n to a goal

Examples - f Measures the Value

Often, in these cases, f is the weighted sum of a set of component values:

•Chess

•School lunch planning

•Autonomous Mars rover

•Perry Mason defense lawyer

Examples – f Measures the Cost to a Goal

In many problems, all goal states are equally good. Then all we care about is the cost of getting to one. So a state is good if the estimated cost of getting from it to a goal is low. Or we may also want to take into account the cost, so far, of getting to it.

• 8–Puzzle

• Water jug

• Route planning

Now We Must Evaluate on Generation

But We Still Have to Check for Alternatives

Search Algorithms – Straight Line or Branching

Branching Algorithms

•A* - See handout (http://www.cs.utexas.edu/users/ear/cs343/A_Star_Algorithm.htm)

•RBFS

•SMA* - A* until it runs out of memory. Then drop the worst node and back up the value.

•AO* - See handout (http://www.cs.utexas.edu/users/ear/cs343/AOstarAlgorithm.htm)

http://www.cs.utexas.edu/users/ear/cs343/A_Star_Algorithm.htm

A*

A

B C

0+2

1+2 5+2

1 5

A*

A

B C

D E

0+2

1+2 5+2

2+2 3+5

1 5

1 2

A*

A

B C

D E

F

0+2

1+2 5+2

2+2

4+2

3+5

1 5

1 2

2

A*

A

B C

D E

F

G

2+0

1+2 5+2

2+2

4+2

3+5

6+2

1 5

1 2

2

2

A*

A

B C

D E

F

G

2+0

1+2 5+2

2+2

4+2

3+5

7+2

1 5

1 2

2

3

1

We’ve found a cheaper path to G. We need to record it.

1

6+2

The Optimality of A*

A

B C

D E

F

G

0+2

1+2 5+2

2+2

4+2

3+5

6+2

1 5

1 2

2

2

h has underestimated the true cost. But A* will find H.

H 6+0

1


A

B C

D E

F

G

0+2

1+2 5+5

2+2

4+2

3+5

8+0

1 5

1 2

2

4

h has overestimated the true cost and missed the optimal path.

H 6+0

1


A* is admissible (i.e., it is guaranteed to find an optimal path to a goal, if one exists) iff h never overestimates the true cost to a goal.

If h rarely overestimates the true cost by more than , then A* will rarely find a solution whose cost is more than greater than the cost of the optimal solution.

Recursive Best First Search

SMA*

Like A* until it runs out of memory. Then drop the worst leaf but backs up the value to the parent. This part of the tree will only be regenerated if all alternatives become worse.

A Version of Depth-First Search - Branch and Bound

Consider the problem of planning a ski vacation.

Fly to A $600 Fly to B $800 Fly to C $2000

Stay D $200

(800)

Stay E $250

(850)

Total cost

(1200)

Problem Reduction

Goal: Acquire TV

Steal TV Earn Money Buy TV

Or another one: Theorem proving in which we reason backwards from the theorem we’re trying to prove.

We Need a New Algorithm - AO*

AO* - Computing Scores

3 1

A

CB(1) (1)

(2)

In A*, we assigned to each node a cost f = g + h. We combined the cost of getting to the node with the expected cost of getting from it to a solution. Here we assign to each node just the cost of getting from it to a solution. We assign to each path the expected cost of getting to a goal by following it. We must update the path costs at each step.


3 1

A

CB(1) (1)

(2)

X (8)

1


3 1

A

CB(1) (9)

(4)

X (8)

1


3

2

1

A

CB

D

(3)

(1)

(9)

X

1

(8)


3

2

11

1

A

CB

D

E F

(1)

(1)

(1)

(9)

(3)

X

1

(8)


3

2

11

1

A

CB

D

E F

(8)

(1)

(6)

(9)

(3)

X

1

(8)


3

2

11

1

A

CB

D

E F

(8)

(1)

(6)

(9)

(3)

X

1

(8)

1


3

2

11

1

A

CB

D

E F

(8)

(1)

(6)

(5)

(3)

X

1

(4)

1

A Detail – Should We Store Ancestor Links?

3

2

11

1

A

CB

D

E F

(1)

(3)

(1)

(7)

(1)


3

2

11

1

A

CB

D

E F

(1)

(3)

(1)

(7)

(1)1


3

2

11

1

A

CB

D

E F

(1)

(3)

(1)

(7)

(1)

1

1

Good Heuristic FunctionsIn any searching problem where there are b choices at each node and a depth of d at the goal node, a naive searching algorithm would have to potentially search around bd nodes before finding a solution. Heuristics improve the efficiency of search algorithms by reducing the branching factor from b to (ideally) a low constant b*.

Although any admissible heuristic will give an optimal answer, a heuristic that gives a lower branching factor is more computationally efficient for a particular problem. It can be shown that a heuristic h2(n) is better than another heuristic

h1(n) , if h2(n) dominates h1(n), ie. h1(n) < h2(n) for all n.

From http://en.wikipedia.org/wiki/Heuristic

Finding Good Heuristic Functions•Solution costs of sub-problems often serve as useful estimates of the overall solution cost. These are always admissible if subproblems don’t overlap. For example, a heuristic for a 10-puzzle might be the cost of moving tiles 1-5 into their correct places. A common idea is to use a pattern database that stores the exact solution cost of every subproblem instance.

•The solution of a relaxed problem often serves as a useful admissible estimate of the original. For example Manhattan distance is a relaxed version of the n-puzzle problem, because we assume we can move each tile to its position in a single step.

•Given a set of admissible heuristic functions, the function h(n) = max{h1(n),h2(n)...hi(n)} is an admissible heuristic that

dominates all of them.

(From: http://en.wikipedia.org/wiki/Heuristic)

Straight Line (Local) Search

•Generate and test – the uninformed version. Start over each time.

•Smarter strategies, in which we make an initial attempt, then attempt to improve on it. We’ll use complete state formulations for these algorithms.

•Hill climbing

•Simulated annealing

•Genetic algorithms

Hill Climbing – The Algorithm

function Hill-Climbing(problem) returns a state that is a local maximum inputs: problem local variables: current, a node; neighbor a node current Make-Node(initial-state[problem]) loop do neighbor a highest-valued successor of current if value[neighbor] value[current] then return state[current] current neighbor

N-Queens

We could solve the problem with depth-first search:

http://www.math.utah.edu/~alfeld/queens/queens.html

Hill Climbing - N-Queens

x

x

x

x

x

x

xx

x

x

x

x

(6)

(4)

(2)

f = # of attacking pairs

N-Queens – An Alternative

It turns out that the N-Queens problem can be solved almost all the time without search at all:

http://www.apl.jhu.edu/~hall/NQueens.html

Hill Climbing – Some Problems

Sometimes the Problem is f

From the initial state, move A to the table. Three choices for what to do next.

A local heuristic function: Add one point for every block that is resting on the thing it is supposed to be resting on. Subtract one point for every block that is sitting on the wrong thing.

Sometimes the Problem is f

From the initial state, move A to the table. Three choices for what to do next.

A global heuristic function: For each block that has the correct support structure (i. e., the complete structure underneath it is exactly as it should be), add one point for every block in the support structure. For each block that has an incorrect support structure, subtract one point for every block in the existing support structure.

Hill Climbing – Is Close Good Enough?

AB

Is A good enough?•8 Queens•Choose winning lottery numbers

Hill Climbing – Is Close Good Enough?

AB

Is A good enough?•8 Queens•Choose winning lottery numbers•Get the cheapest travel itinerary

Simulated Annealing

function Simulated-Annealing(problem,schedule) returns a solution state inputs: problem, a problem schedule, a mapping from time to “temperature” local variables: current, a node next, a node best, a node

T, a “temperature” current Make-Node(initial-state[problem]) best current for t 1 to to T schedule[t] if T = 0 then return best next a randomly selected successor of current if GoalTest(next) then return next if value[next] > value[best] then best next E value[next] - value[current] if E > 0 then current next else current next with p = eE/T

Following Slightly More than One Path

What if we follow some small number of paths?

• Local beam search:

1. Start with k randomly generated nodes.

2. Generate k successors of each.

3. Keep the best k out of the k2.

• Genetic algorithms:

1. Start with k randomly generated nodes called the population.

2. Generate successors by combining pairs of nodes.

3. Allow mutations.

Minimax Search for Games

A

B C D (8) (3) (-2)


A

B C D

E F G H I J K (9) (-6) (0) (0) (-2) (-4) (10)


A (-2)

B (-6) C (-2) D (-4)

E F G H I J K (9) (-6) (0) (0) (-2) (-4) (10)

Minimax with Alpha Beta Cutoffs(a form of Branch and Bound)

A

B (8) C (-2) D (-4)


A ( 5)

B (5) C (-2) D (-4)

E F G (9) (6) (5)


A ( 5)

B (5) C ( -3) D (-4)

E F G H I J (9) (6) (5) (-3)

Other Ideas in Game Tree Search

•Iterative deepening allows a program to use exactly as much time as it has.

•Waiting for quiescence helps to avoid the horizon effect.

•Secondary search double checks a chosen move before committing.

•Saving backed up values trades space for time and effectively extends the depth of the search in common configurations.

•Book moves help at the beginning and the end.

informed search r & n chapter 4, except 4.4. heuristics the word heuristic comes from the greek...

Documents