![Page 1: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/1.jpg)
1
CS 2710, ISSP 2610
Chapter 4, Part 2Heuristic Search
![Page 2: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/2.jpg)
2
Beam Search
• Cheap, unpredictable search• For problems with many solutions,
it may be worthwhile to discard unpromising paths
• Greedy best first search that keeps a fixed number of nodes on the fringe
![Page 3: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/3.jpg)
3
Beam Search
def beamSearch(fringe,beamwidth): while len(fringe) > 0: cur = fringe[0] fringe = fringe[1:] if goalp(cur): return cur newnodes = makeNodes(cur, successors(cur)) fringe=sortByH(newnodes, fringe) fringe = fringe[:beamwidth]return []
![Page 4: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/4.jpg)
4
Beam Search
• Optimal? Complete?• Hardly!• Space?• O(b) (generates the successors)• Often useful
![Page 5: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/5.jpg)
5
Generating Heuristics
• Exact solutions to different (relaxed problems)
• H1 (# of misplaced tiles) is perfectly accurate if a tile could move to any square
• H2 (sum of Manhattan distances) is perfectly accurate if a tile could move 1 square in any direction
![Page 6: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/6.jpg)
6
Relaxed Problems (cont)
• If problem is defined formally as a set of constraints, relaxed problems can be generated automatically
• A tile can move from square A to square B if:– A is adjacent to B, and– B is blank
• 3 relaxed problems by removing one or both constraints
• Absolver (Prieditis, 1993)
![Page 7: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/7.jpg)
7
Combining Heuristics
• If you have lots of heuristics and none dominates the others and all are admissible…
• Use them all!• H(n) = max(h1(n), …, hm(n))
![Page 8: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/8.jpg)
8
Generating Heuristics
• Use the solution cost of a subproblem. E.g., get tiles 1 through 4 in the right location, ignoring the others.
• Pattern databases: store exact solution costs for every possible subproblem instance. – The heuristic function looks up the value in
the DB – DB constructed by searching back from goal
and recording cost of each new pattern • Do the same for tiles 5,6,7,8 and take
max
![Page 9: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/9.jpg)
9
Other Sources of Heuristics
• Ad-hoc, informal, rules of thumb (guesswork)• Approximate solutions to problems (algorithms
course)• Learn from experience (solving lots of 8-
puzzles). – Each optimal solution is a learning example
(node,actual cost to goal)– Learn heuristic function, E.G. H(n) = c1x1(n)
+ c2x2(n) x1 = #misplaced tiles; x2 = #adj tiles also adj in the goal state. c1 & c2 learned (best fit to the training data)
![Page 10: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/10.jpg)
10
Remaining Search Types
• Recall we have…– Backtracking state-space search– Local Search and Optimization – Constraint satisfaction search
![Page 11: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/11.jpg)
11
Local Search and Optimization
• Previous searches: keep paths in memory, and remember alternatives so search can backtrack. Solution is a path to a goal.
• Path may be irrelevant, if the final configuration only is needed (8-queens, IC design, network optimization, …)
![Page 12: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/12.jpg)
12
Local Search
• Use a single current state and move only to neighbors.
• Use little space• Can find reasonable solutions in
large or infinite (continuous) state spaces for which the other algorithms are not suitable
![Page 13: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/13.jpg)
13
Optimization
• Local search is often suitable for optimization problems. Search for best state by optimizing an objective function.
![Page 14: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/14.jpg)
14
Visualization
• States are laid out in a landscape• Height corresponds to the
objective function value• Move around the landscape to find
the highest (or lowest) peak• Only keep track of the current
states and immediate neighbors
![Page 15: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/15.jpg)
15
Local Search Alogorithms
• Two strategies for choosing the state to visit next– Hill climbing– Simulated annealing
• Then, an extension to multiple current states:– Genetic algorithms
![Page 16: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/16.jpg)
16
Hillclimbing (Greedy Local Search)
• Generate nearby successor states to the current state based on some knowledge of the problem.
• Pick the best of the bunch and replace the current state with that one.
• Loop
![Page 17: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/17.jpg)
17
Hill-climbing search problems
• Local maximum: a peak that is lower than the highest peak, so a bad solution is returned
• Plateau: the evaluation function is flat, resulting in a random walk
• Ridges: slopes very gently toward a peak, so the search may oscillate from side to side
Local maximum Plateau Ridge
![Page 18: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/18.jpg)
18
Random restart hill-climbing
• Start different hill-climbing searches from random starting positions stopping when a goal is found
• If all states have equal probability of being generated, it is complete with probability approaching 1 (a goal state will eventually be generated).
• Best if there are few local maxima and plateaux
![Page 19: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/19.jpg)
19
Random restart hill-climbingHmm…
• If all states have equal probability of being generated, it is complete with probability approaching 1 (a goal state will eventually be generated).
• If it is restarted enough times, we considered… • Well, hill-climbing stops when no neighbors are
better than current• It stops on a local optimum (whether or not it
is a global optimum). • So, “more iterations” does not seem to be the
point.
![Page 20: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/20.jpg)
20
Random restart hill-climbingHmm…
• If all states have equal probability of being generated, it is complete with probability approaching 1 (a goal state will eventually be generated). Any way to see this as true?
• R&N,p. 111: A complete local search algorithm always finds a goal if one exists; an optimal algorithm always finds a global minimum/maximum.
• So, “goal” means local minimum/maximum
![Page 21: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/21.jpg)
21
Simulated Annealing
• Based on a metallurgical metaphor– Start with a temperature set very
high and slowly reduce it.– Run hillclimbing with the twist that
you can occasionally replace the current state with a worse state based on the current temperature and how much worse the new state is.
![Page 22: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/22.jpg)
22
Simulated Annealing
• Annealing: harden metals and glass by heating them to a high temperature and then gradually cooling them
• At the start, make lots of moves and then gradually slow down
![Page 23: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/23.jpg)
23
Simulated Annealing
• More formally…– Generate a random new neighbor
from current state.– If it’s better take it.– If it’s worse then take it with some
probability proportional to the temperature and the delta between the new and old states.
![Page 24: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/24.jpg)
24
Simulated annealing
• Probability of a move decreases with the amount ΔE by which the evaluation is worsened
• A second parameter T is also used to determine the probability: high T allows more worse moves, T close to zero results in few or no bad moves
• Schedule input determines the value of T as a function of the completed cycles
![Page 25: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/25.jpg)
25
function Simulated-Annealing(problem, schedule) returns a solution stateinputs: problem, a problem
schedule, a mapping from time to “temperature”local variables: current, a node
next, a nodeT, a “temperature” controlling the
probability of downward stepscurrent ← Make-Node(Initial-State[problem])for t ← 1 to ∞ do
T ← schedule[t]if T=0 then return currentnext ← a randomly selected successor of currentΔE ← Value[next] – Value[current]if ΔE > 0 then current ← nextelse current ← next only with probability eΔE/T
![Page 26: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/26.jpg)
26
Local Beam Search
• Keep track of k states rather than just one, as in hill climbing
• In comparison to beam search we saw earlier, this alg is state-based rather than node-based.
![Page 27: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/27.jpg)
27
Local Beam Search
• Begins with k randomly generated states
• At each step, all successors of all k states are generated
• If any one is a goal, alg halts• Otherwise, selects best k
successors from the complete list, and repeats
![Page 28: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/28.jpg)
28
Local Beam Search
• Successors can become concentrated in a small part of state space
• Stochastic beam search: choose k successors, with probability of choosing a given successor increasing with value
• Like natural selection: successors (offspring) of a state (organism) populate the next generation according to its value (fitness)
![Page 29: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/29.jpg)
29
Genetic Algorithms
• Variant of stochastic beam search• Combine two parent states to
generate successors (sexual versus asexual reproduction)
![Page 30: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/30.jpg)
30
Fun GA (pop, fitness-fn)Repeat new-pop = {} for i from 1 to size(pop): x = rand-sel(pop,fitness-fn) y = rand-sel(pop,fitness-fn) child = reproduce(x,y) if (small rand prob): child mutate(child) add child to new-pop pop = new-popUntil an indiv is fit enough, or out of timeReturn best indiv in pop, according to fitness-fn
![Page 31: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/31.jpg)
31
Fun reproduce(x,y) n = len(x) c = random num from 1 to n return:
append(substr(x,1,c),substr(y,c+1,n)
![Page 32: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/32.jpg)
32
Example: n-queens
• Put n queens on an n × n board with no two queens on the same row, column, or diagonal
•
![Page 33: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/33.jpg)
33
Genetic AlgorithmsNotes
• Representation of individuals– Classic approach: individual is a string over a finite
alphabet with each element in the string called a gene– Usually binary instead of AGTC as in real DNA
• Selection strategy– Random– Selection probability proportional to fitness– Selection is done with replacement to make a very fit
individual reproduce several times
• Reproduction– Random pairing of selected individuals– Random selection of cross-over points– Each gene can be altered by a random mutation
![Page 34: 1 CS 2710, ISSP 2610 Chapter 4, Part 2 Heuristic Search](https://reader035.vdocuments.site/reader035/viewer/2022062720/56649f185503460f94c2ed2f/html5/thumbnails/34.jpg)
34
Genetic AlgorithmsWhen to use them?
• Genetic algorithms are easy to apply
• Results can be good on some problems, but bad on other problems
• Always good to give a try before spending time on something more complicated