Problem Solving Using Search

Reduce a problem to one of searching a graph.View problem solving as a process of moving through a sequence of problem states to reach a goal stateMove from one state to another by taking an actionA sequence of actions and states leading to a goal state is a solution to the problem.

Trees

A tree is made up of nodes and links connected so that there are no loops (cycles).

Nodes are sometimes called vertices.Links are sometimes called edges.

A tree has a root node.Where the tree ”starts”.

Every node except the root has a single parent (aka direct ancestor).An ancestor node is a node that can be reached by repeatedly going to a parent.Each node (except a terminal, aka leaf) has one or more children (aka direct descendants).A descendant node is a node that can be reached by repeatedly going to a child.

Graphs

Set of nodes connected by links.But, unlike trees, loops are allowed.Also, unlike trees, multiple parents are allowed.Two kinds of graphs:

Directed graphs.Links have a direction.

Undirected graphs.Links have no direction.

A tree is a special case of a graph.

Representing Problems with Graphs

Nodes represent cities that are connected by direct flight.

Find route from city A to city B that involves the fewest hops.

Nodes represent a state of the world.Which blocks are on top of what in a blocks scene. The links represent actions that result in a change from one state to the other.A path through the graph represents a plan of action.

A sequence of steps that tell how to get from an initial state to a goal state.

Problem Solving with Graphs

Assume that each state is complete.Represents all (and preferably only) relevant aspects of the problem to be solved.

In the flight planning problem, the identity of the airport is sufficient.But the address of the airport is not necessary.

Assume that actions are deterministic.We know exactly the state after an action has been taken.

Assume that actions are discrete.We don’t have to represent what happens while the action is happening.

We assume that a flight gets us to the scheduled destination without caring what happens during the flight.

Classes of Search

Uninformed, Any-pathDepth-first Breadth-firstLook at all nodes in a search tree in a specific order independent of the goal.Stop when the first path to a goal state is found.

Informed, Any-pathExploit a task specific measure of goodness to try to reach a goal state more quickly.

Classes of Search

Uninformed, optimalGuaranteed to find the ”best” path

As measured by the sum of weights on the graph edges

Does not use any information beyond what is in the graph definition

Informed, optimalGuaranteed to find the best path

Exploit heuristic (”rule of thumb”) information to find the path faster than uninformed methods

Scoring Function

Assigns a numerical value to a board positionThe set of pieces and their locations represents a singel state in the game

Represents the likelihood of winning from a given board positionTypical scoring function is linear

A weighted sum of features of the board positionEach feature is a number that measures a specific characteristic of the position.

”Material” is some measure of which pieces one has in a given position.A number that represents the distribution of the pieces in a position.

Scoring Function

To determine next move:Compute score for all possible next positions.Select the one with the highest score.

If we had a perfect evaluation function, playing chess would be easy!

Such a function exists in principleBut, nobody knows how to write it or compute it directly.

Min-Max Algorithm

Limited look-ahead plus scoringI look ahead two moves (2-ply)

First me – relative level 1Then you – relative level 2

For each group of children at level 21. Check to see which has the minimum score2. Assign that number to the parent

Represents the worst that can happen to me after your move from that parent position

3. I pick the move that lands me in the position where you can do the least damage to me.

This is the position which has the maximum value resulting from applying Step 1.

Can implement this to any number (depth) of min-max level pairs.

Alpha-Beta Pruning

Pure optimization of min-max.No tradeoffs or approximations.

Don’t examine more states than is necessary.

”Cutoff” moves allow us to cut off entire branches of the search tree (see following example)

Only 3 states need to be examined in the following examleTurns out, in general, to be very effective

Move Generation

Assumption of ordered tree is optimistic.

”Ordered” means to have the best move on the left in any set of child nodes.

Node with lowest value for a min node.Node with highest value for a max node.

If we could order nodes perfectly, we would not need alpha-beta search!The good news is that in practice performance is close to optimistic limit.

Move Generator

Goal is to produce ordered movesEncodes a fair bit of knowledge about a game.Example order heuristics:

Value of captured piece – value of attacker.E.g., ”pawn takes Queen” is the highest ranked move in this ordering

Killer Heuristic Keep track of cutoff moves at each level of search Try those first when considering subsequent moves at the same levelBased on idea that many moves are inconsequentialE.g., if your queen is en prise, it doesn't matter whether you advance your pawn at H2 by one or two squares

The opponent will still take the queen. Therefore, if the move "bishop takes queen" has caused a cutoff during the examination of move H2-H3, it might also cause one during the examination of H2-H4, and should be tried first.

Static Evaluation

Other place where substantial game knowledge is encodedIn early programs, evaluation functions were complicated and buggyIn time it was discovered that you could get better results by

A simple reliable evaluatorE.g., a weighted count of pieces on the board.

Plus deeper search

Static Evalution

Deep Blue used static evaluation functions of medium complexity

Implemented in hardware

”Cheap” PC programs rely on quite complex evaluation functions.

Can’t search as deeply as Big Blue

In general there is a tradeoff between Complexity of evaluation functionDepth of search.

TD-Gammon

Neural network that is able to teach itself to play backgammon solely by playing against itself and learning from the results Based on the TD(Lambda) reinforcment learning algorithm Starts from random initial weights (and hence random initial strategy)With zero knowledge built in at the start of learning (i.e. given only a "raw" description of the board state), the network learns to play at a strong intermediate levelWhen a set of hand crafted features is added to the network's input representation, the result is a truly staggering level of performanceThe latest version of TD Gammon is now estimated to play at a strong master level that is extremely close to the world's best human players.