Download - 04- Adversarial Search AI
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E
CSE 481ARTIFICIAL INTELLIGENCE
ADVERSARIAL SEARCH AND GAMES
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E22
Conventional Search
In conventional search
There is no adversary to consider
A solution is a method for finding goal
Heuristic techniques
Could find an optimal solution but not always
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E33
Games: Adversarial Search
• Games are adversarial
– A solution involves strategy that specifies a move for every
possible opponent reply
– We can find an exact solution
• Using search with alternating player moves and assuming an
opponent playing perfectly
• The search explores the game tree to the conclusion and selects
the move with the guaranteed best outcome
– Time limits may force an approximate solution
• A heuristic evaluation function
– Replaces the utility function
• It evaluates the goodness of a game position
– Examples: chess, checkers … etc.
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E44
The Game Prototype
• Formulating games as search problems
1. Initial State
• The board position + the player to move
2. Successor Function
• A list of (move, state) pairs specifying legal moves
3. Terminal Test
• Detects a terminal state (i.e., detects if the game is finished)
4. Utility Function
• Returns a numerical value for terminal states
• Possibly as simple as 1 for win, 0 for lose, and -1 for draw
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E55
Game Search & Game Trees
From the initial state and successor function
We generate the game tree
Each turn is 1 ply (a layer of nodes representing states)
We use the search tree to determine the next move:
1. Search to terminal states
2. Use their utility function values + assumptions about opponent
moves
3. Pick the optimal next move
Example: a complete tree for an example game called
NIM (http://illuminations.nctm.org/ActivityDetail.aspx?ID=140)
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E66
A very small game: NIM Game Rules
Consider a game called NIM
It is played by two players
In this game a number of stones are placed in a pile
In each player’s turn, player can remove either 1 or 2
stones from the pile
The player who removes the last stone wins
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E77
A very small game: NIM Game Representation
Consider the game tree for
the 5 stone version of NIM.
[VAL] represents the first
player (We) node or MAX
node
(VAL) represents the
Opponent node or MIN
node
VAL is the current number
of stones in the pile
We start the game
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E88
A very small game: NIM Node Evaluation
If a node corresponds to MAXmove and the value is [0], thenthat node indicates a Lose forMAX and a utility value of 0 isassigned to this node
Similarly a node with (0) is aWin for MAX and a utility valueof 1 is assigned to this node
To propagate these values backup the list from bottom up asfollow:
if we are evaluating a MAX node, findthe maximum of the utility function ofall branches going out from this node
if we are evaluating a MIN node findthe minimum of the utility function ofall branches going out from this node
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E99
Optimal Strategies: minimax
minimax values for a simple 2-ply example
here's the game tree
with terminal values from a utility function
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E1010
Optimal Strategies: minimax
minimax values for a simple 2-ply example
now we back up the values for min nodes
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E1111
Optimal Strategies: minimax
minimax values for a simple 2-ply example
then we back up values for max node
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E1212
Optimal Strategies: minimax
The optimal play assumption
Optimal play for MAX
Assumes that MIN also plays optimally
i.e. maximizes the worst-case outcome for MAX
What if the assumption doesn't hold
What if MIN does not play optimally?
In this case, MAX will do even better
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E1313
Extending minimax
• Pruning the game search tree
– The number of game states is
• Exponential in the number of moves
– Thus, for real non-trivial games
• There are just too many nodes
– So, how to avoid examining every node in the tree?
• We can still compute the correct minimax decision without
expanding all the nodes
– We can prune the game tree by not expanding portions that
wouldn't make any difference (- pruning algorithm)
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E1414
α-β pruning example
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E1515
α-β pruning example
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E1616
α-β pruning example
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E1717
α-β pruning example
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E1818
α-β pruning example
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E1919 - Pruning Algorithm
Idea: if current path is already worse than some
other known path, stop expanding it.
Improves the speed of the mini-max search
algorithm
is the max lower bound of all solutions. i.e. the value of
the best move so far at any choice point on MAX path
is the min upper bound of all solutions. i.e. value for the
best move so far at any choice point on MIN path
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E2020 - Pruning Algorithm Example
1) Setup phase: Assign to each left-most internal node of the tree,
variables: = -, = +
MAX
MIN
MAX
MIN
MAX
=-
=+
=-
=+
=-
=+
=-
=+
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E2121 - Pruning Algorithm Example
2) Look at first computed final configuration value. It’s 3. Parent is a MIN node, so
set the (min) value to 3
MAX
MIN
MAX
MIN
MAX3
=-
=+
=-
=+
=-
=+
=-
= 3
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E2222 - Pruning Algorithm Example
3) Look at next value, 5. Since parent is a MIN node, we want the minimum of
3 and 5 which is 3. Parent MIN node is done – fill (max) value of its parent MAX
node. Always set for MAX nodes and for MIN nodes. Copy the state of the MAX parent
node into the second unevaluated MIN child
3
3 5
=-
=+
=-
=+
= 3
=+
=-
= 3
= 3
=+
MAX
MIN
MAX
MIN
MAX
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E2323 - Pruning Algorithm Example
4) Look at next value, 2. Since parent node is MIN with =+, 2 is smaller, change
3
3 5 2
=-
=+
=-
=+
= 3
=+
=-
= 3
= 3
= 2
MAX
MIN
MAX
MIN
MAX
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E2424 - Pruning Algorithm Example
5) Now, the MIN parent node has a max value of 3 and MIN value of 2. The value of the 2nd child
does not matter. if it is >2, 2 will be selected for MIN node. If it is <2, it will be selected for MIN node,
but since it is <3 it will not get selected for the parent max node. Thus, we prune the right sub-tree of
the MIN node. Propagate MAX value up the tree
3
3 2
3 5 2
=-
=+
=-
=+
= 3
=+
=-
= 3
= 3
= 2
MAX
MIN
MAX
MIN
MAX
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E2525 - Pruning Algorithm Example
6) MAX node is now done and we can set the value of its parent and propagate node
state to sibling sub-tree’s left-most path
3
3 2
3 5 2
=-
=+
=-
= 3
= 3
=+
=-
= 3
= 3
= 2
=-
= 3
=-
= 3
MAX
MIN
MAX
MIN
MAX
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E2626 - Pruning Algorithm Example
7) The next node is 10. 10 is not smaller than 3, so state of parent does not change. We
still have to look at the 2nd child since is still –
3
3 2
3 5 2 10
=-
=+
=-
= 3
= 3
=+
=-
= 3
= 3
= 2
=-
= 3
=-
= 3
MAX
MIN
MAX
MIN
MAX
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E2727 - Pruning Algorithm Example
8) The next node is 4. Smallest value goes to the parent MIN node. MIN sub-tree is
done, so the parent MAX node gets the (max) value from the child. Note that if the
MAX node had a 2nd sub-tree, we can prune it since >
3 4
3 2 4
3 5 2 10 4
=-
=+
=-
= 3
= 3
=+
=-
= 3
= 3
= 2
= 4
= 3
=-
= 3
MAX
MIN
MAX
MIN
MAX
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E2828 - Pruning Algorithm Example
9) Continue propagating value up the tree, modifying the corresponding and values.
Also, propagate the state of root node down the left-most path of the right sub-tree
3 4
3
3 2 4
3 5 2 10 4
= 3
=+
=-
= 3
= 3
=+
=-
= 3
= 3
= 2
= 4
= 3
=-
= 3
= 3
=+
= 3
=+
= 3
=+
MAX
MIN
MAX
MIN
MAX
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E2929 - Pruning Algorithm Example
10) Next value is a 2. We set the (min) value of the MIN parent to 2. Since no other
children exist, we propagate the value up the tree
3 4 2
3
3 2 4 2
3 5 2 10 4 2
= 3
=+
=-
= 3
= 3
=+
=-
= 3
= 3
= 2
= 4
= 3
=-
= 3
= 3
=+
= 3
=+
= 3
= 2
MAX
MIN
MAX
MIN
MAX
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E3030 - Pruning Algorithm Example
11) We have a value for the 3rd level MAX node, now we can modify the (min) value of
the MIN parent to 2. Now, we have a situation that > and thus the value of the
rightmost sub-tree of the MIN node does not matter, so we prune the whole sub-tree
3 4 2
3
3 2 4 2
3 5 2 10 4 2
= 3
=+
=-
= 3
= 3
=+
=-
= 3
= 3
= 2
= 4
= 3
=-
= 3
= 3
= 2
= 3
=+
= 3
= 2
MAX
MIN
MAX
MIN
MAX
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E3131 - Pruning Algorithm Example
12) Finally, no more nodes remain, we propagate values up the tree. The root has a value of 3 that
comes from the left-most child. Thus, the player should choose the left-most child’s move in order to
maximize his/her winnings. The result is the same as with the original minimax algorithm, but we did not
visit all nodes of the tree
3
3 4 2
3 2
3 2 4 2
3 5 2 10 4 2
= 3
=+
=-
= 3
= 3
=+
=-
= 3
= 3
= 2
= 4
= 3
=-
= 3
= 3
= 2
= 3
=+
= 3
= 2
MAX
MIN
MAX
MIN
MAX
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E3232
Comments on - Pruning Algorithm
Pruning does not change the eventual results though
entire sub-trees can be pruned
Its effectiveness depends partly on
– The order of examining successors
– Perfect ordering would be best successors first which allows
earliest pruning
• With "perfect ordering" time complexity is O(bm/2)
The branching factor reduces to sqrt(b)
From an alternative view, given - pruning and a fixed
amount of time we can look twice as far ahead as with simple
minimax
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E3333
Dealing with games with large game trees
What to do when the game tree is too large
Complete search is not feasible
We can't guarantee optimal play
Replace the utility function with a heuristic evaluation
function that assigns an estimate to intermediate nodes
without going all the way down to the terminal nodes
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E3434
Non-Optimal Game Trees
Heuristic evaluation function
Evaluation function should
1. order terminal states as the utility function would
2. be economical to compute
3. when applied to non-terminals
– Be highly correlated with the actual chance of winning
This introduces uncertainty about the outcome
This uncertainty is due to computational limitations
The game is still characterized by complete information
CS
E 4
81
: A
RT
IF
IC
IA
L I
NT
EL
LI
GE
NC
E3535
Non-Optimal Game Trees
Cutoff depth and heuristic evaluation function
The simplest cutoff is a fixed depth limit
Chosen to allow a move choice within the time limit
A more robust approach
Uses iterative deepening
Selects its move from deepest search completed in time