CS 294-5: Statistical Natural Language Processing

AnnouncementsHomework 1: SearchHas been released! Part I AND Part II due Monday, 2/3, at 11:59pm.Part I through edX online, instant grading, submit as often as you like.Part II through www.pandagrader.com -- submit pdf

Project 1: SearchWill be released soon! Due Friday 2/7 at 5pm.Start early and ask questions. Its longer than most!

SectionsYou can go to any, but have priority in your own.

Exam preferences / conflictsPlease fill out the survey form (link on Piazza)Due tonight!

AI in the news

TechCrunch, 2014/1/25AI in the news

Wired, 2013/12/12

Wired, 2014/01/16CS 188: Artificial Intelligence

Informed Search

Instructors: Dan Klein and Pieter AbbeelUniversity of California, Berkeley[These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials are available at http://ai.berkeley.edu.]Please retain proper attribution, including the reference to ai.berkeley.edu. Thanks!

5TodayInformed SearchHeuristicsGreedy SearchA* Search

Graph Search

6Recap: Search

Recap: SearchSearch problem:States (configurations of the world)Actions and costsSuccessor function (world dynamics)Start state and goal test

Search tree:Nodes: represent plans for reaching statesPlans have costs (sum of action costs)

Search algorithm:Systematically builds a search treeChooses an ordering of the fringe (unexplored nodes)Optimal: finds least-cost plans

Example: Pancake ProblemCost: Number of pancakes flippedGates, W. and Papadimitriou, C., "Bounds for Sorting by Prefix Reversal.", Discrete Mathematics. 27, 47-57, 1979.

9

Example: Pancake Problem

Gates, W. and Papadimitriou, C., "Bounds for Sorting by Prefix Reversal.", Discrete Mathematics. 27, 47-57, 1979.

10Example: Pancake Problem324332224State space graph with costs as weights34342A* expanded 8100 ; Path cost = 33UCS expanded 25263 . Path cost = 33Greedy expanded 10 . Path cost = 41[0, 7, 5, 3, 2, 1, 4, 6](7, 0, 5, 3, 2, 1, 4, 6)(6, 4, 1, 2, 3, 5, 0, 7)(3, 2, 1, 4, 6, 5, 0, 7)(1, 2, 3, 4, 6, 5, 0, 7)(5, 6, 4, 3, 2, 1, 0, 7)(6, 5, 4, 3, 2, 1, 0, 7)(0, 1, 2, 3, 4, 5, 6, 7)11General Tree Search

Action: flip top twoCost: 2Action: flip all fourCost: 4Path to reach goal:Flip four, flip threeTotal cost: 7You know exactly where you came from and how you got there, but you have no idea where youre going. But, youll know it when you see it.12The One QueueAll these search algorithms are the same except for fringe strategiesConceptually, all fringes are priority queues (i.e. collections of nodes with attached priorities)Practically, for DFS and BFS, you can avoid the log(n) overhead from an actual priority queue, by using stacks and queuesCan even code one implementation that takes a variable queuing object

Uninformed Search

Uniform Cost SearchStrategy: expand lowest path cost

The good: UCS is complete and optimal!

The bad:Explores options in every directionNo information about goal location

StartGoalc 3c 2c 1[Demo: contours UCS empty (L3D1)][Demo: contours UCS pacman small maze (L3D3)]Video of Demo Contours UCS Empty

Video of Demo Contours UCS Pacman Small Maze

Informed Search

Search Heuristics

A heuristic is:A function that estimates how close a state is to a goalDesigned for a particular search problemExamples: Manhattan distance, Euclidean distance for pathing10511.2

19Example: Heuristic Function

h(x)Example: Heuristic FunctionHeuristic: the number of the largest pancake that is still out of place4302333443444h(x)A* expanded 8100 ; Path cost = 33UCS expanded 25263 . Path cost = 33Greedy expanded 10 . Path cost = 41[0, 7, 5, 3, 2, 1, 4, 6](7, 0, 5, 3, 2, 1, 4, 6)(6, 4, 1, 2, 3, 5, 0, 7)(3, 2, 1, 4, 6, 5, 0, 7)(1, 2, 3, 4, 6, 5, 0, 7)(5, 6, 4, 3, 2, 1, 0, 7)(6, 5, 4, 3, 2, 1, 0, 7)(0, 1, 2, 3, 4, 5, 6, 7)21Greedy Search

Example: Heuristic Function

h(x)Greedy SearchExpand the node that seems closest

What can go wrong?

Greedy SearchStrategy: expand a node that you think is closest to a goal stateHeuristic: estimate of distance to nearest goal for each state

A common case:Best-first takes you straight to the (wrong) goal

Worst-case: like a badly-guided DFSbb[Demo: contours greedy empty (L3D1)] [Demo: contours greedy pacman small maze (L3D4)]For any search problem, you know the goal. Now, you have an idea of how far away you are from the goal.25Video of Demo Contours Greedy (Empty)

Video of Demo Contours Greedy (Pacman Small Maze)

A* Search

A* Search

UCSGreedyA*Notes: these images licensed from iStockphoto for UC Berkeley use.29Combining UCS and GreedyUniform-cost orders by path cost, or backward cost g(n)Greedy orders by goal proximity, or forward cost h(n)

A* Search orders by the sum: f(n) = g(n) + h(n)SadbGh=5h=6h=218112h=6h=0ch=73eh=11Example: Teg GrenagerSabceddGGg = 0 h=6g = 1 h=5g = 2 h=6g = 3 h=7g = 4 h=2g = 6 h=0g = 9 h=1g = 10 h=2g = 12 h=030When should A* terminate?Should we stop when we enqueue a goal?

No: only stop when we dequeue a goalSBAG2322h = 1h = 2h = 0h = 3Is A* Optimal?What went wrong?Actual bad goal cost < estimated good goal costWe need estimates to be less than actual costs!AGS13h = 6h = 05h = 7Admissible Heuristics

Idea: Admissibility

Inadmissible (pessimistic) heuristics break optimality by trapping good plans on the fringeAdmissible (optimistic) heuristics slow down bad plans but never outweigh true costsAdmissible HeuristicsA heuristic h is admissible (optimistic) if:

where is the true cost to a nearest goal

Examples:

Coming up with admissible heuristics is most of whats involved in using A* in practice.

4

15Optimality of A* Tree Search

Optimality of A* Tree SearchAssume:A is an optimal goal nodeB is a suboptimal goal nodeh is admissible

Claim:A will exit the fringe before B

Optimality of A* Tree Search: BlockingProof:Imagine B is on the fringeSome ancestor n of A is on the fringe, too (maybe A!)Claim: n will be expanded before Bf(n) is less or equal to f(A)

Definition of f-cost

Admissibility of h

h = 0 at a goal

Optimality of A* Tree Search: BlockingProof:Imagine B is on the fringeSome ancestor n of A is on the fringe, too (maybe A!)Claim: n will be expanded before Bf(n) is less or equal to f(A)f(A) is less than f(B)

B is suboptimalh = 0 at a goal

Optimality of A* Tree Search: BlockingProof:Imagine B is on the fringeSome ancestor n of A is on the fringe, too (maybe A!)Claim: n will be expanded before Bf(n) is less or equal to f(A)f(A) is less than f(B) n expands before BAll ancestors of A expand before BA expands before BA* search is optimal

Properties of A*Properties of A*bbUniform-CostA*UCS vs A* ContoursUniform-cost expands equally in all directions

A* expands mainly toward the goal, but does hedge its bets to ensure optimalityStartGoalStartGoal[Demo: contours UCS / greedy / A* empty (L3D1)][Demo: contours A* pacman small maze (L3D5)]43Video of Demo Contours (Empty) -- UCS

Video of Demo Contours (Empty) -- Greedy

Video of Demo Contours (Empty) A*

Video of Demo Contours (Pacman Small Maze) A*

Comparison

GreedyUniform CostA*A* Applications

A* ApplicationsVideo gamesPathing / routing problemsResource planning problemsRobot motion planningLanguage analysisMachine translationSpeech recognition[Demo: UCS / A* pacman tiny maze (L3D6,L3D7)][Demo: guess algorithm Empty Shallow/Deep (L3D8)]

Video of Demo Pacman (Tiny Maze) UCS / A*

Video of Demo Empty Water Shallow/Deep Guess Algorithm

Creating Heuristics

Creating Admissible HeuristicsMost of the work in solving hard search problems optimally is in coming up with admissible heuristics

Often, admissible heuristics are solutions to relaxed problems, where new actions are available

Inadmissible heuristics are often useful too

15

366Example: 8 PuzzleWhat are the states?How many states?What are the actions?How many successors from the start state?What should the costs be?

Start StateGoal State

Actions8 Puzzle IHeuristic: Number of tiles misplacedWhy is it admissible?h(start) =This is a relaxed-problem heuristic8Average nodes expanded when the optimal path has4 steps8 steps12 stepsUCS1126,3003.6 x 106TILES1339227

Start StateGoal StateStatistics from Andrew Moore8 Puzzle IIWhat if we had an easier 8-puzzle where any tile could slide any direction at any time, ignoring other tiles?

Total Manhattan distance

Why is it admissible?

h(start) =

3 + 1 + 2 + = 18Average nodes expanded when the optimal path has4 steps8 steps12 stepsTILES1339227MANHATTAN122573

Start StateGoal State8 Puzzle IIIHow about using the actual cost as a heuristic?Would it be admissible?Would we save on nodes expanded?Whats wrong with it?

With A*: a trade-off between quality of estimate and work per nodeAs heuristics get closer to the true cost, you will expand fewer nodes but usually do more work per node to compute the heuristic itself

Semi-Lattice of HeuristicsTrivial Heuristics, DominanceDominance: ha hc if

Heuristics form a semi-lattice:Max of admissible heuristics is admissible

Trivial heuristicsBottom of lattice is the zero heuristic (what does this give us?)Top of lattice is the exact heuristic

Semi-lattice: x