1
Sections 2.5 and 4.4
Priority Queues and Dijkstra’s Algorithm For Shortest Paths
Some Slides by Kevin Wayne.Copyright © 2005 Pearson-Addison Wesley.All rights reserved.
2
A Problem with Greed Primitive Greedy Algorithm Strategy:
1. Sort the set of items by some criteria2. Select the items in sorted order
Problem: Sometimes the criteria is non-static! Consider:1. Distance from current position to point X changes if I move.2. In cases of uncertainty, the world will change—stock values, political polls, etc.
All issues of optimality aside, these situations make the above greedy approach unnecessarily slow.
In these situations, the strategy becomes: 1. For each choice,
1. Search for the best choice and pick it.2. Update dynamic criteria values O(N2)
O(N LG N)
NEVER FORGETDATA
STRUCTURES AFFECT
RUNTIME!!!
3
Priority Queues: A Greedy Data StructureIf sorting is impractical, and searching is
inevitable, then we need to speed up the search:◦ Binary search trees are good, but use extra memory,
and are fully ordered –We only want to find the smallest value
New abstract data type: Priority Queue◦ Stores items based on some pre-determined key◦ Required operations: Initialize Queue Insert new itemExtract ‘smallest’ itemUpdate a an item’s key
• Find Smallest Item• Delete Item
• Find Item• Change Key
5
Binary HeapsA binary heap is similar to a binary search
tree, but is not fully sortedHeap Property: For any item in a heap,
its key value v must be greater than or equal to its parent’s key value
Usefulness of heap property: ◦ Smallest item is at the root (O(1) time to locate)
Cost of heap property:◦ Adding/deleting/changing keys has a cost for
restoring heap property
6
Array-Based Binary Heap
Requires maximum number of elements to be fixed Never any gaps—all empty space is at the end Root at H[0] For a node at position H[i]:
◦ Left child = H[2*i]◦ Right child = H[2*i + 1]
Parent = H[ ë i /2 û ] For convenience, length(H) = number of items, NOT length of
array
7
Fixing the Heap Property: Heapify-Up
8
Fixing the Heap Property: Heapify-Down
9
Insert ItemSimple strategy:
◦Place new item at the end of the heap◦Fix heap property using Heapify-up
algorithm
10
Delete Item And Extract MinFunction delete_item(H,i):
◦Clear the item from the heap at position i◦Move the right-most item into its place◦If this item is too small, use Heapify-Up to
fix it◦Else if it is too big, use new algorithm
Heapify-Down (next slide) to fix heap property
Function extract_min(H) follows: ◦Return item at root, then delete_item(H,0)
11
Initialize Queue
Filling the Queue (2 scenarios)1. Mostly distinct key values: Use repeated calls to insert_item(H,v),
2. All keys are initially equal: Just copy values into the queue
init_queue(N)return Array of size N
12
Change Key AlgorithmFirst, find the location of the item in
the heap:◦Throw memory at problem to get constant
time Dictionary object location[v] keeps track of the
location of v in the heap Need to gently modify heapify algorithms to
keep track of locationsThen, use heapify-up or heapify-down
depending on the new value
13
Runtime of Heap OperationsEverything depends on Heapify-up or
Heapify-down:◦Both functions are recursive from a specific
start node, and spend constant time at each call
◦Worst case: new key travels full distance between root and leaf
◦Key observation: heaps are always balanced◦Therefore, worst-case performance of
Heapify-up and Heapify-down is O(log2 n)
14
New Greedy Algorithm Form1. Add all items to priority queue PQ
where the key value is the greedy selection criteria
2. While PQ not empty: 1. Take extract_min(PQ) as the next
selection2. For any items whose key values change,
call change_key(PQ,item,value)
4.4 Shortest Paths in a Graph
shortest path from Princeton CS department to Einstein's house
16
Shortest Path ProblemShortest path network.
◦ Directed graph G = (V, E).◦ Source s, destination t.◦ Length e = length of edge e.
Shortest path problem: find shortest directed path from s to t.
Cost of path s-2-3-5-t = 9 + 23 + 2 + 16 = 48.
s
3
t
2
6
7
45
23
18 2
9
14
15 5
30
20
44
16
11
6
19
6
cost of path = sum of edge costs in path
17
Dijkstra's AlgorithmDijkstra's algorithm.
◦ Maintain a set of explored nodes S for which we have determined the shortest path distance d(u) from s to u.
◦ Initialize S = { s }, d(s) = 0.◦ Repeatedly choose unexplored node v which minimizes
add v to S, and set d(v) = (v).
,)(min)(:),( eSuvue
udv
s
v
ud(u)
S
e
shortest path to some u in explored part, followed by a single edge (u, v)
18
Dijkstra's AlgorithmDijkstra's algorithm.
◦ Maintain a set of explored nodes S for which we have determined the shortest path distance d(u) from s to u.
◦ Initialize S = { s }, d(s) = 0.◦ Repeatedly choose unexplored node v which minimizes
add v to S, and set d(v) = (v).
,)(min)(:),( eSuvue
udv
s
v
ud(u)
shortest path to some u in explored part, followed by a single edge (u, v)
S
e
19
Dijkstra's Algorithm: Proof of Correctness
Invariant. For each node u S, d(u) is the length of the shortest s-u path.
Pf. (by induction on |S|) Base case: |S| = 1 is trivial. Inductive hypothesis: Assume true for |S| = k 1.
◦ Let v be next node added to S, and let u-v be the chosen edge.◦ The shortest s-u path plus (u, v) is an s-v path of length (v).◦ Consider any s-v path P. We'll see that it's no shorter than (v).◦ Let x-y be the first edge in P that leaves S,
and let P' be the subpath to x.◦ P is already too long as soon as it leaves S.
(P) (P') + (x,y) d(x) + (x, y) (y) (v)
nonnegativeweights
inductivehypothesis
defn of (y) Dijkstra chose vinstead of y
Ss
y
v
x
P
u
P'
20
Dijkstra's Algorithm: ImplementationFor each unexplored node, explicitly maintain
◦ Next node to explore = node with minimum (v).◦ When exploring v, for each incident edge e = (v, w), update
Efficient implementation. Maintain a priority queue of unexplored nodes, prioritized by (v).
† Individual ops are amortized bounds
PQ OperationInsert
ExtractMinChangeKey
Binary heaplog nlog nlog n
Fib heap †1
log n1
Arraynn1
IsEmpty 1 11
Priority Queue
Total m log n m + n log nn2
Dijkstrannmn
d-way Heapd log d nd log d nlog d n
1m log m/n n
(v) mine (u,v) : u S
d (u) e .
(w) min { (w), (v)e }.
21
Edsger W. DijkstraThe question of whether computers can think is like the question of whether submarines can swim.
Do only what only you can do.
In their capacity as a tool, computers will be but a ripple on the surface of our culture. In their capacity as intellectual challenge, they are without precedent in the cultural history of mankind.
The use of COBOL cripples the mind; its teaching should, therefore, be regarded as a criminal offence.
APL is a mistake, carried through to perfection. It is the language of the future for the programming techniques of the past: it creates a new generation of coding bums.