Heaps

Chapter 6 in CLRS

Motivation

• Dijkstra’s algorithm for single source shortest path

• Prim’s algorithm for minimum spanning trees

Motivation

• Want to find the shortest route from New York to San Francisco

• Model the road-map with a graph

A Graph G=(V,E)

V is a set of vertices

E is a set of edges (pairs of vertices)

Model driving distances by weights on the edges

2 19

9

1

5

13

1710

148

21

V is a set of vertices

E is a set of edges (pairs of vertices)

Source and destination

V is a set of vertices

E is a set of edges (pairs of vertices)

2 19

9

1

5

13

1710

148

21

Dijkstra’s algorithm

• Assume all weights are non-negative

• Finds the shortest path from some fixed vertex s to every other vertex

s

s15

103

1

4

2s3

s2

s4

15 8

Example

Maintain an upper bound d(v) on the shortest path to v

0

∞

∞

∞

∞

s

s15

103

1

4

2s3

s2

s4

15 8

Example

s

s15

103

1

4

2s3

s2

s4

15 8

Maintain an upper bound d(v) on the shortest path to v

0

∞

∞

∞

∞

A node is either scanned (in S) or labeled (in Q)

Initially S = and Q = V

s

s15

103

1

4

2s3

s2

s4

15 8

Pick a vertex v in Q with minimum d(v) and add it to S

0

∞

∞

∞

∞

Initially S = and Q = V

s

s15

103

1

4

2s3

s2

s4

15 8

Pick a vertex v in Q with minimum d(v) and add it to S

0

∞

∞

∞

∞

Initially S = and Q = V

v

w15

For every edge (v,w) where w in Q relax(v,w)

s

s15

103

1

4

2s3

s2

s4

15 8

Relax(v,w)

If d(v) + w(v,w) < d(w) then d(w) ← d(v) + w(v,w) π(w) ← v

0

∞

∞

∞

∞

v

w

For every edge (v,w) where w in Q relax(v,w)

15

s

s15

103

1

4

2s3

s2

s4

15 8

0

∞

∞

∞

∞

S = {s}

Relax(s,s4)

s

s15

103

1

4

2s3

s2

s4

15 8

0

∞

∞

15

∞

Relax(s,s4)

S = {s}

s

s15

103

1

4

2s3

s2

s4

15 8

0

∞

∞

15

∞

Relax(s,s4)Relax(s,s3)

S = {s}

s

s15

103

1

4

2s3

s2

s4

15 8

0

∞

∞

15

10

Relax(s,s4)Relax(s,s3)

S = {s}

s

s15

103

1

4

2s3

s2

s4

15 8

0

∞

∞

15

10

Relax(s,s4)Relax(s,s3)Relax(s,s1)

S = {s}

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

∞

15

10

Relax(s,s4)Relax(s,s3)Relax(s,s1)

S = {s}

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

∞

15

10

S = {s}

Pick a vertex v in Q with minimum d(v) and add it to S

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

∞

15

10

S = {s,s1}

Pick a vertex v in Q with minimum d(v) and add it to S

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

∞

15

10

S = {s,s1}

Pick a vertex v in Q with minimum d(v) and add it to S

Relax(s1,s3)

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

∞

15

9

S = {s,s1}

Pick a vertex v in Q with minimum d(v) and add it to S

Relax(s1,s3)

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

∞

15

9

S = {s,s1}

Pick a vertex v in Q with minimum d(v) and add it to S

Relax(s1,s3)Relax(s1,s2)

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

6

15

9

S = {s,s1}

Pick a vertex v in Q with minimum d(v) and add it to S

Relax(s1,s3)Relax(s1,s2)

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

6

15

9

S = {s,s1}

Pick a vertex v in Q with minimum d(v) and add it to S

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

6

15

9

S = {s,s1,s2}

Pick a vertex v in Q with minimum d(v) and add it to S

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

6

15

9

S = {s,s1,s2}

Pick a vertex v in Q with minimum d(v) and add it to S

Relax(s2,s3)

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

6

15

8

S = {s,s1,s2}

Pick a vertex v in Q with minimum d(v) and add it to S

Relax(s2,s3)

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

6

15

8

S = {s,s1,s2}

Pick a vertex v in Q with minimum d(v) and add it to S

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

6

15

8

S = {s,s1,s2,s3}

Pick a vertex v in Q with minimum d(v) and add it to S

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

6

15

8

S = {s,s1,s2,s3,s4}

Pick a vertex v in Q with minimum d(v) and add it to S

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

6

15

8

S = {s,s1,s2,s3,s4}

When Q = then the d() values are the distances from s

The π function gives the shortest path tree

s

s15

103

1

4

2s3

s2

s4

15 8

0

5

6

15

8

S = {s,s1,s2,s3,s4}

When Q = then the d() values are the distances from s

The π function gives the shortest path tree

Implementation of Dijkstra’s algorithm

• We need to find efficiently the vertex with minimum d() in Q

• We need to update d() values of vertices in Q

Required ADT

• Maintain items with keys subject to

• Insert(x,Q)• min(Q)• Deletemin(Q)• Decrease-key(x,Q,Δ)

Required ADT

• Insert(x,Q)• min(Q)• Deletemin(Q)• Decrease-key(x,Q,Δ): Can simulate

by Delete(x,Q), insert(x-Δ,Q)

• Insert(x,Q)• min(Q)• Deletemin(Q)• Decrease-key(x,Q,Δ): Can simulate

by Delete(x,Q), insert(x-Δ,Q)

How many times we do these operations ?

n = |V|

n

n

m = |E|

Do we know an algorithm for this ADT ?

• Insert(x,Q)• min(Q)• Deletemin(Q)• Decrease-key(x,Q,Δ): Can simulate

by Delete(x,Q), insert(x-Δ,Q)Yes! Use balanced binary search trees (RB-trees)

Heap

• Heap is A complete binary tree The items at the children of v are

greater than the one at v

Heap-ordered tree

23

7

17

3365

1924 2629

7940

• Representing a heap with an array Get the elements from top to

bottom, from left to right

Array Representation

23

7

17

3365

1924 2629

7 23 17 24 29 26 19 65 33 40 79

7940

Q

Array Representation

23

7

17

3365

1924 2629

7 23 17 24 29 26 19 65 33 40 79

7940

• Left(i): 2i+1• Right(i): 2i+2• Parent(i): (i-1)/2

Q

Find the minimum

23

7

17

3365

1924 2629

7 23 17 24 29 26 19 65 33 40 79

7940

• Return Q[0]

Q

Delete the minimum

23

7

17

3365

1924 2629

7 23 17 24 29 26 19 65 33 40 79

7940

Q

Delete the minimum

23 17

3365

1924 2629

23 17 24 29 26 19 65 33 40 79

7940

Q

Delete the minimum

23

79

17

3365

1924 2629

79 23 17 24 29 26 19 65 33 40

40

Q

Delete the minimum

23

17

79

3365

1924 2629

17 23 79 24 29 26 19 65 33 40

40

Q

Delete the minimum

23

17

19

3365

7924 2629

17 23 19 24 29 26 79 65 33 40

40

Q

Delete the minimum

23

17

19

3365

7924 2629

17 23 19 24 29 26 79 65 33 40

40

Q

Q[0] ← Q[size(Q)-1]

Heapify-down(Q,0)

size(Q) ← size(Q)-1

Heapify-down(Q,i)• Heapify-down(Q, i)• l ← left(i)• r ← right(i)• smallest ← i• if l < size(Q) and Q[l] < Q[smallest]

then smallest ← l• if r < size(Q) and Q[r] < Q[smallest]

then smallest ← r• if smallest > i then Q[i] ↔ Q[smallest]• Heapify-down(Q,

smallest)

Inserting an item

23

17

19

3365

7924 2629

17 23 19 24 29 26 79 65 33 40

40

Q

Insert(15,Q)

Inserting an item

23

17

19

3365

7924 2629

17 23 19 24 29 26 79 65 33 40 15

40

Q

Insert(15,Q)Q[size(Q)] ← 15

15

Heapify-up(Q,size(Q))size(Q) ← size(Q) + 1

Inserting an item

23

17

19

3365

7924 2615

17 23 19 24 15 26 79 65 33 40 29

40

Q

29

Inserting an item

15

17

19

3365

7924 2623

17 15 19 24 23 26 79 65 33 40 29

40

Q

29

Inserting an item

17

15

19

3365

7924 2623

15 17 19 24 23 26 79 65 33 40 29

40

Q

29

Q

Heapify-up

• Heapify-up(Q, i)• while i > 0 and Q[i] <

Q[parent(i)]• do Q[i] ↔ Q[parent(i)]• i ← parent(i)

17

15

19

3365

7924 2623

40 29

15 17 19 24 23 26 79 65 33 40 29

Other operations

• Decrease-key(x,Q,Δ)• Delete(x,Q)

• Can implement them easily using heapify

Heapsort (Williams, Floyd, 1964)

• Put the elements in an array• Make the array into a heap• Do a deletemin and put the

deleted element at the last position of the array

Put the elements in the heap

65

79

26

3323

2924 1519

79 65 26 24 19 15 29 23 33 40 7

740

Q

65

79

26

3323

2924 1519

79 65 26 24 19 15 29 23 33 40 7

740

Q

Make the elements into a heap

Make the elements into a heap

65

79

26

3323

2924 1519

79 65 26 24 19 15 29 23 33 40 7

740

Q

Heapify-down(Q,4)

65

79

26

3323

2924 157

79 65 26 24 7 15 29 23 33 40 19

1940

Q

Heapify-down(Q,4)

65

79

26

3323

2924 157

79 65 26 24 7 15 29 23 33 40 19

1940

Q

Heapify-down(Q,3)

65

79

26

3324

2923 157

79 65 26 23 7 15 29 24 33 40 19

1940

Q

Heapify-down(Q,3)

65

79

26

3324

2923 157

79 65 26 23 7 15 29 24 33 40 19

1940

Q

Heapify-down(Q,2)

65

79

15

3324

2923 267

79 65 15 23 7 26 29 24 33 40 19

1940

Q

Heapify-down(Q,2)

65

79

15

3324

2923 267

79 65 15 23 7 26 29 24 33 40 19

1940

Q

Heapify-down(Q,1)

7

79

15

3324

2923 2665

79 7 15 23 65 26 29 24 33 40 19

1940

Q

Heapify-down(Q,1)

7

79

15

3324

2923 2619

79 7 15 23 19 26 29 24 33 40 65

6540

Q

Heapify-down(Q,1)

7

79

15

3324

2923 2619

79 7 15 23 19 26 29 24 33 40 65

6540

Q

Heapify-down(Q,0)

79

7

15

3324

2923 2619

7 79 15 23 19 26 29 24 33 40 65

6540

Q

Heapify-down(Q,0)

19

7

15

3324

2923 2679

7 19 15 23 79 26 29 24 33 40 65

6540

Q

Heapify-down(Q,0)

19

7

15

3324

2923 2640

7 19 15 23 40 26 29 24 33 79 65

6579

Q

Heapify-down(Q,0)

23

7

15

3379

2924 2619

6540

We have at most n/2 nodes heapified at height 1

How much time does it take to build the heap this way ?

We have at most n/4 nodes heapified at height 2We have at most n/8 nodes heapified at height 3Total time =

23

7

15

3379

2924 2619

6540

We have at most n/2 nodes heapified at height 1

How much time does it take to build the heap this way ?

We have at most n/4 nodes heapified at height 2We have at most n/8 nodes heapified at height 3Total time =

Summary

• We can build the heap in linear time

• We still have to deletemin the elements one by one in order to sort that will take O(nlog(n))

An example

• Given an array of length n, find the k smallest numbers.

77