heapsort and priority queues - cs.anu.edu.au · 1 heapsort and priority queues reading: cormen et...

1

Heapsort and priority queues

Reading: Cormen et al, Chapter 6

A heap is a useful data structure for storing information in a waythat respects an order relation on the information. Conceptually, aheap is a certain kind of binary tree, but it can be efficiently andneatly implemented by an array.

Heapsort is an in-place sorting algorithm that runs in O(n lg n)which uses the heap data structure. Heapsort is an asymptoticallyoptimal sorting algorithm. It only takes a constant amount ofspace outside the input array.

Heaps are also used to efficiently implement priority queues, whichhave many applications.

COMP3600/6466 Algorithms 2018 Lecture 9 1

2

HeapsA (binary) heap is a nearly complete binary tree withoutchild-parent pointers, usually stored as an array. Each node (arrayelement) contains a value (key) and perhaps data associated withthe key. The tree is completely filled on all levels except possiblythe lowest (bottom) level, which is filled from the left.

A heap is stored in an array A[1..A.length]. However, onlyelements in A[1..A.heap size], where 0 ≤ A.heap size ≤ A.length,are valid elements of the heap.

Cormen et al, p.152


3

Heaps 2

The root of the tree is A[1].

The tree structure of each binary heap does not need pointers,since the parent and children of a node are stored in array positionsthat can be easily calculated:

If i is the index of a node, its parent is at index bi/2c, its left childat 2i , and its right child at 2i + 1 (when they exist).

These operations can be done efficiently at the bit level.COMP3600/6466 Algorithms 2018 Lecture 9 3

4

Heaps 3

There are two kinds of heap: max-heaps and min-heaps.

In a max-heap, the max-heap property is satisfied which is that forevery node i , other than the root, the value of a node is at mostthe value of its parent, that is, A[Parent(i)] ≥ A[i ].

The largest element in a max-heap is stored at the root, and thesubtree rooted at a node contains values no larger than thatcontained in the root itself.

In a min-heap, the min-heap property is satisfied which is that forevery node i , other than the root, the value of a node is at leastthe value of its parent, that is, A[Parent(i)] ≤ A[i ].

The smallest element in a min-heap is stored at the root, and thesubtree rooted at a node contains values no smaller than thatcontained in the root itself.


5

Heaps 4We consider the following operations on max-heaps. (There areanalogous operations for min-heaps.)

I Max-Heapify(A, i) – Make the subtree rooted at i into amax-heap, assuming that the subtrees rooted at Left(i) andRight(i) are max-heaps, but A[i ] might be smaller than itschildren, thus violating the max-heap property.

I Build-Max-Heap(A) – Make an array A into a max-heap.

I Heap-Maximum(A) – Return the largest key in a max-heapA.

I Max-Heap-Insert(A, key) – Insert a new key into amax-heap A.

I Heap-Extract-Max(A) – Delete the largest key from amax-heap A and return it.

I Heap-Increase-Key(A, i , key) – Increase the key ofelement i in a max-heap A to key.


6

Maintaining the heap property

When an element of a max-heap violates the max-heap property,which of course means the underlying array is not actually amax-heap at all, the problem can be fixed by repeatedly using oneof the following operations.

I Swap up. If a key is greater than the key of its parent, thenswap them.

I Swap down. If a key is less than the key of one of itschildren, then swap the key with the larger key of the children.

Exercise: What are the maximum and minimum numbers ofelements in a heap of height h?

Exercise: Show that an n-element heap has height blg nc.


7

Max-Heapify

Max-Heapify(A, i) – Make the subtree rooted at i into amax-heap, assuming that the subtrees rooted at Left(i) andRight(i) are max-heaps, but A[i ] might be smaller than itschildren.

Cormen et al, p.154


8

Max-Heapify 2The value 4 in the following figure violates the max-heap property.

Cormen et al, p.155


9

Max-Heapify 3

The running time of Max-Heapify on a subtree of size n rootedat node i is the O(1) time to fix up the relationships among theelements A[i ], A[Left], and A[Right], plus the time to runMax-Heapify on a subtree rooted at one of the children of nodei .

The childrens’ subtrees each have size at most 2n/3 – the extremecase occurs when the bottom level is exactly half full.

Thus we obtain the recurrence

T (n) ≤ T (2n/3) + Θ(1).

Case 2 of the Master Theorem shows that T (n) = O(lg n).

Alternatively, we can say that the running time of Max-Heapifyon a node of height h is O(h).


10

Build-Max-Heap

Build-Max-Heap turns an array of length n into a max-heap.

The idea is to use Max-Heapify in a bottom up manner to buildthe max-heap.

Nodes indexed by bn/2c+ 1 and later are leaves, so they satisfythe max-heap property. Hence Max-Heapify can start from thenode indexed by bn/2c.

Cormen et al, p.157

Exercise: Show that, with the array representation for storing ann-element heap, the leaves are at the nodes indexed bybn/2c+ 1, bn/2c+ 2, . . . , n.


11

Build-Max-Heap 2

Cormen et al, p.158COMP3600/6466 Algorithms 2018 Lecture 9 11

12

Build-Max-Heap 3What is the running time of Build-Max-Heap?

There are at most dn/2h+1e nodes of height h in a heap and theheight of the heap is blg nc.

The time required by Max-Heapify when called on a node ofheight h is O(h).

So the time for Build-Max-Heap is bounded above by

blg nc∑h=1

⌈ n

2h+1

⌉O(h).

We show this expression is O(n).

First, note that∞∑h=0

h

2h=

1/2

(1− 1/2)2= 2. [

∞∑k=0

kxk =x

(1− x)2, for |x | < 1]


13

Build-Max-Heap 3blg nc∑h=1

⌈ n

2h+1

⌉O(h)

≤ C

blg nc∑h=0

⌈ n

2h+1

⌉h, for some C > 0

≤ C

blg nc∑h=0

3n

2h+1h

≤ 3Cn/2

blg nc∑h=0

h

2h

≤ 3Cn/2∞∑h=0

h

2h= 3Cn, for all n ≥ 2

Thus the running time of Build-Max-Heap is O(n).COMP3600/6466 Algorithms 2018 Lecture 9 13

14

Heapsort

Heapsort is an algorithm for sorting an array.

First, the array is turned into a max-heap. Then the maximumelement of the heap is repeatedly removed from the heap and putinto its correct place in the array.

Cormen et al, p.160

The running time of Heapsort is O(n lg n), since the call toBuild-Max-Heap takes time O(n) and each of the n− 1 calls toMax-Heapify takes time O(lg n).


15

Heapsort 2

Cormen et al, p.164


16

Priority Queues

A priority queue is a data structure for maintaining a set S ofelements, each with an associated value called a key. There aretwo types of priority queues: max-priority queues and min-priorityqueues.

A max-priority queue supports the following operations:

I Insert(S , x) inserts the element x into the set S , which isequivalent to the operation S = S ∪ {x}.

I Maximum(S) returns the element of S with the largest key.

I Extract-Max(S) removes and returns the element of Swith the largest key.

I Increase-Key(S , x , k) increases the value of element x ’skey to the new value k , which is assumed to be at least aslarge as x ’s current key value.


17

Priority Queues 2

Priority queue are often implemented using heaps.

Cormen et al, p.163

The time required for Heap-Extract-Max is O(lg n).


18

Priority Queues 3

If increasing a key causes the max-heap property to be violated(since the new key exceeds the key of the parent), then swap upuntil the key is in the correct place.

Cormen et al, p.164

The time required for Heap-Increase-Key is O(lg n).


19

Priority Queues 4

Cormen et al, p.165


20

Priority Queues 5

To add a new element, first expand the heap by adding to the treea leaf whose key is −∞. Then call Max-Increase-Key to setthe key to its correct value and maintain the max-heap property.

Cormen et al, p.164

The time required for Max-Heap-Insert is O(lg n).

In summary, a heap can be used to support all the priority-queueoperations on a set of size n in O(lg n) time.


heapsort and priority queues - cs.anu.edu.au · 1 heapsort and priority queues reading: cormen et...

Documents