outline

Outline

• Priority Queues

• Binary Heaps

• Randomized Mergeable Heaps

Priority Queues• A priority queue stores a collection of Comparable elements

• Operations:

• add(x) : Add x to the collection

• findMin() : Return the smallest element in the collection• deleteMin() : Remove the smallest element in the collection

• In the JCF Queue interface, these are:

• add(x)/offer(x) : add(x)

• peek() : findMind()

• remove()/poll() : deleteMin()

Priority Queues: Applications

• We can sort a set of elements using a priority queue. How?

public static <T> void sort (T[] a) {Queue <T> pq = new PriorityQueue <T >();

for (T x : a)pq. add(x);

for (int i = 0; i < a. length ; i++)a[i] = pq. remove ();}

• We just need to insert them all into a priority queue and then removing them

Priority Queues: Applications

• In simulations, priority queues store events ordered by time of occurrence

Priority Queues

• Which structure should we use to implement a priority queue?

• Priority queues can be implemented as (binary) heaps• A binary heap is a binary tree where each node u stores a value u.prio• Heap property:

• u.prio < u.left.prio and u.prio < u.right.prio

Priority Queues

• Heap property:

• u.prio < u.left.prio and u.prio < u.right.prio

• A binary heap is a binary tree where each node u stores a value u.prio

5.19.21.32.1

1.10.9

0.5 0.3

0.1

1.9

0.40.8

Priority Queues

• A complete binary tree of height d has up to 2d+1 - 1 nodes• To store n nodes, we require 2d+1 - 1 ≥ n

• A complete heap uses a complete binary tree:

5.19.21.32.1

1.10.9

0.5 0.3

0.1

1.9

0.40.8

• 2d+1 ≥ n + 1

• d+1 ≥ log2(n + 1)

• d ≥ log2(n + 1) – 1 = O(log n)• A complete heap of size n has height O(log n)

Priority Queues

• How can we maps the nodes of a complete binary tree to the positions of an array?

• Eytzinger method:• a[0] is the root

• the left child of a[i] is a[2*i+1]

• the right child of a[i] is a[2*i+2]

• the parent of a[i] is a[(i-1)/2]

Eytzinger method

• Eytzinger method:

10987

43

1 2

0

11

65

141312

112 3 4 5 6 7 8 9

100 1

13

12

14

Eytzinger method


10987

43

1 2

0

11

65

141312

112 3 4 5 6 7 8 9

100 1

13

12

14

rootroot Left childLeft child Right childRight child

a[2*0+1] = a[1]a[2*0+1] = a[1]

a[2*0+2]= a[2]a[2*0+2]= a[2]

i = 0i = 0

Eytzinger method


10987

43

1 2

0

11

65

141312

lc d e f g h i j ka b nm o

Left childLeft child Right childRight child

a[2*5+1] = a[11]a[2*5+1] = a[11] a[2*0+2]=

a[12]a[2*0+2]= a[12]

i = 5i = 5

Implicit Binary Heap

• An implicit binary heap represents a complete binary heap in an array a using the Eytzinger method

1.90.3 0.9 1.1 0.8 0.4 2.1 1.3 9.2 5.10.10.5

5.19.21.32.1

1.10.9

0.5 0.3

0.1

1.9

0.40.8

• No extra pointers, all the data lives in a

Implicit Binary Heappublic class BinaryHeap <T extends Comparable <T>>

extends AbstractQueue <T> {

protected T[] a;protected int n;

protected int left (int i) { return 2*i + 1;}

protected int right (int i) { return 2*i + 2;}

protected int parent (int i) { return (i -1)/2;} ...}

Finding the minimum

public T peek () {

return a [0];

}

• Finding the minimum value in a heap is easy

• It's stored at the root

• This is a[0]

• Runs in O(1) time

• How to find the minimum value in a heap?

Inserting elements

• To insert x into a (implicit binary) heap, we

1. add x as a leaf

2. while x is smaller than parent(x)

• swap x with its parent

• Runs in O(log n) time

• How to insert an element into a heap?

1.9

0.3

0.9

1.1

0.8

0.4

2.1

1.3

9.2

5.1

0.1

0.5

5.19.21.32.1

1.10.9

0.5 0.3

0.1

1.9

0.40.8

0.2

0.2

1. add x as a leaf

0.20.2

Inserting elements

Inserting elements

1.9

0.3

0.9

1.1

0.2

0.4

2.1

1.3

9.2

5.1

0.1

0.5

5.19.21.32.1

1.10.9

0.5 0.3

0.1

1.9

0.40.2

0.8

0.8

1. add x as a leaf2. while x is smaller than parent(x)


0.20.2

Inserting elements

1.9

0.2

0.9

1.1

0.3

0.4

2.1

1.3

9.2

5.1

0.1

0.5

5.19.21.32.1

1.10.9

0.5 0.2

0.1

1.9

0.40.3

0.8

0.8

1. add x as a leaf2. while x is smaller than parent(x)


0.20.2

Inserting elements

public boolean offer (T x) {if (n + 1 > a. length )grow ();a[n++] = x;bubbleUp (n -1);return true ;

}protected void bubbleUp (int i) {

int p = parent (i);while (i > 0 && a[i]. compareTo (a[p]) < 0) {swap (i,p);i = p;p = parent (i);}

}

Removing the minimum

• To delete the minimum from a (implicit binary) heap, we

1. copy x=a[n-1] into a[0]

2. while x is larger than either of its children

• swap x with the smaller of its children

• Runs in O(log n) time

• How can we remove the minimum value from a heap?

1.9

0.2

0.9

1.1

0.3

0.4

2.1

1.3

9.2

5.1

0.1

0.5

5.19.21.32.1

1.10.9

0.5 0.2

0.1

1.9

0.40.3

0.8

0.8




1.9

0.2

0.9

1.1

0.3

0.4

2.1

1.3

9.2

5.1

0.8

0.5

5.19.21.32.1

1.10.9

0.5 0.2

0.8

1.9

0.40.3



0.80.8


0.2

0.8

0.8

0.2


1.9

0.8

0.9

1.1

0.3

0.4

2.1

1.3

9.2

5.1

0.2

0.5

5.19.21.32.1

1.10.9

0.5 0.8

0.2

1.9

0.40.3



0.80.8


0.3

0.8

0.8

0.3

0.8

public T poll () {T x = a [0];swap (0, --n);trickleDown (0);return x;

}



protected void trickleDown (int i) {do {

int j = -1;int r = right (i);if (r < n && a[r]. compareTo (a[i]) < 0) {

int l = left (i);if (a[l]. compareTo (a[r]) < 0) { j = l; } else { j = r; }

} else {int l = left (i);if (l < n && a[l]. compareTo (a[i]) < 0) { j = l; }

}if (j >= 0) swap (i, j);i = j;

} while (i >= 0);}

Summary

• Theorem: A (implicit) binary heap supports the operations add(x) and deleteMin() in O(log n) (amortized) time and supports the findMin() operation in O(1) time.

Building a heap

• We can insert elements a[0],...,a[n-1] one at a time.

• This takes O(1 + log(1)) + O(1 + log(2)) + O(1 + log n)• Runs in O(nlog n) time

• How can we build a heap from an unsorted array a?• How quickly can we make a into an implicit binary heap?

public BinaryHeap (T[] ia) {a = ia;for (int n = 1; n < a. length ; n++) { add(a[n]); }

}

Can we do better?Can we do better?

Building a heap

• First build n/2 heaps of size 1

• We can do it in O(n) time. How?

• By working bottom up.

public BinaryHeap (T[] ia) {a = ia;n = a. length ;for (int i = n/2; i >= 0; i --) { trickleDown (i); }

}

• Next build n/4 heaps of size 3• Next build n/8 heaps of size 7• build 1 heap of size n…

Building a heap in linear time (analysis)

• We call trickleDown(i), n/2j times where j is the root of a heap of size 2j - 1

• (n/4) * O(1)

• trickleDown(i) then takes O(log(2j - 1)) = O(j)

time• Total running time is

• (n/8) * O(2)

• (n/16) * O(3)

• 1 * O(log n)

• = O(n)

…

Heapsort• The heapsort algorithm for sorting an array a of length n:• Make a into a heap in (O(n) time)• Repeat n times• Delete the minimum

• Each deletion takes O(log n) time• Runs in O(n log n) time• Doesn't require any extra space ( does all work inside of input array a)

public T deleteMin () {swap (0, --n);trickleDown (0);

}

Merging two heaps• We know how to add one element to a heap.• What can we do if we want to add another heap?• In other words, how can we merge 2 heaps?

public Node<T> merge(Node<T> h1, Node<T> h2){

while(h2.size()>0){T u = h2.peek();h2.deleteMin();h1.add(u);

}}

• The cost for deleteMin() and add() is O(log n)• We perform those operations n times• Total cost O(n log n)

Merging two heaps

• Can we do better?

• Actually we can do it in O(log n). How?

• merge(h1,h2) can be defined recursively

1. If either of h1 or h2 is nil, we just return h2 or h1.

2. We take the one with the biggest minimum value and merge it either with the right or left child of the other.

3. We continue the process recursively.

Merging two heaps

• What is the complexity of this operation?

• How can we reduce it?

• Using a heap ordered binary tree structure we can reduce the cost to O(log n). Why?

• in an implicit binary heap it would be O(n)• we need to copy all the elements in a

new array

Merging two heaps

Node<T> merge(Node<T> h1, Node<T> h2) { if (h1 == nil) return h2; if (h2 == nil) return h1;

if (compare(h2.x, h1.x) < 0) return merge(h2, h1); // now we know h1.x <= h2.x

if (rand.nextBoolean()) {

h1.left = merge(h1.left, h2); h1.left.parent = h1;

} else {

h1.right = merge(h1.right, h2); h1.right.parent = h1;

}return h1; }

Merging two heaps

• Why we need the random function to merge the heaps?• How can we define the add() and remove() operations in a heap ordered binary tree ? public boolean add(T x) { Node<T> u = newNode(); u.x = x; r = merge(u, r); r.parent = nil; n++; return true; }

public T remove() { T x = r.x; r = merge(r.left, r.right); if (r != nil) r.parent = nil; n--; return x; }

Analysis of merge(h1,h2)

• Lemma: The expected length of a random walk in a binary tree with n nodes is at most log(n+1)• proof by induction: For n=0 is true log(0+1)=0• Suppose it is true for every n0 < n

• n1 the size of the left sub-tree and n2 = n - n1 -1

• E[W] = 1 + log(n1+1)/2 + log(n2+1)/2

• E[W] ≤ 1 + log( (n-1)/2 + 1)

= 1 + log( (n + 1)/2 )

= log(n + 1)

Summary

• Lemma: The expected running time of

merge(h1,h2) is at most O(log n),

where n = h1.size() + h2.size()

• Theorem: A MeldableHeap implements

the (priority) Queue interface and

supports the operations add(x), remove()

and merge(h1,h2) in O(log n), expected

time per operation.

• quicksort(S):

Recall Quicksort

2 1 30 4 6 9 85 7 p=5p=5

S<

S<

S=S= S

>

S>

–Pick a random element p from S

–Compare every element to p to get 3 sets•S< = {x ϵ S : x < p}

•S= = {x ϵ S : x = p}

•S> = {x ϵ S : x > p}

2 3 40 1 7 8 95 6

–quicksort(S<)

–quicksort(S>)

Quicksort

2 1 85 4 6 9 03 75 p=5p=5

<<

>>2 1 80 4 6 9 53 7 5

iijj

jjii

2 1 80 4 6 9 83 75ii jj

2 1 30 4 6 9 83 75

ii jj

i = j

i = j

2 1 30 4 6 9 83 75

Quicksort

2 1 85 4 6 9 03 7

2 1 30 4 6 9 875

2 1 30 4 7 9 865

2 3 40 1 7 8 965

2 1 40 3 7 8 965

2 3 40 1 7 8 965

2 3 40 1 7 8 95 6

Recall Heapsort

• The heapsort algorithm for sorting an array a of length n:

• Make a into a heap in (O(n) time)• Repeat n times• Delete the minimum

2 1 85 4 6 9 03 7

2 1 40 5 6 9 83 7

2 3 40 1 7 8 95 6

Recall Heapsort2 1 85 4 6 9 03 7

2 5 40 1 6 9 83 7

2 3 40 1 5 6

2 5 81 4 6 93 7

3 5 82 4 69 7

6 5 83 4 9 7

6 7 84 5 9

6 9 85 7

8 96 7

Recall Heapsort

2 3 40 1 7 8 95 6

6 5 83 4 9 7

6 7 84 5 9

6 9 85 7

8 96 7

87 9

8 9

9

Mergesort

• The mergesort algorithm for sorting an array a of length n:

• Divide a in 2 parts a1 and a2

• Recursively, sort a1 and a2 with mergesort • Merge a1 and a22 1 85 4 6 9 03 7

2 1 85 4 6 9 03 7

4 5 81 2 6 7 90 3

2 3 40 1 7 8 95 6

Merging process

4 5 81 2

6 7 90 3

2 3 40 7 8 95 6

i0i0

i1i1

i0 < i1i0 < i1

i1 < i0i1 < i0

i0++

i0++

i1++

i1++

1

Analysis of Mergesort

• The merging process ( step 3 ) costs (O(n) time)

• The mergesort algorithm for sorting an array a of length n:

1. Divide a in 2 parts a1 and a2

2. Recursively, sort a1 and a2 with mergesort

3. Merge a1 and a2

• It is performed log n times.

• Total costs (O(n log n) time)

Theorem: The Mergesort algorithm can sort an array of n items in O(n log n) time

Sorting Algorithms

• So far, we have seen 3 sorting algorithms:

• Quicksort: O(n log n) expected time

• Heapsort: O(n log n) time

• Mergesort: O(n log n) time

• Is there a faster sorting algorithm (maybe O(n) time)?

• Answer: No and yes

Comparision based Sorting Algorithms

• The algorithm Quicksort, Heapsort and Mergesort are sort by comparing the elements.

• Every comparison-based sorting algorithm takes (n log n) time for some input

• A comparison tree is a full binary tree:

• each internal node u is labelled with a pair u.i and u.j

• each leaf is labelled with a permutation of { 0, ... , n – 1}


a[0](>/<)a[1]

a[1](>/<)a[2] a[0](>/<)a[2]

a[0]<a[1]<a[2] a[0](>/<)a[2]

a[1]<a[0]<a[2] a[1](>/<)a[2]

a[0]<a[2]<a[1] a[2]<a[0]<a[1]

a[1]<a[2]<a[0] a[2]<a[1]<a[0]

<<

<<

<<

<<

<<

>>

>>

>>

>>

>>


• Lemma: Every comparison tree that sorts any input of length n has at least n! Leaves

•Theorem: Every comparison tree that sorts any input of length n has height at least (n/2)log2(n/2)

• The height is log2(n!).

• n!=O(nn).

• log2(nn) = n log2(n).

• The height is O( n log n ).

Sorting in O(n) time

• In-class problem:

• Design an algorithm that takes an array a of n integers in the range {0, ... , k-1} and sorts them in O(n + k) time

Counting sortint [] countingSort (int [] a, int k) { int c[] = new int [k];

for (int i = 0; i < a. length ; i ++) c[a[i ]]++;

for (int i = 1; i < k; i ++) c[i] += c[i -1];

int b[] = new int [a. length ];

for (int i = a.length -1; i >= 0; i --) b[--c[a[i ]]] = a[i];

return b;}

Sorting in O(n) time

1. First count the repetition of any possible number.2. Compute the starting position of the number3. Copy the numbers on the output array

3

Recall Heapsort9 0 17 2 9 7 42 0

0 0 00 0 0 0 00 0

9 1 04 6 2 5 99 3

00 11 22 66 77 9933 44 55 88

11 11 1 22 2212 1 323 413 1 5

022 12 13 1 5

2+32+3 3+53+5

5 8

8+18+1

9

9+29+2

11

11+1

11+1

12

12+1

12+1

13

13+2

13+2

15

15+0

15+0

15

15+5

15+5

20

0 1 10 0 2 3 42 2 6 7 74 5 9 9 99 9

9 0 17 2 9 7 42 0 9 1 04 6 2 5 99 3

Radix sort

• Radix-sort uses the counting sort algorithm to sort integers one “digit" at a time• integers have w bits• “digit" has d bits• uses w/d passes of counting-sort

• Starts by sorting least-significant digits first• works up to most significant digits

• Correctness depends on fact that counting sort is stable• if a[i] = a[j] and i < j then a[i] appears before a[j] in the output

Radix sort

• Theorem: The radix-sort algorithm can sort an array a of n w-bit integers in O(n + 2d) time

• Theorem: The radix-sort algorithm can sort an array a of n integers in the range {0, ... , nc – 1} in O(cn) time.

Summary

• Comparison based sorting algorithms:

• Quicksort: O(n log n) expected time

• Heapsort: O(n log n) time

• Mergesort: O(n log n) time

• counting-sort: can sort an array a of n integers in the range {0, ... , k-1} in O(n + k) time• radix-sort can sort an array a of n integers in the range {0, ... , nc – 1} in O(cn) time.

• No comparison based sorting algorithms:

outline

Documents

complete binary heap

priority queues store

prio u

deleteminpriority queues

thempriority queues

value u

return i

prio heap property