1/15/20151 analysis of algorithms lecture analysis of quick sort

04/11/23 1

Analysis of Algorithms

Lecture

Analysis of quick sort

04/11/23 2

Quick sort

• Another divide and conquer sorting algorithm – like merge sort

• Anyone remember the basic idea?

• The worst-case and average-case running time?

• Learn some new algorithm analysis tricks

04/11/23 3

Quick sort

Quicksort an n-element array:

1. Divide: Partition the array into two subarrays around a pivot x such that elements in lower subarray x elements in upper subarray.

2. Conquer: Recursively sort the two subarrays.

3. Combine: Trivial.

x x xx ≥ x≥ x

Key: Linear-time partitioning subroutine.

04/11/23 4

Partition

• All the action takes place in the partition() function– Rearranges the subarray in place– End result: two subarrays

• All values in first subarray all values in second

– Returns the index of the “pivot” element separating the two subarrays

x x xx ≥ x≥ xp rq

04/11/23 5

Pseudocode for quicksort

QUICKSORT(A, p, r)if p < r

then q PARTITION(A, p, r) QUICKSORT(A, p, q–1)QUICKSORT(A, q+1, r)

Initial call: QUICKSORT(A, 1, n)

04/11/23 6

Idea of partition

• If we are allowed to use a second array, it would be easy

66 1010 55 88 1313 33 22 1111

66 55 33 22 1111 1313 88 1010

22 55 33 66 1111 1313 88 1010

04/11/23 7

Another idea

• Keep two iterators: one from head, one from tail

66 1010 55 88 1313 33 22 1111

66 22 55 33 1313 88 1010 1111

33 22 55 66 1313 88 1010 1111

04/11/23 8

In-place Partition

66 1010 55 88 1313 33 22 11112 3 8 103 6

04/11/23 9

Partition In Words

• Partition(A, p, r):– Select an element to act as the “pivot” (which?)

– Grow two regions, A[p..i] and A[j..r]• All elements in A[p..i] <= pivot

• All elements in A[j..r] >= pivot

– Increment i until A[i] > pivot

– Decrement j until A[j] < pivot

– Swap A[i] and A[j]

– Repeat until i >= j

– Swap A[j] and A[p]

– Return j

Note: different from book’s partition(), which uses two iterators that both move forward.

04/11/23 10

Partition CodePartition(A, p, r) x = A[p]; // pivot is the first element i = p; j = r + 1; while (TRUE) {

repeat i++; until A[i] > x or i >= j; repeat j--; until A[j] < x or j < i; if (i < j) Swap (A[i], A[j]); else break;

} swap (A[p], A[j]); return j;

What is the running time of partition()?

partition() runs in (n) time

04/11/23 11

i j66 1010 55 88 1313 33 22 1111x = 6

p r

i j66 1010 55 88 1313 33 22 1111

i j66 22 55 88 1313 33 1010 1111

i j66 22 55 88 1313 33 1010 1111

i j66 22 55 33 1313 88 1010 1111

ij66 22 55 33 1313 88 1010 1111

33 22 55 66 1313 88 1010 1111qp r

scan

scan

scan

swap

swap

final swap

Partition example

04/11/23 12

66 1010 55 88 1111 33 22 1313

33 22 55 66 1111 88 1010 1313

22 33 55 66 88 1010 1111 1313

22 33 55 66 1010 88 1111 1313

22 33 55 66 88 1010 1111 1313

Quick sortexample

04/11/23 13

Analysis of quicksort

• Assume all input elements are distinct.

• In practice, there are better partitioning algorithms for when duplicate input elements may exist.

• Let T(n) = worst-case running time on an array of n elements.

04/11/23 14

Worst-case of quicksort

• Input sorted or reverse sorted.• Partition around min or max element.• One side of partition always has no elements.

)(

)()1(

)()1()1(

)()1()0()(

2n

nnT

nnT

nnTTnT

(arithmetic series)

04/11/23 15

Worst-case recursion treeT(n) = T(0) + T(n–1) + n

04/11/23 16

Worst-case recursion treeT(n) = T(0) + T(n–1) + n

T(n)

04/11/23 17

n

T(0) T(n–1)

Worst-case recursion treeT(n) = T(0) + T(n–1) + n

04/11/23 18

n

T(0) (n–1)

Worst-case recursion treeT(n) = T(0) + T(n–1) + n

T(0) T(n–2)

04/11/23 19

n

T(0) (n–1)

Worst-case recursion treeT(n) = T(0) + T(n–1) + n

T(0) (n–2)

T(0)

T(0)

04/11/23 20

n

T(0) (n–1)

Worst-case recursion treeT(n) = T(0) + T(n–1) + n

T(0) (n–2)

T(0)

T(0)

2

1

nkk

height

height = n

04/11/23 21

n

T(0) (n–1)

Worst-case recursion treeT(n) = T(0) + T(n–1) + n

T(0) (n–2)

T(0)

T(0)

2

1

nkk

n

height = n

04/11/23 22

n

(n–1)

Worst-case recursion treeT(n) = T(0) + T(n–1) + n

(n–2)

(1)

2

1

nkk

n

height = n

(1)

(1)

(1)T(n) = (n) + (n2)

= (n2)

04/11/23 23

Best-case analysis(For intuition only!)

If we’re lucky, PARTITION splits the array evenly:

T(n) = 2T(n/2) + (n)= (n log n) (same as merge sort)

What if the split is always 109

101 : ?

)()(109

101 nnTnTnT

What is the solution to this recurrence?

04/11/23 24

Analysis of “almost-best” case

)(nT

04/11/23 25

Analysis of “almost-best” case

nT101 nT

109

n

04/11/23 26

Analysis of “almost-best” case

n101 n10

9

nT100

1 nT100

9 nT100

9 nT10081

n

04/11/23 27

Analysis of “almost-best” case

n1001

(1)

(1)

… …log10/9n

n

n

n

…O(n) leavesO(n) leaves

n101 n10

9

n1009 n100

9 n10081

n

04/11/23 28

log10n

Analysis of “almost-best” case

n

(1)

(1)

… …log10/9n

n

n

n

T(n) n log10/9n + (n)

…

n log10n

O(n) leavesO(n) leaves

(n log n)

n1001

n101 n10

9

n1009 n100

9 n10081

04/11/23 29

Quicksort Runtimes

• Best-case runtime Tbest(n) (n log n)

• Worst-case runtime Tworst(n) (n2)

• Worse than mergesort? Why is it called quicksort then?

• Its average runtime Tavg(n) (n log n )

• Better even, the expected runtime of randomized quicksort is (n log n)

04/11/23 30

Randomized quicksort

• Randomly choose an element as pivot– Every time need to do a partition, throw a die to

decide which element to use as the pivot– Each element has 1/n probability to be selected

Rand-Partition(A, p, r) d = random(); // a random number between 0 and 1 index = p + floor((r-p+1) * d); // p<=index<=r swap(A[p], A[index]); Partition(A, p, r); // now do partition using A[p] as pivot

04/11/23 31

Running time of randomized quicksort

T(n) =

T(0) + T(n–1) + dn if 0 : n–1 split,T(1) + T(n–2) + dn if 1 : n–2 split,T(n–1) + T(0) + dn if n–1 : 0 split,