merge sort quick sort
TRANSCRIPT
-
8/12/2019 Merge Sort Quick Sort
1/43
Analysis of Algorithms
CS 477/677
SortingPart B
Instructor: George Bebis
(Chapter 7)
-
8/12/2019 Merge Sort Quick Sort
2/43
2
Sorting
Insertion sort Design approach:
Sorts in place:
Best case:
Worst case:
Bubble Sort
Design approach:
Sorts in place:
Running time:
Yes
(n)
(n2)
incremental
Yes(n2)
incremental
-
8/12/2019 Merge Sort Quick Sort
3/43
3
Sorting
Selection sort Design approach:
Sorts in place:
Running time:
Merge Sort
Design approach:
Sorts in place:
Running time:
Yes
(n2)
incremental
No
Lets see!!
divide and conquer
-
8/12/2019 Merge Sort Quick Sort
4/43
4
Divide-and-Conquer
Dividethe problem into a number of sub-problems
Similar sub-problems of smaller size
Conquerthe sub-problems
Solve the sub-problems recursively
Sub-problem size small enough solve the problems in
straightforward manner
Combinethe solutions of the sub-problems
Obtain the solution for the original problem
-
8/12/2019 Merge Sort Quick Sort
5/43
5
Merge Sort Approach
To sort an array A[p . . r]:
Divide
Divide the n-element sequence to be sorted into two
subsequences of n/2elements each
Conquer
Sort the subsequences recursively using merge sort
When the size of the sequences is 1 there is nothing
more to do
Combine
Merge the two sorted subsequences
-
8/12/2019 Merge Sort Quick Sort
6/43
6
Merge Sort
Alg.:MERGE-SORT(A, p, r)
if p < r Check for base casethen q
(p + r)/2
Divide
MERGE-SORT(A, p, q) ConquerMERGE-SORT(A, q + 1, r) Conquer
MERGE(A, p, q, r)
Combine
Initial call:MERGE-SORT(A, 1, n)
1 2 3 4 5 6 7 8
62317425
p rq
-
8/12/2019 Merge Sort Quick Sort
7/43
7
ExamplenPower of 2
1 2 3 4 5 6 7 8
q = 462317425
1 2 3 4
7425
5 6 7 8
6231
1 2
25
3 4
74
5 6
31
7 8
62
1
5
2
2
3
4
4
7 1
6
3
7
2
8
6
5
Divide
-
8/12/2019 Merge Sort Quick Sort
8/43
8
ExamplenPower of 2
1
5
2
2
3
4
4
7 1
6
3
7
2
8
6
5
1 2 3 4 5 6 7 8
76543221
1 2 3 4
7542
5 6 7 8
6321
1 2
52
3 4
74
5 6
31
7 8
62
Conquer
and
Merge
-
8/12/2019 Merge Sort Quick Sort
9/43
9
ExamplenNot a Power of 2
62537416274
1 2 3 4 5 6 7 8 9 10 11
q = 6
416274
1 2 3 4 5 6
62537
7 8 9 10 11
q = 9q = 3
274
1 2 3
416
4 5 6
537
7 8 9
62
10 11
74
1 2
2
3
16
4 5
4
6
37
7 8
5
9
2
10
6
11
4
1
7
2
6
4
1
5
7
7
3
8
Divide
-
8/12/2019 Merge Sort Quick Sort
10/43
10
ExamplenNot a Power of 2
77665443221
1 2 3 4 5 6 7 8 9 10 11
764421
1 2 3 4 5 6
76532
7 8 9 10 11
742
1 2 3
641
4 5 6
753
7 8 9
62
10 11
2
3
4
6
5
9
2
10
6
11
4
1
7
2
6
4
1
5
7
7
3
8
74
1 2
61
4 5
73
7 8
Conquerand
Merge
-
8/12/2019 Merge Sort Quick Sort
11/43
11
Merging
Input:Array Aand indices p, q, rsuch thatp q < r
Subarrays A[p . . q]and A[q + 1 . . r]are sorted
Output: One single sorted subarray A[p . . r]
1 2 3 4 5 6 7 8
63217542
p rq
-
8/12/2019 Merge Sort Quick Sort
12/43
12
Merging
Idea for merging:
Two piles of sorted cards
Choose the smaller of the two top cards
Remove it and place it in the output pile
Repeat the process until one pile is empty
Take the remaining input pile and place it face-down
onto the output pile
1 2 3 4 5 6 7 8
63217542
p rq
A1A[p, q]
A2A[q+1, r]
A[p, r]
-
8/12/2019 Merge Sort Quick Sort
13/43
13
Example: MERGE(A, 9, 12, 16)
r
-
8/12/2019 Merge Sort Quick Sort
14/43
14
Example: MERGE(A, 9, 12, 16)
-
8/12/2019 Merge Sort Quick Sort
15/43
15
Example (cont.)
-
8/12/2019 Merge Sort Quick Sort
16/43
16
Example (cont.)
-
8/12/2019 Merge Sort Quick Sort
17/43
17
Example (cont.)
Done!
-
8/12/2019 Merge Sort Quick Sort
18/43
18
Merge - Pseudocode
Alg.:MERGE(A, p, q, r)1. Compute n1andn2
2. Copy the first n1elements intoL[1 . . n1+ 1]and the next n2elements into R[1 . . n2+ 1]
3. L[n1+ 1] ; R[n2+ 1] 4. i1; j1
5. for kpto r
6. do if L[ i ] R[ j ]
7. then A[k]L[ i ]8. ii + 1
9. else A[k]R[ j ]
10.j
j + 1
p q
7542
6321
rq + 1
L
R
1 2 3 4 5 6 7 8
63217542
p rq
n1 n2
R i Ti f M
-
8/12/2019 Merge Sort Quick Sort
19/43
19
Running Time of Merge
(assume last forloop)
Initialization (copying into temporary arrays): (n1+ n2) = (n)
Adding the elements to the final array:
- niterations, each taking constant time (n)
Total time for Merge:
(n)
-
8/12/2019 Merge Sort Quick Sort
20/43
20
Analyzing Divide-and Conquer Algorithms
The recurrence is based on the three steps of
the paradigm:
T(n)running time on a problem of size n
Dividethe problem into asubproblems, each of size
n/b
:takes
D(n)
Conquer(solve) the subproblems aT(n/b)
Combinethe solutions C(n)
(1) if n cT(n) = aT(n/b) + D(n) + C(n) otherwise
-
8/12/2019 Merge Sort Quick Sort
21/43
21
MERGE-SORT Running Time
Divide: compute qas the average of pand r:D(n) = (1)
Conquer:
recursively solve 2 subproblems, each of size n/2
2T (n/2) Combine:
MERGE on an n-element subarray takes (n)time
C(n) =
(n)
(1) if n =1
T(n) = 2T(n/2) + (n) if n > 1
-
8/12/2019 Merge Sort Quick Sort
22/43
22
Solve the Recurrence
T(n) = c if n = 12T(n/2) + cn if n > 1
Use Masters Theorem:
Compare n with f(n) = cn
Case 2: T(n) = (nlgn)
-
8/12/2019 Merge Sort Quick Sort
23/43
23
Merge Sort - Discussion
Running time insensitive of the input
Advantages:
Guaranteed to run in (nlgn)
Disadvantage
Requires extra space N
-
8/12/2019 Merge Sort Quick Sort
24/43
24
Sorting Challenge 1
Problem: Sort a file of huge records with tinykeys
Example application: Reorganize your MP-3 files
Which method to use?A. merge sort, guaranteed to run in time NlgN
B. selection sort
C. bubble sortD. a custom algorithm for huge records/tiny keys
E. insertion sort
S ti Fil ith H R d d
-
8/12/2019 Merge Sort Quick Sort
25/43
25
Sorting Files with Huge Records and
Small Keys
Insertion sort or bubble sort?
NO, too many exchanges
Selection sort?
YES, it takes lineartime for exchanges
Merge sort or custom method?
Probably not: selection sort simpler, does less swaps
-
8/12/2019 Merge Sort Quick Sort
26/43
26
Sorting Challenge 2
Problem: Sort a huge randomly-ordered file ofsmall records
Application: Process transaction record for a
phone company
Which sorting method to use?A. Bubble sort
B. Selection sortC. Mergesort guaranteed to run in time NlgN
D. Insertion sort
-
8/12/2019 Merge Sort Quick Sort
27/43
27
Sorting Huge, Randomly - Ordered Files
Selection sort? NO, always takes quadratic time
Bubble sort?
NO, quadratic time for randomly-ordered keys
Insertion sort?
NO, quadratic time for randomly-ordered keys
Mergesort?
YES, it is designed for this problem
-
8/12/2019 Merge Sort Quick Sort
28/43
28
Sorting Challenge 3
Problem: sort a file that is already almost inorder
Applications:
Re-sort a huge database after a few changes
Doublecheck that someone else sorted a file
Which sorting method to use?A. Mergesort, guaranteed to run in time NlgN
B. Selection sortC. Bubble sort
D. A custom algorithm for almost in-order files
E. Insertion sort
-
8/12/2019 Merge Sort Quick Sort
29/43
29
Sorting Files That are Almost in Order
Selection sort? NO, always takes quadratic time
Bubble sort?
NO, bad for some definitions of almost in order
Ex:B C D E F G H I J K L M N O P Q R S T U V W X Y Z A
Insertion sort?
YES, takes linear time for most definitions of almost
in order
Mergesort or custom method?
Probably not: insertion sort simpler and faster
-
8/12/2019 Merge Sort Quick Sort
30/43
30
Quicksort
Sort an array A[pr] Divide
Partition the array Ainto 2 subarrays A[p..q]and A[q+1..r], such
that each element of A[p..q]is smaller than or equal to each
element in A[q+1..r]
Need to find index qto partition the array
A[pq] A[q+1r]
-
8/12/2019 Merge Sort Quick Sort
31/43
31
Quicksort
Conquer
Recursively sort A[p..q]and A[q+1..r]using Quicksort
Combine
Trivial: the arrays are sorted in place
No additional work is required to combine them
The entire array is now sorted
A[pq] A[q+1r]
-
8/12/2019 Merge Sort Quick Sort
32/43
32
QUICKSORT
Alg.:QUICKSORT(A, p, r)
ifp < r
thenqPARTITION(A, p, r)
QUICKSORT (A, p, q)
QUICKSORT (A, q+1, r)
Recurrence:
Initially: p=1, r=n
PARTITION())T(n) = T(q) + T(nq) + f(n)
-
8/12/2019 Merge Sort Quick Sort
33/43
33
Partitioning the Array
Choosing PARTITION() There are different ways to do this
Each has its own advantages/disadvantages
Hoare partition (see prob. 7-1, page 159)
Select a pivot element xaround which to partition
Grows two regions
A[pi] x
xA[jr]
A[pi] x x A[jr]
i j
-
8/12/2019 Merge Sort Quick Sort
34/43
34
Example
73146235
i j
75146233
i j
75146233
i j
75641233
i j
73146235
i j
A[pr]
75641233
ij
A[pq] A[q+1r]
pivot x=5
-
8/12/2019 Merge Sort Quick Sort
35/43
35
Example
-
8/12/2019 Merge Sort Quick Sort
36/43
36
Partitioning the Array
Alg.PARTITION (A, p, r)
1. x A[p]
2. i p 1
3. j r + 1
4. whileTRUE5. do repeatj j 1
6. untilA[j] x
7. do repeati i + 1
8. untilA[i] x9. ifi < j
10. thenexchange A[i] A[j]
11. elsereturnj
Running time: (n)n = r p + 1
73146235
i j
A:
arap
ij=q
A:
A[pq] A[q+1r]
p r
Each element is
visited once!
-
8/12/2019 Merge Sort Quick Sort
37/43
37
Recurrence
Alg.:QUICKSORT(A, p, r)
ifp < r
thenqPARTITION(A, p, r)
QUICKSORT (A, p, q)
QUICKSORT (A, q+1, r)
Recurrence:
Initially: p=1, r=n
T(n) = T(q) + T(nq) + n
-
8/12/2019 Merge Sort Quick Sort
38/43
38
Worst Case Partitioning
Worst-case partitioning One region has one element and the other has n1 elements
Maximally unbalanced
Recurrence: q=1
T(n) = T(1) + T(n1) + n,
T(1) = (1)
T(n) = T(n1) + n
=
2 2
1
1 ( ) ( ) ( )n
k
n k n n n
n
n - 1n - 2
n - 3
2
1
11
1
1
1
n
n
nn - 1
n - 2
3
2
(n2)
When does the worst case happen?
-
8/12/2019 Merge Sort Quick Sort
39/43
39
Best Case Partitioning
Best-case partitioning
Partitioning produces two regions of size n/2
Recurrence: q=n/2
T(n) = 2T(n/2) + (n)
T(n) =
(nlgn)(Master theorem)
-
8/12/2019 Merge Sort Quick Sort
40/43
40
Case Between Worst and Best
9-to-1 proportional splitQ(n) = Q(9n/10) + Q(n/10) + n
-
8/12/2019 Merge Sort Quick Sort
41/43
41
How does partition affect performance?
-
8/12/2019 Merge Sort Quick Sort
42/43
42
How does partition affect performance?
-
8/12/2019 Merge Sort Quick Sort
43/43
Performance of Quicksort
Average case
All permutations of the input numbers are equally likely
On a random input array, we will have a mixof well balanced
and unbalanced splits
Good and bad splits are randomly distributed across throughout
the tree
Alternate of a goodand a bad split
Nearly wellbalanced split
n
n - 11
(n 1)/2(n 1)/2
n
(n 1)/2(n 1)/2 + 1
Running time of Quicksort when levels alternatebetween good and bad splits is O(nlgn)
combined partitioning cost:
2n-1 = (n)
partitioning cost:
n = (n)