cosc 3101a - design and analysis of algorithms 2
DESCRIPTION
COSC 3101A - Design and Analysis of Algorithms 2. Asymptotic Notations Continued Proof of Correctness: Loop Invariant Designing Algorithms: Divide and Conquer. Typical Running Time Functions. 1 (constant running time): Instructions are executed once or a few times logN (logarithmic) - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/1.jpg)
COSC 3101A - Design and Analysis of Algorithms
2
Asymptotic Notations Continued
Proof of Correctness: Loop Invariant
Designing Algorithms: Divide and Conquer
![Page 2: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/2.jpg)
5/11/2004 Lecture 2 COSC3101A 2
Typical Running Time Functions
• 1 (constant running time):
– Instructions are executed once or a few times
• logN (logarithmic)
– A big problem is solved by cutting the original problem in smaller
sizes, by a constant fraction at each step
• N (linear)
– A small amount of processing is done on each input element
• N logN
– A problem is solved by dividing it into smaller problems, solving
them independently and combining the solution
![Page 3: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/3.jpg)
5/11/2004 Lecture 2 COSC3101A 3
Typical Running Time Functions
• N2 (quadratic)
– Typical for algorithms that process all pairs of data items (double
nested loops)
• N3 (cubic)
– Processing of triples of data (triple nested loops)
• NK (polynomial)
• 2N (exponential)
– Few exponential algorithms are appropriate for practical use
![Page 4: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/4.jpg)
5/11/2004 Lecture 2 COSC3101A 4
Logarithms
• In algorithm analysis we often use the notation “log n”
without specifying the base
nn
nn
elogln
loglg 2
yxlogBinary logarithm
Natural logarithm
)lg(lglglg
)(lglg
nn
nn kk
xy log
xylog yx loglog
y
xlog yx loglog
xalog xb ba loglog
abx logxba log
![Page 5: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/5.jpg)
5/11/2004 Lecture 2 COSC3101A 5
Review: Asymptotic Notations(1)
))(( ng))(( ngO))(( ng
)()(0:,0,0|)())(( 00 ncgnfnnncnfngo
)()(0:,0,0|)())(( 00 nfncgnnncnfng
)()(0:,0,0|)())(( 00 nfncgnnncnfng
)()()(:,0,0,0|)())(( 120021 ngcnfngcnnnccnfng
)()(0:,0,0|)())(( 00 ncgnfnnncnfngO
))(( ngo ))(( ng
![Page 6: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/6.jpg)
5/11/2004 Lecture 2 COSC3101A 6
Review: Asymptotic Notations(2)
))(( ng))(( ngO ))(( ng
))(()( ngonf
))(( ngo ))(( ng
if and only if 0)(
)(lim
ng
nfn
))(()( ngnf if and only if )(
)(lim
ng
nfn
L0L L0
![Page 7: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/7.jpg)
5/11/2004 Lecture 2 COSC3101A 7
Review: Asymptotic Notations(3)
• A way to describe behavior of functions in the limit
– How we indicate running times of algorithms
– Describe the running time of an algorithm as n grows to
• O notation: asymptotic “less than”: f(n) “≤”
g(n)
notation: asymptotic “greater than”: f(n) “≥” g(n)
notation: asymptotic “equality”: f(n) “=” g(n)
![Page 8: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/8.jpg)
5/11/2004 Lecture 2 COSC3101A 8
Big-O Examples(1)
– 2n2 = O(n3):
– n2 = O(n2):
– 1000n2+1000n = O(n2):
– n = O(n2):
2n2 ≤ cn3 2 ≤ cn c = 1 and n0= 2
n2 ≤ cn2 c ≥ 1 c = 1 and n0= 1
1000n2+1000n ≤ cn2 1000n+1000 ≤ cn c=2000 and n0 = 1
n ≤ cn2 cn ≥ 1 c = 1 and n0= 1
![Page 9: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/9.jpg)
5/11/2004 Lecture 2 COSC3101A 9
Big-O Examples(2)
• E.g.: prove that n2 ≠ O(n)
– Assume c & n0 such that: n≥ n0: n2 ≤ cn
– Choose n = max (n0, c)
– n2 = n n ≥ n c n2 ≥ cn
contradiction!!!
![Page 10: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/10.jpg)
5/11/2004 Lecture 2 COSC3101A 10
More on Asymptotic Notations
• There is no unique set of values for n0 and c in proving the
asymptotic bounds
• Prove that 100n + 5 = O(n2)
– 100n + 5 ≤ 100n + n = 101n ≤ 101n2
for all n ≥ 5
n0 = 5 and c = 101 is a solution
– 100n + 5 ≤ 100n + 5n = 105n ≤ 105n2
for all n ≥ 1
n0 = 1 and c = 105 is also a solution
Must find SOME constants c and n0 that satisfy the asymptotic notation relation
![Page 11: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/11.jpg)
5/11/2004 Lecture 2 COSC3101A 11
Big- Examples
– 5n2 = (n)
– 100n + 5 ≠ (n2)
– n = (2n), n3 = (n2), n = (logn)
c, n0 such that: 0 cn 5n2 cn 5n2 c = 1 and n0 = 1
c, n0 such that: 0 cn2 100n + 5
100n + 5 100n + 5n ( n 1) = 105n
cn2 105n n(cn – 105) 0
Since n is positive cn – 105 0 n 105/c
contradiction: n cannot be smaller than a constant
![Page 12: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/12.jpg)
5/11/2004 Lecture 2 COSC3101A 12
Examples
– n2/2 –n/2 = (n2)
• ½ n2 - ½ n ≤ ½ n2 n ≥ 0 c2= ½
• ½ n2 - ½ n ≥ ½ n2 - ½ n * ½ n ( n ≥ 2 ) = ¼ n2 c1= ¼
– n ≠ (n2): c1 n2 ≤ n ≤ c2 n2 only holds for: n ≤ 1/c1
– 6n3 ≠ (n2): c1 n2 ≤ 6n3 ≤ c2 n2 only holds for: n ≤
c2 /6
– n ≠ (logn): c1 logn ≤ n ≤ c2 logn
c2 ≥ n/logn, n≥ n0 – impossible
![Page 13: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/13.jpg)
5/11/2004 Lecture 2 COSC3101A 13
Comparisons of Functions
• Theorem:f(n) = (g(n)) f = O(g(n)) and f = (g(n))
• Transitivity:– f(n) = (g(n)) and g(n) = (h(n)) f(n) = (h(n))– Same for O and
• Reflexivity:– f(n) = (f(n))– Same for O and
• Symmetry:– f(n) = (g(n)) if and only if g(n) = (f(n))
• Transpose symmetry:– f(n) = O(g(n)) if and only if g(n) = (f(n))
![Page 14: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/14.jpg)
5/11/2004 Lecture 2 COSC3101A 14
More Examples(1)
• For each of the following pairs of functions, either f(n) is O(g(n)), f(n) is Ω(g(n)), or f(n) = Θ(g(n)). Determine which relationship is correct.
– f(n) = log n2; g(n) = log n + 5
– f(n) = n; g(n) = log n2
– f(n) = log log n; g(n) = log n
– f(n) = n; g(n) = log2 n
– f(n) = n log n + n; g(n) = log n
– f(n) = 10; g(n) = log 10
– f(n) = 2n; g(n) = 10n2
– f(n) = 2n; g(n) = 3n
f(n) = (g(n))
f(n) = (g(n))
f(n) = O(g(n))
f(n) = (g(n))
f(n) = (g(n))
f(n) = (g(n))
f(n) = (g(n))
f(n) = O(g(n))
![Page 15: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/15.jpg)
5/11/2004 Lecture 2 COSC3101A 15
More Examples(2)
notation
– n2/2 – n/2
– (6n3 + 1)lgn/(n + 1)
– n vs. n2
notation
– n vs. 2n
– n3 vs. n2
– n vs. logn
– n vs. n2
= (n2)
n ≠ (n2)
= (n2lgn)
n = (2n)
• O notation
– 2n2 vs. n3
– n2 vs. n2
– n3 vs. nlogn
n3 = (n2)
n = (logn)
n (n2)
2n2 = O(n3)
n2 = O(n2)
n3 O(nlgn)
![Page 16: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/16.jpg)
5/11/2004 Lecture 2 COSC3101A 16
Asymptotic Notations in Equations
• On the right-hand side (n2) stands for some anonymous function in (n2)2n2 + 3n + 1 = 2n2 + (n) means:
There exists a function f(n) (n) such that
2n2 + 3n + 1 = 2n2 + f(n)
• On the left-hand side2n2 + (n) = (n2)No matter how the anonymous function is chosen on
the left-hand side, there is a way to choose the anonymous function on the right-hand side to make the equation valid.
![Page 17: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/17.jpg)
5/11/2004 Lecture 2 COSC3101A 17
Limits and Comparisons of Functions
Using limits for comparing orders of growth:
• compare ½ n (n-1) and n2
))(()(:)(thangrowthoforderlargerahas)(,
))(()(:)(asgrowthofordersamethehas)(,
))(()(:)(thangrowthofordersmallerahas)(,0
)(
)(lim
ngntngnt
ngntngntc
ngontngnt
ng
ntn
2
111lim
2
1lim
2
1)1(21
lim2
2
2
nn
nn
n
nn
nnn
![Page 18: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/18.jpg)
5/11/2004 Lecture 2 COSC3101A 18
Limits and Comparisons of Functions
L’Hopital rule:
)('
)('lim
)(
)(lim
ng
nt
ng
ntnn
n• compare and
n
nn
lglim
n
nn
lglim
n
en
n
2
1
lg1
limn
ne
n limlg2 0
nlg
![Page 19: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/19.jpg)
5/11/2004 Lecture 2 COSC3101A 19
Loop Invariant
• A loop invariant is a relation among program variables that – is true when control enters a loop, – remains true each time the program executes the
body of the loop, – and is still true when control exits the loop.
• Understanding loop invariants can help us – analyze algorithms, – check for errors, – and derive algorithms from specifications.
![Page 20: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/20.jpg)
5/11/2004 Lecture 2 COSC3101A 20
Proving Loop Invariants
• Initialization (base case): – It is true prior to the first iteration of the loop
• Maintenance (inductive step): – If it is true before an iteration of the loop, it remains true before
the next iteration
• Termination: – When the loop terminates, the invariant - usually along with the
reason that the loop terminated - gives us a useful property that
helps show that the algorithm is correct
– Stop the induction when the loop terminates
• Proving loop invariants works like induction
![Page 21: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/21.jpg)
5/11/2004 Lecture 2 COSC3101A 21
Loop Invariant for Insertion Sort(1)
Alg.: INSERTION-SORT(A)
for j ← 2 to n
do key ← A[ j ]
Insert A[ j ] into the sorted sequence A[1 . . j -1]
i ← j - 1
while i > 0 and A[i] > keydo A[i + 1] ← A[i] i ← i – 1
A[i + 1] ← key
a8a7a6a5a4a3a2a1
1 2 3 4 5 6 7 8
key
Invariant: at the start of the for loop the elements in A[1 . . j-1] are in sorted order
![Page 22: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/22.jpg)
5/11/2004 Lecture 2 COSC3101A 22
Loop Invariant for Insertion Sort(2)
• Initialization:
– Just before the first iteration, j = 2:
the subarray A[1 . . j-1] = A[1],
(the element originally in A[1]) – is
sorted
![Page 23: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/23.jpg)
5/11/2004 Lecture 2 COSC3101A 23
Loop Invariant for Insertion Sort(3)
• Maintenance: – the while inner loop moves A[j -1], A[j -2], A[j -3],
and so on, by one position to the right until the proper position for key (which has the value that started out in A[j]) is found
– At that point, the value of key is placed into this position.
![Page 24: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/24.jpg)
5/11/2004 Lecture 2 COSC3101A 24
Loop Invariant for Insertion Sort(4)
• Termination: – The outer for loop ends when j > n (i.e, j = n + 1)
j-1 = n– Replace n with j-1 in the loop invariant:
• the subarray A[1 . . n] consists of the elements originally in A[1 . . n], but in sorted order
• The entire array is sorted!
jj - 1
![Page 25: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/25.jpg)
5/11/2004 Lecture 2 COSC3101A 25
Steps in Designing Algorithms(1)
1. Understand the problem
• Specify the range of inputs the algorithm should handle
2. Learn about the model of the implementation technology
• RAM (Random-access machine), sequential execution
3. Choosing between an exact and an approximate solution
• Some problems cannot be solved exactly: nonlinear equations,
evaluating definite integrals
• Exact solutions may be unacceptably slow
4. Choose the appropriate data structures
![Page 26: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/26.jpg)
5/11/2004 Lecture 2 COSC3101A 26
Steps in Designing Algorithms(2)
5. Choose an algorithm design technique
• General approach to solving problems algorithmically that is
applicable to a variety of computational problems
• Provide guidance for developing solutions to new problems
6. Specify the algorithm
• Pseudocode: mixture of natural and programming language
7. Prove the algorithm’s correctness
• Algorithm yields the correct result for any legitimate input, in a
finite amount of time
• Mathematical induction, loop-invariants
![Page 27: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/27.jpg)
5/11/2004 Lecture 2 COSC3101A 27
Steps in Designing Algorithms(3)
8. Analyze the Algorithm
• Predicting the amount of resources required:
• memory: how much space is needed?
• computational time: how fast the algorithm runs?
• FACT: running time grows with the size of the input
• Input size (number of elements in the input)
– Size of an array, polynomial degree, # of elements in a matrix, # of bits in the binary
representation of the input, vertices and edges in a graph
Def: Running time = the number of primitive operations (steps)
executed before termination
– Arithmetic operations (+, -, *), data movement, control, decision making ( if, while),
comparison
![Page 28: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/28.jpg)
5/11/2004 Lecture 2 COSC3101A 28
Steps in Designing Algorithms(4)
9. Coding the algorithm
• Verify the ranges of the input
• Efficient/inefficient implementation
• It is hard to prove the correctness of a program (typically done
by testing)
![Page 29: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/29.jpg)
5/11/2004 Lecture 2 COSC3101A 29
Classification of Algorithms
• By problem types
– Sorting
– Searching
– String processing
– Graph problems
– Combinatorial problems
– Geometric problems
– Numerical problems
• By design paradigms
– Divide-and-conquer
– Incremental
– Dynamic programming
– Greedy algorithms
– Randomized/probabilistic
![Page 30: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/30.jpg)
5/11/2004 Lecture 2 COSC3101A 30
Divide-and-Conquer
• Divide the problem into a number of subproblems
– Similar sub-problems of smaller size
• Conquer the sub-problems
– Solve the sub-problems recursively
– Sub-problem size small enough solve the problems in
straightforward manner
• Combine the solutions to the sub-problems
– Obtain the solution for the original problem
![Page 31: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/31.jpg)
5/11/2004 Lecture 2 COSC3101A 31
Merge Sort Approach
• To sort an array A[p . . r]:
• Divide– Divide the n-element sequence to be sorted into two
subsequences of n/2 elements each
• Conquer
– Sort the subsequences recursively using merge sort
– When the size of the sequences is 1 there is nothing
more to do
• Combine
– Merge the two sorted subsequences
![Page 32: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/32.jpg)
5/11/2004 Lecture 2 COSC3101A 32
Merge Sort
Alg.: MERGE-SORT(A, p, r)
if p < r Check for base case
then q ← (p + r)/2 Divide
MERGE-SORT(A, p, q) Conquer
MERGE-SORT(A, q + 1, r) Conquer
MERGE(A, p, q, r) Combine
• Initial call: MERGE-SORT(A, 1, n)
1 2 3 4 5 6 7 8
62317425
p rq
![Page 33: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/33.jpg)
5/11/2004 Lecture 2 COSC3101A 33
Example – n Power of 2
1 2 3 4 5 6 7 8
q = 462317425
1 2 3 4
7425
5 6 7 8
6231
1 2
25
3 4
74
5 6
31
7 8
62
1
5
2
2
3
4
4
7 1
6
3
7
2
8
6
5
Example
![Page 34: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/34.jpg)
5/11/2004 Lecture 2 COSC3101A 34
Example – n Power of 2
1
5
2
2
3
4
4
7 1
6
3
7
2
8
6
5
1 2 3 4 5 6 7 8
76543221
1 2 3 4
7542
5 6 7 8
6321
1 2
52
3 4
74
5 6
31
7 8
62
![Page 35: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/35.jpg)
5/11/2004 Lecture 2 COSC3101A 35
Example – n Not a Power of 2
62537416274
1 2 3 4 5 6 7 8 9 10 11
q = 6
416274
1 2 3 4 5 6
62537
7 8 9 10 11
q = 9q = 3
274
1 2 3
416
4 5 6
537
7 8 9
62
10 11
74
1 2
2
3
16
4 5
4
6
37
7 8
5
9
2
10
6
11
4
1
7
2
6
4
1
5
7
7
3
8
![Page 36: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/36.jpg)
5/11/2004 Lecture 2 COSC3101A 36
Example – n Not a Power of 2
77665443221
1 2 3 4 5 6 7 8 9 10 11
764421
1 2 3 4 5 6
76532
7 8 9 10 11
742
1 2 3
641
4 5 6
753
7 8 9
62
10 11
2
3
4
6
5
9
2
10
6
11
4
1
7
2
6
4
1
5
7
7
3
8
74
1 2
61
4 5
73
7 8
![Page 37: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/37.jpg)
5/11/2004 Lecture 2 COSC3101A 37
Merging
• Input: Array A and indices p, q, r such that p ≤ q < r– Subarrays A[p . . q] and A[q + 1 . . r] are sorted
• Output: One single sorted subarray A[p . . r]
1 2 3 4 5 6 7 8
63217542
p rq
![Page 38: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/38.jpg)
5/11/2004 Lecture 2 COSC3101A 38
Merging
• Idea for merging:
– Two piles of sorted cards
• Choose the smaller of the two top cards
• Remove it and place it in the output pile
– Repeat the process until one pile is empty
– Take the remaining input pile and place it face-down
onto the output pile
![Page 39: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/39.jpg)
5/11/2004 Lecture 2 COSC3101A 39
Merge - Pseudocode
Alg.: MERGE(A, p, q, r)1. Compute n1 and n2
2. Copy the first n1 elements into L[1 . . n1 + 1] and the next n2 elements into R[1 . . n2 + 1]
3. L[n1 + 1] ← ; R[n2 + 1] ←
4. i ← 1; j ← 15. for k ← p to r6. do if L[ i ] ≤ R[ j ]7. then A[k] ← L[ i ]8. i ←i + 19. else A[k] ← R[ j ]10. j ← j + 1
p q
7542
6321rq + 1
L
R
1 2 3 4 5 6 7 8
63217542
p rq
n1 n2
![Page 40: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/40.jpg)
5/11/2004 Lecture 2 COSC3101A 40
Example: MERGE(A, 9, 12, 16)p rq
![Page 41: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/41.jpg)
5/11/2004 Lecture 2 COSC3101A 41
Example: MERGE(A, 9, 12, 16)
![Page 42: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/42.jpg)
5/11/2004 Lecture 2 COSC3101A 42
Example (cont.)
![Page 43: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/43.jpg)
5/11/2004 Lecture 2 COSC3101A 43
Example (cont.)
![Page 44: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/44.jpg)
5/11/2004 Lecture 2 COSC3101A 44
Example (cont.)
Done!
![Page 45: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/45.jpg)
5/11/2004 Lecture 2 COSC3101A 45
Running Time of Merge
• Initialization (copying into temporary arrays):
(n1 + n2) = (n)
• Adding the elements to the final array (the last
for loop):
– n iterations, each taking constant time (n)
• Total time for Merge: (n)
![Page 46: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/46.jpg)
5/11/2004 Lecture 2 COSC3101A 46
Analyzing Divide-and Conquer Algorithms
• The recurrence is based on the three steps of the paradigm:– T(n) – running time on a problem of size n– Divide the problem into a subproblems, each of size
n/b: takes D(n)– Conquer (solve) the subproblems aT(n/b) – Combine the solutions C(n)
(1) if n ≤ c
T(n) = aT(n/b) + D(n) + C(n)otherwise
![Page 47: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/47.jpg)
5/11/2004 Lecture 2 COSC3101A 47
MERGE-SORT Running Time
• Divide: – compute q as the average of p and r: D(n) = (1)
• Conquer: – recursively solve 2 subproblems, each of size n/2
2T (n/2)
• Combine: – MERGE on an n-element subarray takes (n) time
C(n) = (n)
(1) if n =1
T(n) = 2T(n/2) + (n) if n > 1
![Page 48: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/48.jpg)
5/11/2004 Lecture 2 COSC3101A 48
Correctness of Merge Sort
• Loop invariant
(at the start of the for loop)
– A[p…k-1] contains the k-p smallest elements
of L[1 . . n1 + 1] and R[1 . . n2 + 1] in
sorted order
– L[i] and R[j] are the smallest elements not yet
copied back to A
p r
![Page 49: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/49.jpg)
5/11/2004 Lecture 2 COSC3101A 49
Proof of the Loop Invariant
• Initialization
– Prior to first iteration: k = p
subarray A[p..k-1] is empty
– A[p..k-1] contains the k – p = 0 smallest elements
of L and R
– L and R are sorted arrays (i = j = 1)
L[1] and R[1] are the smallest elements in L and R
![Page 50: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/50.jpg)
5/11/2004 Lecture 2 COSC3101A 50
Proof of the Loop Invariant
• Maintenance
– Assume L[i] ≤ R[j] L[i] is the smallest element not
yet copied to A
– After copying L[i] into A[k], A[p..k] contains the k – p + 1 smallest elements of L and R
– Incrementing k (for loop) and i reestablishes the loop
invariant
![Page 51: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/51.jpg)
5/11/2004 Lecture 2 COSC3101A 51
Proof of the Loop Invariant
• Termination
– At termination k = r + 1
– By the loop invariant: A[p..k-1] = A[p…r] contains
the k – p = r – p + 1 smallest elements of L and R in
sorted order
– Exactly the number of elements to be sorted
MERGE(A, p, q, r) is correctk = r + 1
![Page 52: COSC 3101A - Design and Analysis of Algorithms 2](https://reader035.vdocuments.site/reader035/viewer/2022062721/568136df550346895d9e79a3/html5/thumbnails/52.jpg)
5/11/2004 Lecture 2 COSC3101A 52
Readings
• Chapters 2.2, 3• Appendix A