divide & conquer
DESCRIPTION
Divide & Conquer. Themes Reasoning about code (correctness and cost) iterative code, loop invariants, and sums recursion, induction, and recurrence relations Divide and Conquer Examples sorting (insertion sort & merge sort) computing powers Euclidean algorithm (computing gcds). - PowerPoint PPT PresentationTRANSCRIPT
Divide & ConquerThemes
Reasoning about code (correctness and cost) iterative code, loop invariants, and sums recursion, induction, and recurrence relationsDivide and Conquer
Examples sorting (insertion sort & merge sort) computing powersEuclidean algorithm (computing gcds)
Identities for Sums ii
ifcffccfcficf )()()( 2121
iii
igifigif )()()()(
iii
igifigif )()()()(
Some Common Sums
10,1
1
1,1
16
1212
1
,)1(
0
1
0
1
2
10
rr
r
rrrr
mmmi
mmii
abcabc
i
i
mm
i
i
m
i
m
i
m
i
b
ai
Arithmetic SeriesSum of consecutive integers (written slightly
differently):
2)1(
)1(321
1
1
1
nnb
bnbbbibbin
i
n
i
See http://www.cs.drexel.edu/~kschmidt/CS520/Lectures/2/arithmetic.swf
Arithmetic Series (Proof)
2)1)(2(
2)1()1(22/)1()1(
inductionBy
)1(
1n sizefor trueisit implies assumption theshow andn sizefor trueis theorem theAssume
2/)11(11 :1)(n case Base
1
1
1
1
1
nnnnnnnn
ini
i
n
i
n
i
i
Geometric Series
= 1 + x + x2 + … + xn-1 =
1
0
n
i
ix
1,11
1,
xx
xn
xn
See http://www.cs.drexel.edu/~kschmidt/CS520/Lectures/2/geometric.swf
Geometric Series (Proof)
)1()1(
)1()1()1(
)1()1(
induction by which
n. sizefor show and 1-n sizefor trueis theorem theAssume
)1()1(1 :0)(n case Base
1 xAssume n times. one sumsimply 1 When x
1
1
00
0
0
xxx
x
xx
xxx
xx
xxx
x
nnn
nn
n
i
inn
i
i
ii
Floor and CeilingLet x be a real numberThe floor of x, x, is the largest integer less than
or equal to x If an integer k satisfies k x < k+1, k = xE.G. 3.7 = 3, 3 = 3
The ceiling of x, x, is the smallest integer greater than or equal to x If an integer k satisfies k-1 < x k, k = xE.G. 3.7 = 4, 3 = 3
logarithmy = logb(x) by = x
Two important cases ln(x) = loge(x) lg(x) = log2(x) [frequently occurs in CS]
Properties log(cd) = log(c) + log(d) log(cx) = xlog(c) logb(bx) = x = blogb(x) d ln(x)/dx = 1/x
logarithm2k x < 2k+1 k = lg(x)
E.G. 16 25 < 32 4 lg(25) < 5lg(25) 4.64
Change of baselogc(x) = logb(x) / logb(c)
Proof. y = logc(x) cy = x ylogb(c) = logb(x) y = logb(x) / logb(c)
Insertion Sort
To sort x0,…,xn-1, recursively sort x0,…,xn-2
insert xn-1 into x0,…,xn-2
(see code for details)Loop invariant (just before test, i<n)
x0,…, xi-1 sortedinitialize t = xi
Insertion Sort (Example)(7,6,5,4,3,2,1,0)after recursive call (1,2,3,4,5,6,7,0)Insert 0 into sorted subarray
Let 0 “fall” as far as it canNumber of comparisons to sort inputs that
are in reverse sorted order (worst case) C(n) = C(n-1) + (n-1) C(1) = 0
See (http://www.cs.drexel.edu/~kschmidt/CS520/Programs/sorts.cc)
Merge Sort
To sort x0,…,xn-1, recursively sort x0,…,xa-1 and xa,…,xn-1, where a
= n/2merge two sorted lists
Insertion sort is a special case where a=1loop invariant for merge similar to insert
(depends on implementation)
Merge Sort (Example)(7,6,5,4,3,2,1,0)after recursive calls (4,5,6,7) and (0,1,2,3)Number of comparisons needed to sort, worst case
(the merged lists are perfectly interleaved) M(n) = 2M(n/2) + (2n-2) M(1) = 0What is the best case (all in one list > other list)?
M(n) = 2M(n/2) + n/2
Comparison of Insertion and Merge Sort
Count the number of comparisons for different n=2k (see and run sort.cpp)
M(n)/C(n) 0 as n increasesC(n) is of higher order. I.e.,
C(n) bounds M(n) from above, but not tightly
C(2n)/C(n) 4So, apparently quadratic.C(n) = Θ(n2)
M(2n)/M(n) 2 as n increases
n M(n) C(n) M(n)/C(n)
2 1 1 1
4 4 6 0.667
8 12 28 0.429
16 32 120 0.267
32 80 496 0.161
64 192 2016 0.095
128 4481 8128 0.055
Solve Recurrence for C(n)
2/)1(
)()1()()(
)1()2()2()1()1()(
1
1
1
11
nni
inCinknC
nnnCnnCnC
n
i
n
i
k
i
See (http://www.cs.drexel.edu/~kschmidt/CS520/Lectures/2/insertionSortSwaps.swf)
Solve Recurrence for M(n)
)2/(*)lg()2/()2/()2/(2
...)2/(3)2/(2
)2/(2]2/)2/(2[2
)2/(2)2/(2
2/]4/)4/(2[22/)2/(2)(
2 n Assume
33
332
22
nnnknknM
nnM
nnnM
nnM
nnnMnnMnM
kk
k
See http://www.cs.drexel.edu/~kschmidt/CS520/Lectures/2/mergeSortSwaps.swf
Computing PowersRecursive definition, multiplying as we
learned in 4th grade (whatever):an = a an-1, n > 0a0 = 1
Number of multiplicationsM(n) = M(n-1) + 1, M(0) = 0M(n) = n
Binary Powering (Recursive)Binary powering(see http://www.cs.drexel.edu/~kschmidt/CS520/Programs/power.cc)
x16 = (((x2)2)2)2, 16 = (10000)2
x23 = (((x2)2x)2x)2x, 23 = (10111)2
Recursive (right-left) xn = (xn/2)2 x(n % 2)
M(n) = M(n/2) + [n % 2]
Binary Powering (Iterative)
Loop invariant xn = y zN
N = n; y = 1; z = x;while (N != 0) { if (N % 2 == 1) y = z*y; z = z*z; N = N/2; }
• Example N y z1 1 x11 x x2
5 x3 x4 2 x7 x8
1 x7 x16
0 x23 x32
(See http://www.cs.drexel.edu/~kschmidt/CS520/Programs/power.cc
Binary PoweringNumber of multiplicationsLet (n) = number of 1bits in binary
representation of nnum bits = lg(n) +1
M(n) = lg(n) + (n)(n) ≤ lg(n) =>M(n) ≤ 2lg(n)
Greatest Common Divisorsg = gcd(a,b) g|a and g|b if e|a and e|b e|g
gcd(a,0) = agcd(a,b) = gcd(b,a%b)
since if g|a and g|b then g|a%b and if g|b and g|a%b then g|a
Euclidean Algorithm (Iterative)(see http://www.cs.drexel.edu/~kschmidt/CS520/Programs/gcd.cc)
a0 = a, a1 = bai = qi ai+1+ ai+2 , 0 ai+2 < ai+1
…an = qn an+1
g = an+1
Number of Divisions
ai = qi ai+1+ ai+2 , 0 ai+2 < ai+1 =>
ai qi ai+2+ ai+2 = (qi + 1)ai+2 2ai+2
=> an 2an+2 => an / an+2 2
a1 / a3 a2 / a4 … an-1 / an+1 an / an+2 2n
n lg(a1a2 /g2) if a,b N, n 2lg(N)
(blank, for notes)