Divide & ConquerThemes

Reasoning about code (correctness and cost) iterative code, loop invariants, and sums recursion, induction, and recurrence relationsDivide and Conquer

Examples sorting (insertion sort & merge sort) computing powersEuclidean algorithm (computing gcds)

Identities for Sums∑∑ =++=++=ii

ifcffccfcficf )()()( 2121

( ) ∑∑∑ +=+iii

igifigif )()()()(

∑∑∑ ≠iii

igifigif )()()()(

Some Common Sums

( )

( ) ( )

10,1

1

1,1

16

1212

1

,)1(

0

1

0

1

2

10

<<−

=

≠−

−=

++=

+==

≥+−=

∑

∑

∑

∑∑

∑

∞

=

+

=

=

==

=

rr

r

rrrr

mmmi

mmii

abcabc

i

i

mm

i

i

m

i

m

i

m

i

b

ai

Arithmetic SeriesSum of consecutive integers (written slightly

differently):

2)1(

)1(321

1

1

1

nnb

bnbbbibbin

i

n

i

−=

−++++== ∑∑−

=

−

=

See http://www.cs.drexel.edu/~kschmidt/CS520/Lectures/2/arithmetic.swf

Arithmetic Series (Proof)

2)1)(2(

2)1()1(22/)1()1(

inductionBy

)1(

1n sizefor trueisit implies assumption theshow andn sizefor trueis theorem theAssume

2/)11(11 :1)(n case Base

1

1

1

1

1

++=+++=

+++=

++=

+

+===

∑∑

∑

=

+

=

=

nnnnnnnn

ini

i

n

i

n

i

i

Geometric Series

= 1 + x + x2 + … + xn-1 = ∑−

=

1

0

n

i

ix

≠−−

=

1,11

1,

xx

xn

xn

See http://www.cs.drexel.edu/~kschmidt/CS520/Lectures/2/geometric.swf

Geometric Series (Proof)

)1()1(

)1()1()1(

)1()1(

induction by which

n. sizefor show and 1-n sizefor trueis theorem theAssume

)1()1(1 :0)(n case Base

1 xAssume n times. one sumsimply 1 When x

1

1

00

0

0

−−

=−

−+−=

−−

+=

+=

−−===

≠=

+

−

==

=

∑∑

∑

xxx

x

xx

xxx

xx

xxx

x

nnn

nn

n

i

inn

i

i

ii

Floor and CeilingLet x be a real numberThe floor of x, x, is the largest integer less than

or equal to x If an integer k satisfies k ≤ x < k+1, k = xE.G. 3.7 = 3, 3 = 3

The ceiling of x, x, is the smallest integer greater than or equal to x If an integer k satisfies k-1 < x ≤ k, k = xE.G. 3.7 = 4, 3 = 3

logarithmy = logb(x) ⇒ by = x

Two important cases ln(x) = loge(x) lg(x) = log2(x) [frequently occurs in CS]

Properties log(cd) = log(c) + log(d) log(cx) = xlog(c) logb(bx) = x = blogb(x) d ln(x)/dx = 1/x

logarithm

2k ≤ x < 2k+1 ⇒ k = lg(x)E.G. 16 ≤ 25 < 32 ⇒ 4 ≤ lg(25) < 5lg(25) ≈ 4.64

Change of baselogc(x) = logb(x) / logb(c) Proof. y = logc(x) ⇒ cy = x ⇒ ylogb(c) = logb(x)

⇒ y = logb(x) / logb(c)

Insertion Sort

To sort x0,…,xn-1, recursively sort x0,…,xn-2

insert xn-1 into x0,…,xn-2

(see code for details)Loop invariant (just before test, i<n)

x0,…, xi-1 sortedinitialize t = xi

Insertion Sort (Example)(7,6,5,4,3,2,1,0)after recursive call (1,2,3,4,5,6,7,0)Insert 0 into sorted subarray

Let 0 “fall” as far as it canNumber of comparisons to sort inputs that

are in reverse sorted order (worst case) C(n) = C(n-1) + (n-1) C(1) = 0

See (http://www.cs.drexel.edu/~kschmidt/CS520/Programs/sorts.cc)

Merge Sort

To sort x0,…,xn-1, recursively sort x0,…,xa-1 and xa,…,xn-1, where a

= n/2merge two sorted lists

Insertion sort is a special case where a=1loop invariant for merge similar to insert

(depends on implementation)

Merge Sort (Example)(7,6,5,4,3,2,1,0)after recursive calls (4,5,6,7) and (0,1,2,3)Number of comparisons needed to sort, worst case

(the merged lists are perfectly interleaved) M(n) = 2M(n/2) + (2n-2) M(1) = 0What is the best case (all in one list > other list)?

M(n) = 2M(n/2) + n/2

Comparison of Insertion and Merge Sort

Count the number of comparisons for different n=2k (see and run sort.cpp)

M(n)/C(n) →0 as n increasesC(n) is of higher order. I.e.,

C(n) bounds M(n) from above, but not tightly

C(2n)/C(n) ≈ 4So, apparently quadratic.C(n) = Θ(n2)

M(2n)/M(n) → 2 as n increases

n M(n) C(n) M(n)/C(n)

2 1 1 1

4 4 6 0.667

8 12 28 0.429

16 32 120 0.267

32 80 496 0.161

64 192 2016 0.095

128 4481 8128 0.055

Solve Recurrence for C(n)

2/)1(

)()1()()(

)1()2()2()1()1()(

1

1

1

11

−==

−+=−+−=

−+−+−=−+−=

∑

∑∑−

=

−

==

nni

inCinknC

nnnCnnCnC

n

i

n

i

k

i

See (http://www.cs.drexel.edu/~kschmidt/CS520/Lectures/2/insertionSortSwaps.swf)

Solve Recurrence for M(n)

)2/(*)lg()2/()2/()2/(2

...)2/(3)2/(2

)2/(2]2/)2/(2[2

)2/(2)2/(2

2/]4/)4/(2[22/)2/(2)(

2 n Assume

33

332

22

nnnknknM

nnM

nnnM

nnM

nnnMnnMnM

kk

k

==+

+=

++=

+=

=++=+=

=

See http://www.cs.drexel.edu/~kschmidt/CS520/Lectures/2/mergeSortSwaps.swf

Computing Powers

Recursive definition, multiplying as we learned in 4th grade (whatever):an = a × an-1, n > 0a0 = 1

Number of multiplicationsM(n) = M(n-1) + 1, M(0) = 0M(n) = n

Binary Powering (Recursive)Binary powering(see http://www.cs.drexel.edu/~kschmidt/CS520/Programs/power.cc)

x16 = (((x2)2)2)2, 16 = (10000)2

x23 = (((x2)2x)2x)2x, 23 = (10111)2

Recursive (right-left) xn = (xn/2)2 × x(n % 2)

M(n) = M(n/2) + [n % 2]

Binary Powering (Iterative)

Loop invariant xn = y × zN

N = n; y = 1; z = x;while (N != 0) { if (N % 2 == 1) y = z*y; z = z*z; N = N/2; }

• Example N y z23 1 x11 x x2

5 x3 x4 2 x7 x8

1 x7 x16

0 x23 x32

(See http://www.cs.drexel.edu/~kschmidt/CS520/Programs/power.cc

Binary Powering

Number of multiplicationsLet ν(n) = number of 1bits in binary

representation of nnum bits = lg(n) +1

M(n) = lg(n) + ν(n)ν(n) ≤ lg(n) =>M(n) ≤ 2lg(n)

Greatest Common Divisors

g = gcd(a,b) ⇔ g|a and g|b if e|a and e|b ⇒ e|g

gcd(a,0) = agcd(a,b) = gcd(b,a%b)

since if g|a and g|b then g|a%b and if g|b and g|a%b then g|a

Euclidean Algorithm (Iterative)(see http://www.cs.drexel.edu/~kschmidt/CS520/Programs/gcd.cc)

a0 = a, a1 = bai = qi ai+1+ ai+2 , 0 ≤ ai+2 < ai+1

…an = qn an+1

g = an+1

Number of Divisions

ai = qi ai+1+ ai+2 , 0 ≤ ai+2 < ai+1 =>ai ≥ qi ai+2+ ai+2 = (qi + 1)ai+2 ≥ 2ai+2

=> an ≥ 2an+2 => an / an+2 ≥ 2

a1 / a3 × a2 / a4 × … × an-1 / an+1 × an / an+2 ≥ 2n

⇒ n ≤ lg(a1a2 /g2)⇒ if a,b ≤ N, n ≤ 2lg(N)

(blank, for notes)