cs 253: algorithms chapter 4 divide-and-conquer recurrences master theorem credit: dr. george bebis

CS 253: Algorithms

Chapter 4

Divide-and-Conquer

Recurrences

Master Theorem

Credit: Dr. George Bebis

Recurrences and Running Time

Recurrences arise when an algorithm contains recursive calls

to itself

Running time is represented by an equation or inequality that

describes a function in terms of its value on smaller inputs.

T(n) = T(n-1) + n

What is the actual running time of the algorithm? i.e. T(n)

= ?

Need to solve the recurrence

◦ Find an explicit formula of the expression

◦ Bound the recurrence by an expression that involves n

Example Recurrences

T(n) = T(n-1) + n Θ(n2)Recursive algorithm that loops through the input to eliminate one item

T(n) = T(n/2) + c Θ(lgn)Recursive algorithm that halves the input in one step

T(n) = T(n/2) + n Θ(n)Recursive algorithm that halves the input but must examine every item in the input

T(n) = 2T(n/2) + 1 Θ(n)Recursive algorithm that splits the input into 2 halves and does a constant amount of other work

BINARY-SEARCH

Finds if x is in the sorted array A[lo…hi]

Alg.: BINARY-SEARCH (A, lo, hi, x)

if (lo > hi)return FALSE

mid (lo+hi)/2if x = A[mid]

return TRUEif ( x < A[mid] )

BINARY-SEARCH (A, lo, mid-1, x)if ( x > A[mid] )

BINARY-SEARCH (A, mid+1, hi, x)

12

11

10

97532

1 2 3 4 5 6 7 8

midlo hi

Example 1

A[8] = {1, 2, 3, 4, 5, 7, 9, 11} lo = 1 hi = 8 x = 7

mid = 4, lo = 5, hi = 8

mid = 6, A[mid] = x Found!

119754321

119754321

1 2 3 4 5 6 7 8

8765

A[8] = {1, 2, 3, 4, 5, 7, 9, 11}lo = 1 hi = 8 x = 6

lo = 1, hi = 8, mid = 4. A[4]=4

lo = 5, hi = 8, mid = 6, A[6]=7

11

9754321

11

9754321

1 2 3 4 5 6 7 8

11

9754321 lo = 5, hi = 5, mid = 5, A[5]=5

11

9754321

low high

low

lowhigh

high

lo = 6, hi = 5 NOT FOUND!

Example 1I

Analysis of BINARY-SEARCH

Alg.: BINARY-SEARCH (A, lo, hi, x)if (lo > hi)

return FALSEmid (lo+hi)/2if x = A[mid]

return TRUEif ( x < A[mid] )

BINARY-SEARCH (A, lo, mid-1, x)if ( x > A[mid] )

BINARY-SEARCH (A, mid+1, hi, x)

T(n) = c + T(n/2)

constant time: c2

same problem of size n/2

same problem of size n/2

constant time: c1

constant time: c3

8

Methods for Solving Recurrences

Iteration method

Recursion-tree method

Master method

The Iteration Method

Convert the recurrence into a summation and solve it using a

known series

Example: T(n) = c + T(n/2)

T(n) = c + T(n/2)

= c + c + T(n/4)

= c + c + c + T(n/8)

= c + c + c + c + T(n/24)

Assume n=2k then k = lg n and

T(n) = c + c + c + c + c + … + T(n/2k)

(k times)

T(n) = k * c + T(1)

T(n) = clg n

Iteration Method – Example 2

T(n) = n + 2T(n/2) Assume n=2k k = lg

n

T(n) = n + 2T(n/2)

= n + 2(n/2 + 2T(n/4))

= n + n + 4T(n/4)

= n + n + 4(n/4 + 2T(n/8))

= n + n + n + 8T(n/8)

T(n) = 3n + 23T(n/23)

= kn + 2kT(n/2k)

= nlgn + nT(1)

T(n) = O(nlgn)

11


Iteration method


Master method

The recursion-tree method

Convert the recurrence into a tree:

◦ Each node represents the cost incurred at various

levels of recursion

◦ Sum up the costs of all levels

Used to “guess” a solution for the recurrence

Example 1 W(n) = 2W(n/2) + n2

Subproblem size at level i = n/2i

At level i: Cost of each node = (n/2i)2 # of nodes = 2i Total cost = (n2/2i)

h = Height of the tree n/2h=1 h = lgn Total cost at all levels:

W(n) = O(n2)

22

0

2lg

0

2lg

0

2

2

21

1

1

2

1

2

1

2)( nnnn

nnW

i

in

i

in

ii

Example 2 T(n) = 3T(n/4) + cn2

Subproblem size at level i = n/4i

At level i: Cost of each node= c(n/4i)2 # of nodes= 3i Total cost =cn2(3/16)i

h = Height of the tree n/4h=1 h = log4n

Total cost at all levels: (last level has 3log4n = nlog

43 nodes)

T(n) = O(n2)

)(

163

1

1

16

3

16

3)( 23log23log2

0

3log21log

0

444

4

nOncnncnncnnTi

iin

i

15

Example 3

• The longest path from the root to a

leaf:

n (2/3)n (2/3)2 n … 1

(2/3)in =1 i = log3/2n

• Cost of the problem at level i = n

• Total cost:

via further analysis W(n) =Θ(nlgn)

W(n) = W(n/3) + W(2n/3) + n

3/2

lg( ) (log ) ( lg )

lg(3 / 2)

nW n n n n O n n

16


Iteration method


Master method

Master Theorem

“Cookbook” for solving recurrences of the form:

Idea: compare f(n) with nlogba

f(n) is asymptotically smaller or larger than

nlogba by a polynomial factor n

OR

f(n) is asymptotically equal with nlogba

0 and 1, b 1, a where )()(

f(n)nfb

naTnT

18

Master Theorem

Case 1: if f(n) = O(nlogba -) for some > 0, then: T(n) =

(nlogba)

Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)

Case 3: if f(n) = (nlogba +) for some > 0, and if

af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n,

then:

T(n) = (f(n))

regularity condition

0 and 1, b 1, a where )()(

f(n)nfb

naTnT

19

Example 1

T(n) = 2T(n/2) + n

a = 2, b = 2, log22 = 1

Compare nlogba=n1 with f(n) = n

f(n) = (nlogba=n1) Case 2

T(n) = (nlogba lgn) = (nlgn)

Case 1: if f(n) = O(nlogba -) for some > 0, then: T(n) = (nlogba)



af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then: T(n)

=(f(n))

T(n) = 2T(n/2) + n2

a = 2, b = 2, log22 = 1

Compare nlog22=n1 with f(n) = n2

f(n) = (nlog22+) Case 3 (* need to verify regularity

cond.)

a f(n/b) ≤ c f(n) 2 n2/4 ≤ c n2 (½ ≤ c <1)

T(n) = (f(n)) = (n2)

Example 2Case 1: if f(n) = O(nlogba -) for some > 0, then: T(n) = (nlogba)




=(f(n))

21

T(n) = 2T(n/2) + √n

a = 2, b = 2, log22 = 1

Compare n with f(n) = n1/2

f(n) = O(n1-) Case 1

T(n) = (nlogba) = (n)





=(f(n))

T(n) = 3T(n/4) + nlgn

a = 3, b = 4, log43 = 0.793

Compare n0.793 with f(n) = nlgn

f(n) = (nlog43+) Case 3

Check regularity condition:

3(n/4)lg(n/4) ≤ (3/4)nlgn = c f(n), (3/4 ≤ c < 1)

T(n) = (nlgn)





=(f(n))

T(n) = 2T(n/2) + nlgn

a = 2, b = 2, log22 = 1

Compare n with f(n) = nlgn

Is this a candidate for Case 3 ?

Case 3: if f(n) = (nlogba +) for some > 0

In other words, If nlgn ≥ n1.n for some > 0

lgn is asymptotically less than n for any > 0 !!

Therefore, this in not Case 3! (somewhere between Case 2 & 3)





=(f(n))

T(n) = 2T(n/2) + nlgn

Use Iteration Method to show that

T(n) = nlg2n

T(n) = nlgn + 2T(n/2)

= nlgn + 2(n/2*lg(n/2) + 2T(n/4))

= nlgn + nlg(n/2) + 22T(n/22)

= ….

Exercise:

25

Changing variables

Try to transform the recurrence to one that you have seen before

◦Rename: m = lgn n = 2m

T (2m) = 2T(2m/2) + m

◦Rename: k=m and S(k) = T(2k)

S(k) = 2S(k/2) + k S(k) = (k*lgk)

T(n) = T(2m) = S(m) = (mlgm)= (lgn*lglgn)

**See Page 86 of the Textbook.

T(n) = 2T( ) + lgn n

26

Use Iteration Method to show that

T(n) = (lgn*lglgn)

T(n) = lgn + 2T(n1/2)

= lgn + 2(lg(n1/2) + 2T(n1/4))

= lgn + lgn + 22T(n1/4)

= lgn + lgn + … + 2kT(2) k=lglgn

T(n) = 2T( ) + lgn n

Exercise:

Appendix A

Summations

2

)1(321

1

nnnk

n

k

6

)12)(1(321

1

22222

nnnnk

n

k

4

)1(321

22

1

33333

nnnk

n

k

xxx

x

xxxxx

n

k

k

n

nn

k

nk

1

1lim then 1|| If

1

11

0

1

0

2

)1(ln1

3

1

2

11

1 Series Harmonic

1

Onnk

Hn

kn

cs 253: algorithms chapter 4 divide-and-conquer recurrences master theorem credit: dr. george bebis

Documents