cosc 3101winter, 2004 cosc 3101 design and analysis of algorithms lecture 2. relevant mathematics:...

COSC 3101 Winter, 2004

COSC 3101 Design and Analysis of Algorithms

Lecture 2. Relevant Mathematics:

–The time complexity of an algorithm

–Adding made easy

–Recurrence relations


Question from last class: recurrence relations•T(1) = k for some constant k•T(n) = 4 T(n/2) + k’ n + k’’ for some constants k’ and k’’

MULT(X,Y):

If |X| = |Y| = 1 then RETURN XY

Break X into a;b and Y into c;d

RETURN

MULT(a,c) 2n + (MULT(a,d) + MULT(b,c)) 2n/2 + MULT(b,d)Question from last class: how does Mult know the length of X and Y (i.e. n)?

Answer: data type of X, Y, a, b, c, d must include length field.


The Time Complexity of an Algorithm

Specifies how the running time depends on the size of the input.


Purpose?


Purpose

• To estimate how long a program will run.

• To estimate the largest input that can reasonably be given to the program.

• To compare the efficiency of different algorithms.

• To help focus on the parts of code that are executed the largest number of times.

• To choose an algorithm for an application.


Time Complexity Is a Function

• Specifies how the running time depends on the size of the input.

• A function mapping “size” of input “time” T(n) executed .


Definition of Time?


Definition of Time

• # of seconds (machine dependent).

• # lines of code executed.

• # of times a specific operation is performed (e.g., addition).

Which?


Definition of Time

• # of seconds (machine dependent). • # lines of code executed. • # of times a specific operation is performed

(e.g., addition).

• These are all reasonable definitions of time, because they are within a constant factor of each other.


Size of Input Instance?

83920

Suppose input to the algorithm is the integer 83920.


Size of Input Instance

• Size of paper:• # of bits:• # of digits:• Value:

n = 2 in2

n = 17 bits

n = 5 digits

n = 83920

2’’

83920

5

1’’

Which are reasonable?



• Intuitive

2’’

83920

5

1’’


n = 2 in2

n = 17 bits

n = 5 digits

n = 83920




n = 2 in2

n = 17 bits

n = 5 digits

n = 83920

• Intuitive• Formal

2’’

83920

5

1’’




n = 2 in2

n = 17 bits

n = 5 digits

n = 83920

• Intuitive• Formal• Reasonable

# of bits =

3.32 * # of digits

2’’

83920

5

1’’



• Size of paper• # of bits• # of digits• Value

- n = 2 in2

- n = 17 bits

- n = 5 digits

- n = 83920

• Intuitive• Formal• Reasonable• Unreasonable

# of bits = log2(Value) Value = 2# of bits

2’’

83920

5

1’’


Two Example Algorithms

• Sum N array entries: A(1) + A(2) + A(3) + …

• Factor value N: Input N=5917 & Output N=97*61.

Algorithm: N/2, N/3, N/4, …. ?

Time?





Algorithm N/2, N/3, N/4, …. ?

Time = N

Time = N






Is this reasonable?

Time = N

Time = N






No!

Time = N

Time = N

One is considered fast and the other slow!






size n = N

size n = log N






Time as function of input size?

size n = N

size n = log N






Time = N = n

Time = N = 2n

size n = N

size n = log N






Linear vs Exponential Time!

Time = N = n

Time = N = 2n

size n = N

size n = log N



14,23,25,30,31,52,62,79,88,98



• # of elements: n = 10 elements

14,23,25,30,31,52,62,79,88,98

10



• # of elements: n = 10 elements

14,23,25,30,31,52,62,79,88,98

10

Is this reasonable?



• # of elements: n = 10 elements ~ Reasonable

14,23,25,30,31,52,62,79,88,98

10

If each element has size c

# of bits = c * # of elements



• # of elements: n = 10 elements Reasonable

6, 3, 8, 1, 2, 10, 4, 9, 7, 5

10

If each element is in [1..n]

each element has size log n

# of bits = n log n ≈ n


Time Complexity Is a Function

• Specifies how the running time depends on the size of the input.

• A function mapping: – “size” of input “time” T(n) executed .

• Or more precisely:– # of bits n needed to represent the input # of

operations T(n) executed .


Which Input of size n?

There are 2n inputs of size n.Which do we consider

for the time T(n)?


Which Input of size n?

• Provides an upper bound for all possible inputs.

• Easiest to compute

Worst Case

For what distribution?Average Case

But what is typical?Typical Input


Time Complexity of an Algorithm

• O(n2): Prove that for every input of size n, the algorithm takes no more than cn2 time.

• Ω(n2): Find one input of size n, for which the algorithm takes at least this much time.

• θ (n2): Do both.

The time complexity of an algorithm isthe largest time required on any input of size n.


What is the height of tallest person in the class?

Bigger than this?

Need to look at only one person

Need to look at every person

Smaller than this?


Time Complexity of a Problem

• O(n2): Provide an algorithm that solves the problem in no more than this time.

• Ω(n2): Prove that no algorithm can solve it faster.

• θ (n2): Do both.

The time complexity of a problem is the time complexity of the fastest algorithm that solves the problem.


Adding: The Classic Techniques

Evaluating ∑i=1 f(i).n


∑i=1..n i = 1 + 2 + 3 + . . . + n

= ?

Arithmetic Sum


1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S


1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S

2

1)(n n S


1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S

Let’s restate this argument using a

geometric representation

Algebraic argument


1 2 . . . . . . . . n

= number of white dots.1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S


1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S

1 2 . . . . . . . . n

= number of white dots

= number of yellow dots

n . . . . . . . 2 1


1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S

n+1 n+1 n+1 n+1 n+1



n

n

n

n

n

n

There are n(n+1) dots in the grid


1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

(n+1) + (n+1) + (n+1) + . . . + (n+1) + (n+1) = 2S

n (n+1) = 2S

n+1 n+1 n+1 n+1 n+1



n

n

n

n

n

n

2

1)(n n S

Note = (# of terms · last term))


∑i=1..n i = 1 + 2 + 3 + . . . + n

= (# of terms · last term)

Arithmetic Sum

True whenever terms increase slowly


∑i=0..n 2i = 1 + 2 + 4 + 8 +. . . + 2n

= ?

Geometric Sum


Geometric Sum


∑i=0..n 2i = 1 + 2 + 4 + 8 +. . . + 2n

= 2 · last term - 1

Geometric Sum


∑i=0..n ri = r0 + r1 + r2 +. . . + rn

= ?

Geometric Sum


S

Sr r r r ... r r

S Sr 1 r

Sr 1

r 1

2 3 n n 1

n 1

n 1

1 r r r ...2 3 rn

Geometric Sum


∑i=0..n rir 1

r 1

n 1

Geometric SumWhen r>1?


∑i=0..n rir 1

r 1

n 1

Geometric Sum

θ(rn)Biggest TermWhen r>1


∑i=0..n ri = r0 + r1 + r2 +. . . + rn

= (biggest term)

Geometric Increasing

True whenever terms increase quickly


∑i=0..n ri

Geometric SumWhen r<1?

1 r

1 r

n 1


∑i=0..n ri

Geometric Sum

1 r

1 r

n 1

θ(1)When r<1

Biggest Term


∑i=0..n ri = r0 + r1 + r2 +. . . + rn

= (1)

Bounded Tail

True whenever terms decrease quickly


∑i=1..n 1/i

= 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + …+ 1/n

= ?

Harmonic Sum


f(i) = 1∑i=1..n f(i) = n

n

Sum of Shrinking Function


f(i) = ?∑i=1..n f(i) = n1/2

n



f(i) = 1/2i

∑i=1..n f(i) = 2



f(i) = 1/i∑i=1..n f(i) = ?

n



Harmonic Sum

2 2 21 1

From this it f ollows that

1 1 1log log 1 (log )

2

n n

i i

n n ni i

NB: Error in Jeff’s notes, p.30, bottom:


∑i=1..n 1/i

= 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + …+ 1/n

= (log(n))

Harmonic Sum


Approximating Sum by Integrating

The area under the curveapproximates the sum

∑i=1..n f(i) ≈ ∫x=1..n f(x) dx


Approximating Sums by Integrating:

1 1

0 0

1( ) ( )

1

(number of terms last term)

(True whenever terms increase slowly)

nnc c c c c

i

i x dx n n n nc

Arithmetic Sums


Approximating Sums by Integrating:

Geometric Sums

0 0

1( 1) ( )

ln

(last term)

(True whenever terms increase rapidly)

nni x n n

i

b b dx b bb


Harmonic Sum




Problem: Integrating may be hard too.


Adding Made Easy

We will now classify (most) functions f(i) into four classes:

–Geometric Like–Arithmetic Like–Harmonic–Bounded Tail

For each class, we will give an easy rule for approximating its sum θ( ∑i=1..n f(i) )


Classifying Animals

Vertebrates

Birds

Mam

mals

Reptiles

Fish

Dogs

Giraffe


Classifying Functions

Functions

Poly.

Exp.

2θ(n) ∑i=1..n f(i) = θ(f(n))

8·2n / n100 + n3

θ(2n / n100)

nθ(1) 2θ(n)

θ(2n)

f(n) = 8·2n / n100 + n3

θ(2n / n100) ∑i=1..n f(i) = θ(2n / n100)

20.5n < < 2n8· 2n / n100 + n3

Significant Less significant

Irrelevant


Adding Made Easy

11

1

2

34

Four Classes of Functions


Adding Made Easy


If the terms f(i) grow sufficiently quickly,

then the sum will be dominated by the largest

term.

f(n) 2Ω(n) ∑i=1..n f(i) = θ(f(n))

Geometric Like:

Classic example:

∑i=1..n 2i = 2n+1-1 ≈ 2 f(n)


If the terms f(i) grow sufficiently quickly,

then the sum will be dominated by the largest

term.

f(n) 2Ω(n) ∑i=1..n f(i) = θ(f(n))

Geometric Like:

For which functions f(i) is this true?

How fast and how slow can it grow?


r 1r 1

n 1

θ( )r n Last Term

when r>1.

∑i=1..n ri 1 r r r . . .2 3 r n

θ(f(n))

∑i=1..n (1.0001)i ≈ 10,000 (1.0001)n

= 10,000 f(n)Lower Extreme:

Upper Extreme: ∑i=1..n (1000)i ≈ 1.001(1000)n

= 1.001 f(n)

When f(n) = rn = 2θ(n)

Geometric Like:


Because the constant is sooo big, the statement is just barely true.

∑i=1..n (1.0001)i ≈ 10,000 (1.0001)n



= 1.001 f(n)

f(n) 2Ω(n) ∑i=1..n f(i) = θ(f(n))Geometric Like:


∑i=1..n (1.0001)i ≈ 10,000 (1.0001)n



= 1.001 f(n)

Even bigger?



2n2i

∑i=1..n 22 ≈ 22 = 1f(n)

No Upper Extreme:

Even bigger!

∑i=1..n (1.0001)i ≈ 10,000 (1.0001)n




2n2i

∑i=1..n 22 ≈ 22 = 1f(n)

No Upper Extreme:

Functions in between?

∑i=1..n (1.0001)i ≈ 10,000 (1.0001)n




f(n) 2Ω(n) ∑i=1..n f(i) = θ(f(n))

Geometric Like:

3100 100

2 2e.g. ( ) 8 ( ( )) ( )

(The strongest f unction determines the class.)

n n

f n n f nn n



f(n) = 2n8·n100

∑i=1..n f(i)f(n) c·f(n)f(i) ·f(n)f(i) ·f(i+1)

Easy Hard

(i+1)100

i1002

= (1+1/i)100

12 1.9

=8·2(i+1)

(i+1)100f(i+1)f(i) 8·2i

i100

=

1/.51

·f(i+1)f(i)


1001

2Example: ( ) 8 Claim: ( ) ( ( ))

n n

i

f n fi f nn

Proof :

1 1

1) ( ) ( ) ( ) ( ( ))n n

i i

fi f n fi f n

0 01

2) Seek 0, >0 such that ( ) ( ) n nn

i

c n fi cf n

0 0

( )Suppose that (*) 0 1

( 1)fi

n i nfi

0Then ( ) ( 1) f or some 1,fi fi i n ( ) ( ) ( ) ( )k n ifi fi k fi f n

1 1 1

1Thus ( ) ( ) ( ) ( ) ( ( ))

1

n n nn i

i i i

fi f n f n fi O f n

I t remains to prove (*)

100 100 100

1 100 100

( ) 2 / ( 1) (1 1/ )( 1) 2 / ( 1) 2 2

i

i

fi i i ifi i i

100.01

.01

( ) (1 1/ ) 1Thus 1 iff 1 1 1/ 2 143.8

( 1) 2 2 1fi i

i ifi

01 1

Thus (*) holds f or 144 ( ) ( ( )) ( ) ( ( ))n n

i i

n fi O f n fi f n


In general, if f(n) = c ban nd loge (n)c ?a ?b ?d ?e ?

f(n) = Ω, θ ?

Then ∑i=1..n f(i) = θ(f(n)).

2Ω(n)

> 0 0 1

(-,)(-,)

f(n) 2Ω(n) ∑i=1..n f(i) = θ(f(n))

Geometric Like:



Examples:


Do All functions in 2Ω(n) have this property?

Maybe not.



Functions that oscillate with exponentially increasing amplitude do not have this property.



Functions expressed with +, -, , , exp, log

do not oscillate continually.

They are well behaved for sufficiently large n.

These do have this property.



Adding Made Easy

Done


Adding Made Easy


If the terms f(i) are increasing or decreasing relatively slowly, then the sum is roughly the number of terms, n, times the final value.

f(n) = nθ(1)-1 ∑i=1..n f(i) = θ(n·f(n))Arithmetic Like:



Simple example:∑i=1..n 1 = n · 1




Is the statement true for this function?

∑i=1..n i = 1 + 2 + 3 + . . . + n



Half the terms are roughly the same (within a multiplicative constant)

∑i=1..n i =

1 + . . . + n/2 + . . . + n



But half the terms are roughly the same

and the sum is roughly the number

terms, n, times this value

∑i=1..n i =

1 + . . . + n/2 + . . . + n

∑i=1..n i = θ(n · n)



Is the statement true for this function?

∑i=1..n i2 = 12 + 22 + 32 + . . . + n2



Again half the terms are roughly the same.

∑i=1..n i =

12 + . . . + (n/2)2 + . . . + n2

1/4 n2



Again half the terms are roughly the same.

∑i=1..n i =

12 + . . . + (n/2)2 + . . . + n2

1/4 n2

∑i=1..n i2 = θ(n · n2)



∑i=1..n f(i) ≈ area under curve


Consider f(i) = nθ(1)

(i.e. f(i) increasing)


area of small square ∑i=1..n f(i)

≈ area under curve area of big square



n/2 · f(n/2)

= area of small square ∑i=1..n f(i)


= n · f(n)



θ(n · f(n)) = n/2 · f(n/2)



= n · f(n)



θ(n · f(n)) = n/2 · f(n/2)



= n · f(n)

?



∑i=1..n i2 = 12 + 22 + 32 + . . . + n2


f(n) = n2

f(n/2) = θ(f(n))

The key property is

= ?


The key property is

f(n/2) = (n/2)2 = 1/4 n2 = θ(n2) = θ(f(n))

∑i=1..n i2 = 12 + 22 + 32 + . . . + n2


f(n) = n2

= θ(n·f(n)) = θ(n · n2)


∑i=1..n f(i) = ?


f(n) = nθ(1)


∑i=1..n ir = 1r + 2r + 3r + . . . + nr


f(n) = nθ(1)

f(n/2) = θ(f(n))

The key property is


The key property is

f(n/2) = (n/2)r = 1/2r nr = θ(nr) = θ(f(n))

∑i=1..n ir = 1r + 2r + 3r + . . . + nr


= θ(n·f(n)) = θ(n · nr)

f(n) = nθ(1)


∑i=1..n 2i = 21 + 22 + 23 + . . . + 2n


f(n) = nθ(1)

f(n/2) = θ(f(n))

The key property is


The key property is

∑i=1..n 2i = 21 + 22 + 23 + . . . + 2n

f(n/2) = 2(n/2) = θ(2n) = θ(f(n))


f(n) = nθ(1)

= θ(n·f(n)) = θ(n · 2n)


Upper Extreme:

All functions in between.


1000 1001

1

1 1( )

1001 1001

n

i

i n n f n


Adding Made Easy

Half done


f(i) = 1∑i=1..n f(i) = n

n



f(i) = ?∑i=1..n f(i) = n1/2

n


1/i1/2


f(i) = 1/i∑i=1..n f(i) = log n

n



f(i) = 1/2i

∑i=1..n f(i) = 2



If most of the terms f(i) have roughly the same value, then the sum is roughly the

number of terms, n, times this value.

Does the statement hold for functions f(i) that shrink?



If most of the terms f(i) have roughly the same value, then the sum is roughly the

number of terms, n, times this value.

∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n

Does the statement hold for the Harmonic Sum?




∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n

∑i=1..n f(i) = ?

θ(n · f(n))




∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n

∑i=1..n f(i)

θ(n · f(n))

= ∑i=1..n 1/i = ?




∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n

∑i=1..n f(i)

θ(n · f(n))

= ∑i=1..n 1/i = θ(log n)


2 2 21 1


2

n n

i i

n n ni i



∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n

∑i=1..n f(i)

= θ(n · f(n))

= ∑i=1..n 1/i = θ(log n)

?




∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n

∑i=1..n f(i)

= θ(n · f(n))

= ∑i=1..n 1/i = θ(log n)

θ(1) = θ(n · 1/n)




∑i=1..n 1/i = 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + … + 1/n

∑i=1..n f(i)

= θ(n · f(n))

= ∑i=1..n 1/i = θ(log n)

θ(1) = θ(n · 1/n)≠

No the statement does not hold!



Adding Made Easy

not included


Does the statement hold for the almost Harmonic Sum?

∑i=1..n 1/i0.9999

Shrinks slightly slower than harmonic.




∑i=1..n 1/i0.9999

∑i=1..n f(i)

= θ(n · f(n))

= ∑i=1..n 1/i0.9999

= ?




∑i=1..n 1/i0.9999

∑i=1..n f(i)

= θ(n · f(n))

= ∑i=1..n 1/i0.9999

= θ(n0.0001)




∑i=1..n 1/i0.9999

∑i=1..n f(i)

= θ(n · f(n))

= ∑i=1..n 1/i0.9999

= θ(n0.0001)

?




∑i=1..n 1/i0.9999

∑i=1..n f(i)

= θ(n · f(n))

= ∑i=1..n 1/i0.9999

= θ(n0.0001)

θ(n0.0001) = θ(n · 1/n0.9999)




∑i=1..n 1/i0.9999

∑i=1..n f(i)

= θ(n · f(n))

= ∑i=1..n 1/i0.9999

= θ(n0.0001)

θ(n0.0001) = θ(n · 1/n0.9999)

=

The statement does hold!



∑i=1..n 1 = n · 1 = n · f(n)

Intermediate Case:

Lower Extreme: ∑i=1..n 1/i0.9999

= θ(n0.0001) = θ(n · f(n))




∑i=1..n 1 = n · 1 = n · f(n)

Intermediate Case:

Lower Extreme: ∑i=1..n 1/i0.9999

= θ(n0.0001) = θ(n · f(n))

Upper Extreme: ∑i=1..n i1000 = 1/1001 n1001

= 1/1001 n · f(n)



Arithmetic Like

If f(n) = c ban nd loge (n)c ?a ?b ?d ?e ?

f(n) = Ω, θ ?

then ∑i=1..n f(i) = θ(n · f(n)).

n-1+θ(1)

> 0 0

or b 1> -1

(-,)

(For +, -, , , exp, log functions f(n))

Conclusion


Adding Made Easy

Done


Adding Made Easy

Harmonic


Harmonic Sum

∑i=1..n 1/i

= 1/1 + 1/2 + 1/3 + 1/4 + 1/5 + …+ 1/n

= (log(n))

2 2 21 1


2

n n

i i

n n ni i


Adding Made Easy


If the terms f(i) decrease towards zerosufficiently quickly,

then the sum will be a constant.

The classic example ∑i=0..n 1/2i = 1 + 1/2 + 1/4 + 1/8 + … = 2.

f(n) n-1-Ω(1) ∑i=1..n f(i) = θ(1)Bounded Tail:


Upper Extreme: ∑i=1..n 1/i1.0001

= θ(1)



Upper Extreme: ∑i=1..n 1/i1.0001

= θ(1)

No Lower Extreme:2i

∑i=1..n 22 = θ(1).

1




Case 1:

( ) logan d ef n cb n n

0c

0 1b

0a

( , )d

( , )e

Case 2:

( ) logd ef n cn n

0c 1d ( , )e

Bounded TailConclusion


Adding Made Easy

Done


Adding Made EasyMissing Functions

n n

n nlo g

1/nlogn

logn/n


Adding Made Easy • Geometric Like: If f(n) 2Ω(n), then ∑i=1..n f(i) = θ(f(n)).

• Arithmetic Like: If f(n) = nθ(1)-1, then ∑i=1..n f(i) = θ(n · f(n)).

• Harmonic: If f(n) = 1/n , then ∑i=1..n f(i) = logen + θ(1).

• Bounded Tail: If f(n) n-1-Ω(1), then ∑i=1..n f(i) = θ(1).

(For +, -, , , exp, log functions f(n))

This may seem confusing, but it is really not.

It should help you compute most sums easily.


Recurrence Relations

T(1) = 1

T(n) = a T(n/b) + f(n)


Recurrence Relations Time of Recursive Program

procedure Eg(int n)

if(n1) then

put “Hi”

else

loop i=1..f(n)

put “Hi”

loop i=1..a

Eg(n/b)

Recurrence relationsarise from the timing of recursive programs.

Let T(n) be the # of “Hi”s on an input of “size” n.


procedure Eg(int n)

if(n1) then

put “Hi”

else

loop i=1..f(n)

put “Hi”

loop i=1..a

Eg(n/b)

Given size 1,the program outputs

T(1)=1 Hi’s.



procedure Eg(int n)

if(n1) then

put “Hi”

else

loop i=1..f(n)

put “Hi”

loop i=1..a

Eg(n/b)

Given size n,the stackframe outputs

f(n) Hi’s.



procedure Eg(int n)

if(n1) then

put “Hi”

else

loop i=1..f(n)

put “Hi”

loop i=1..a

Eg(n/b)

Recursing on a instances of size n/bgenerates T(n/b) “Hi”s.



procedure Eg(int n)

if(n1) then

put “Hi”

else

loop i=1..f(n)

put “Hi”

loop i=1..a

Eg(n/b)

Recursing a times

generates a·T(n/b) “Hi”s.



procedure Eg(int n)

if(n1) then

put “Hi”

else

loop i=1..f(n)

put “Hi”

loop i=1..a

Eg(n/b)

For a total of T(1) = 1 T(n) = a·T(n/b) + f(n)

“Hi”s.



Solving Technique 1Guess and Verify

•Recurrence Relation:

T(1) = 1 & T(n) = 4T(n/2) + n

•Guess: G(n) = 2n2 – n

•Verify: Left Hand Side Right Hand Side

T(1) = 2(1)2 – 1

T(n)

= 2n2 – n

1

4T(n/2) + n

= 4 [2(n/2)2 – (n/2)] + n

= 2n2 – n



T(1) = 1 & T(n) = 4T(n/2) + n

•Guess: G(n) = an2 + bn + c


T(1) = a+b+c

T(n)

= an2 +bn+c

1

4T(n/2) + n

= 4 [a (n/2)2 + b (n/2) +c] + n

= an2 +(2b+1)n + 4c

Solving Technique 2Guess Form and Calculate Coefficients



T(1) = 1 & T(n) = 4T(n/2) + n

•Guess: G(n) = an2 + bn + 0


T(1) = a+b+c

T(n)

= an2 +bn+c

1

4T(n/2) + n

= 4 [a (n/2)2 + b (n/2) +c] + n

= an2 +(2b+1)n + 4c


c=4cc=0



T(1) = 1 & T(n) = 4T(n/2) + n

•Guess: G(n) = an2 - 1n + 0


T(1) = a+b+c

T(n)

= an2 +bn+c

1

4T(n/2) + n

= 4 [a (n/2)2 + b (n/2) +c] + n

= an2 +(2b+1)n + 4c


b=2b+1b=-1



T(1) = 1 & T(n) = 4T(n/2) + n

•Guess: G(n) = an2 - 1n + 0


T(1) = a+b+c

T(n)

= an2 +bn+c

1

4T(n/2) + n

= 4 [a (n/2)2 + b (n/2) +c] + n

= an2 +(2b+1)n + 4c


a=a



T(1) = 1 & T(n) = 4T(n/2) + n

•Guess: G(n) = 2n2 - 1n + 0


T(1) = a+b+c

T(n)

= an2 +bn+c

1

4T(n/2) + n

= 4 [a (n/2)2 + b (n/2) +c] + n

= an2 +(2b+1)n + 4c


a+b+c=1a-1+0=1a=2



T(1) = 1 & T(n) = aT(n/b) + f(n)

Solving Technique 3Approximate Form and

Calculate Exponent

which is bigger?

Guess



T(1) = 1 & T(n) = aT(n/b) + f(n)

•Guess: aT(n/b) << f(n)

•Simplify: T(n) f(n)

Solving Technique 3Calculate Exponent

In this case, the answer is easy. T(n) = (f(n))



T(1) = 1 & T(n) = aT(n/b) + f(n)

•Guess: aT(n/b) >> f(n)

•Simplify: T(n) aT(n/b)


In this case, the answer is harder.



T(1) = 1 & T(n) = aT(n/b)

•Guess: G(n) = cn


T(n)

= cn aT(n/b)

= a [c (n/b) ]

= c a b n


(log a/log b)= cn

1 = a b-

b = alog b = log a = log a/log b



T(1) = 1 & T(n) = 4T(n/2)

•Guess: G(n) = cn


T(n)

= cn aT(n/b)

= c [a (n/b) ]

= c a b n


1 = a b-

b = alog b = log a = log a/log b

(log a/log b)= cn(log 4/log 2)= cn

2= cn



T(1) = 1 & T(n) = aT(n/b) + f(n)


If bigger then

T(n) = (f(n))

If bigger then(log a/log b)T(n) = (n )

And if aT(n/b) f(n) what is T(n) then?


Technique 4: Decorate The Tree

• T(n) = a T(n/b) + f(n)

f(n)

T(n/b) T(n/b) T(n/b) T(n/b)

T(n) =

T(1)

•T(1) = 1

1=

a


f(n)

T(n/b) T(n/b) T(n/b) T(n/b)

T(n) =a


f(n)

T(n/b) T(n/b) T(n/b)

T(n) =a

f(n/b)

T(n/b2)T(n/b2)T(n/ b2)T(n/ b2)

a


f(n)T(n) =a

f(n/b)


af(n/b)


af(n/b)


af(n/b)


a


11111111111111111111111111111111 . . . . . . 111111111111111111111111111111111

f(n)T(n) =a

f(n/b)


af(n/b)


af(n/b)


af(n/b)


a


Evaluating: T(n) = aT(n/b)+f(n)

Level

0

1

2

i

h



LevelInstance

size

0

1

2

i

h



LevelInstance

size

0 n

1

2

i

h



LevelInstance

size

0 n

1 n/b

2

i

h



LevelInstance

size

0 n

1 n/b

2 n/b2

i

h



LevelInstance

size

0 n

1 n/b

2 n/b2

i n/bi

h n/bh



LevelInstance

size

0 n

1 n/b

2 n/b2

i n/bi

h n/bh = 1

base case



LevelInstance

size

0 n

1 n/b

2 n/b2

i n/bi

h n/bh = 1



LevelInstance

size

0 n

1 n/b

2 n/b2

i n/bi

h = log n/log b n/bh = 1bh = nh log b = log nh = log n/log b



LevelInstance

size

Work

in stack

frame

0 n

1 n/b

2 n/b2

i n/bi

h = log n/log b 1



LevelInstance

size

Work

in stack

frame

0 n f(n)

1 n/b f(n/b)

2 n/b2 f(n/b2)

i n/bi f(n/bi)

h = log n/log b 1 T(1)



LevelInstance

size

Work

in stack

frame

# stack frames

0 n f(n)

1 n/b f(n/b)

2 n/b2 f(n/b2)

i n/bi f(n/bi)

h = log n/log b n/bh T(1)



LevelInstance

size

Work

in stack

frame

# stack frames

0 n f(n) 1

1 n/b f(n/b) a

2 n/b2 f(n/b2) a2

i n/bi f(n/bi) ai

h = log n/log b n/bh T(1) ah



LevelInstance

size

Work

in stack

frame

# stack frames

0 n f(n) 1

1 n/b f(n/b) a

2 n/b2 f(n/b2) a2

i n/bi f(n/bi) ai

h = log n/log b n/bh T(1) ah

ah = a

= n

log n/log b

log a/log b



LevelInstance

size

Work

in stack

frame

# stack frames

Work in Level

0 n f(n) 1

1 n/b f(n/b) a

2 n/b2 f(n/b2) a2

i n/bi f(n/bi) ai

h = log n/log b n/bh T(1) nlog a/log b



LevelInstance

size

Work

in stack

frame

# stack frames

Work in Level

0 n f(n) 1 1 · f(n)

1 n/b f(n/b) a a · f(n/b)

2 n/b2 f(n/b2) a2 a2 · f(n/b2)

i n/bi f(n/bi) ai ai · f(n/bi)

h = log n/log b n/bh T(1) nlog a/log b n · T(1)

log a/log b

Total Work T(n) = ∑i=0..h aif(n/bi)



= ∑i=0..h aif(n/bi)

If a Geometric Sum∑i=0..n xi θ(max(first term, last term))



LevelInstance

size

Work

in stack

frame

# stack frames

Work in Level

0 n f(n) 1 1 · f(n)

1 n/b f(n/b) a a · f(n/b)

2 n/b2 f(n/b2) a2 a2 · f(n/b2)

i n/bi f(n/bi) ai ai · f(n/bi)

h = log n/log b n/bh T(1) nlog a/log b n · T(1)

log a/log b

Dominated by Top Level or Base Cases


Master Theorem: I ntuition

Suppose ( ) ( / ) ( ), 1, 1T n aT n b f n a b

Work at top level ( ) f n

log log / logWork at bottom level number of base cases = b a a bn n

logRunning time = max(work at top, work at bottom) = max( ( ), )b af n n

I f they are equal, then all levels are important:

logRunning time = sum of work over all levels = logb an n


Theorem 4.1 (Master Theorem)

Suppose ( ) ( / ) ( ), 1, 1T n aT n b f n a b

log0 such that ( ) (1. I F )b af n O n

logTH ( ) (E )N b aT n n

log2. I F ( ) ( ) b af n n

log( ) (TH l g )EN o b aT n n n

log0 such that 3. I F ( ) ( )b af n n

AND

0 01, >0 such that ( / ) ( ) c n af n b cf n n n

( )TH ( ( )EN )T n f n

Dominated by base cases

Work at each level is comparable:

Sum work over all levels

Dominated by top level work

Additional regularity condition


3 5Example: ( ) 4 ( / 2) / logT n T n n n

4

2

a

blog 2b an n

3 5( ) / logf n n n

logThus ( ) ( ) (Case 3: dominated by top level)b af n n

0 0

1, >0

Additional r

such that

egularity condition:

( / ) ( ) c n af n b cf n n n

3 5 3 5Thus we require that 4(n/ 2) / log ( / 2) / logn cn n

5

.2log log12 1.15

2 log / 2 log / 2n n

cn n

1.15/ 0.152 203.2.n

5 5

0

log2561 1 8. . Let 256 0.975

2 log128 2 7e g n c

0Thus regularity condition holds f or 256, 0.98n c

3 5Thus ( ) ( ( )) ( / log ) T n f n n n


Example 2: ( ) 4 ( / 2) 2nT n T n

4

2

a

blog 2b an n

( ) 2nf n

logThus ( ) ( ) (Case 3: dominated by top level)b af n n

0 0

1, >0

Additional r

such that

egularity condition:

( / ) ( ) c n af n b cf n n n

n/ 2Thus we require that 4 2 2nc

/ 24 2 nc

0

1Let 6

2n c

0regularity condition holds f or 6, 0.5n c

Thus ( ) ( ( )) (2 ) nT n f n


5Example 2: ( ) 4 ( / 2) logT n T n n n

4

2

a

b log 2b an n

5( ) logf n n n

logThus ( ) ( ) (Case 1: dominated by base cases)b af n O n

logThus T(n)= ( ) b an


2Example 2: ( ) 4 ( / 2)T n T n n

4

2

a

b log 2b an n

2( )f n n

logThus ( ) ( ) (Case 2: all levels signifi cant)b af n n

logThus ( ) ( log ) b aT n n n


Sufficiently close to: T(n) = 4T(n/2)+ n3 = θ(n3). T2(n) = ?

Evaluating: T2(n) = 4 T2(n/2+n1/2)+ n3

θ(n3).


Evaluating: T(n) = aT(n-b)+f(n)

h = ? n-hb

n-ib

n-2b

n-b

T(0)

f(n-ib)

f(n-2b)

f(n-b)

f(n)n

Work in Level

# stack frames

Work

in stack

frame

Instance

size

a

ai

a2

a

1

n/b a · T(0)

ai · f(n-ib)

a2 · f(n-2b)

a · f(n-b)

1 · f(n)

n/b

|base case| = 0 = n-hb

h = n/b

h = n/b

i

2

1

0

Level

Likely dominated by base cases Exponential


Evaluating: T(n) = 1T(n-b)+f(n)

h = ?

Work in Level

# stack frames

Work

in stack

frame

Instance

size

1

1

1

1

1

f(0)

f(n-ib)

f(n-2b)

f(n-b)

f(n)

n-b

n

n-hb

n-ib

n-2b

T(0)

f(n-ib)

f(n-2b)

f(n-b)

f(n)

h = n/b

i

2

1

0

Level

Total Work T(n) = ∑i=0..h f(b·i) = θ(f(n)) or θ(n·f(n))


End

cosc 3101winter, 2004 cosc 3101 design and analysis of algorithms lecture 2. relevant mathematics:...

Documents