Lecture 5Jianjun Hu

Department of Computer Science and Engineering

University of South Carolina2009.9.

CSCE350 Algorithms and Data Structure

Outline

How to analyze the Time Efficiency of non-recursive algorithms

How to analyze time efficiency of recursive algorithms

Analyze the Time Efficiency of An AlgorithmNonrecursive Algorithm

Recursive Algorithm f

i*ff

ni

f

nFactorial

return

do to for

ALGORITHM

1

1

)(

n*nFactorial

n

nFactorial

)1(

1

0

)(

return

else

return

if

ALGORITHM

Element Uniqueness

Check whether all the elements in a given array are distinct

Input: An array A[0…n-1]Output: Return “true” if all the elements in A

are distinct and “false” otherwise

true return

false return if

do to for

do to for

ALGORITHM

][][

11

20

])1..0[(

jAiA

n-ij

n-i

nAentsUniqueElem

)()( 2nnCworst 2 1

0 1

1n n

i j i

Selection sort

)()()()( 2nnCnCnC averagebestworst

Insertion Sort

?)(),()(),()( 2 nCnnCnnC averagebestworst

Insert Sorting—running time cases The best case input is an array that is already

sorted. In this case insertion sort has a linear running time (i.e., O(n)). During each iteration, the first remaining element of the input is only compared with the right-most element of the sorted subsection of the array.

The worst case input is an array sorted in reverse order. In this case every iteration of the inner loop will scan and shift the entire sorted subsection of the array before inserting the next element. For this case insertion sort has a quadratic running time (i.e., O(n2)).

The average case is also quadratic, which makes insertion sort impractical for sorting large arrays. However, insertion sort is one of the fastest algorithms for sorting arrays containing fewer than ten elements.

Improvement of Insert sorting

 Shell sort: two simple variants requiring O(n3/2) and O(n4/3) running time.

binary insertion sort  In 2004 Bender, Farach-Colton, and

Mosteiro published a new variant of insertion sort called library sort or gapped insertion sort high probability in O(n log n) time

Example: Find the Number of Binary Digits

Find the Number of Binary Digits in the Binary Representation of a Positive Decimal Integer

count

n/n

count count

n

count

n

n

nBinary

return

do while

tionrepresenta binary s' in //

digits binary of number The :Output //

integer decimal positive A :Input

ALGORITHM

2

1

1

1

//

)(

How to identify inefficiency and speed it up?

What is it doing?Time complexity?How to speed it up?

How to identify inefficiency and speed it up?

2.4.7 Matrix multiplicationFor i0 to n-1 do for j 0 to n-1 do C[i,j]0 for k0 to n-1 do C[i,j] C[i,j]+A[i,k]*B[k,j]

1 1 1

0 0 0

1n n n

i j k

3 3 2n n n

Example Recursive evaluation of n !

Recursive algorithm for n!

Input size: nBasic operation:

multiplication “*”Let M(n) be the

number of multiplications needed to compute n!, then

n*nFactorial

n

nFactorial

)1(

1

0

)(

return

else

return

if

ALGORITHM

01)1()(

0)0(

nnMnM

M

forTo compute Factorial(n-1)

To multiply Factorial(n-1) by n

Solve the Recurrence

n

nnnM

nM

nM

nM

nnMnM

M

)(

...

3)3(

2)2(

)(

01)1()(

0)0(

for

Therefore, the number of multiplications needed to compute n! in this algorithm is n.

The complexity of this algorithm is

)(n

Time Efficiency of Recursive Algorithms

Steps in mathematical analysis of recursive algorithms:Decide on parameter n indicating input sizeIdentify algorithm’s basic operationDetermine worst, average, and best case for input of

size nSet up a recurrence relation and initial condition(s) for

C(n)-the number of times the basic operation will be executed for an input of size n (alternatively count recursive calls).

Solve the recurrence to obtain a closed form or estimate the order of magnitude of the solution (see Appendix B)

Another Example: Tower of Hanoin different-size disks, 3 pegs, move disks

from the left peg to the right one using the middle one as an auxiliary

Rules: you can move one disk each time and it is forbidden to place a larger disk on top of a smaller one

Design and algorithm and analyze its complexity

Recursive AlgorithmInput size: n (disks)Basic operation: one move of a diskInitial condition: n=1 only one direct moveTo build the recurrence: suppose you have a

way to move n-1 disks. Then you can move the top n-1 disks from the left

peg to the middle peg using the right peg as an auxiliary.

Move the bottom disk from the left peg to the right peg.

Move n-1 disks from the middle peg to the right peg using the left one as an auxiliary.

Illustration

1

2

3

Algorithm ComplexityLet M(n) be the number of needed moves Initialization M(1)=1Recurrence

1)1(1)1()( nnMnMnM for

121222)1(2

...

122)3(212]1)3(2[2

12)2(21]1)2(2[2

11)1(2)(

112

0

1

232

2

nnnn

i

in M

nMnM

nMnM

nnMnM for

Important Recurrence Types: One (constant) operation reduces problem size by one.

T(n) = T(n-1) + c T(1) = dSolution: T(n) = (n-1)c + d linear

A pass through input reduces problem size by one.T(n) = T(n-1) + cn T(1) = dSolution: T(n) = [n(n+1)/2 – 1] c + d quadratic

One (constant) operation reduces problem size by half. T(n) = T(n/2) + c T(1) = dSolution: T(n) = c lg n + d logarithmic

A pass through input reduces problem size by half.T(n) = 2T(n/2) + cn T(1) = dSolution: T(n) = cn lg n + d n n log n

A General Divide-and-Conquer Recurrence: Master Theorem

T(n) = aT(n/b) + f (n) where f (n) ∈ Θ(nk)

a < bk T(n) ∈ Θ(nk)

a = bk T(n) ∈ Θ(nk lg n ) a > bk T(n) ∈ Θ(nlog b a)

Note: the same results hold with O instead of Θ.

Example: Find the Number of Binary Digits (Recursive Algorithm)Find the Number of Binary Digits in the

Binary Representation of a Positive Decimal Integer using a recursive algorithm

Recurrence 1)2(

11

//

)(

n/BinRec

n

n

nBinRec

return else

return if

tionrepresenta binary s' in //

digits binary of number The :Output //

n integer decimal positive A :Input

ALGORITHM

0)1(

1,1)2n/()(

A

nAnA for )(loglog)( 2 nnnA