algorithm analysis examples

Algorithm AnalysisE lExamples

Muhammad Shoaib Farooq

1M.Shoaib Farooq

Complexity of Algorithm

- Each instruction executes within const timetime

e.g. Assignment Statementx=a ....................1x a ....................1x= a+b*c/h-u ......1a>b 1a>b ......................1

M.Shoaib Farooq 2

If StatementComplexity= ConstTime(bool exp) +

max(ThanPart /else Part)max(ThanPart /else Part)

e g if (a>b) than 1e.g. if (a>b) than .........1a=2......................1

elseelsex=x+2*3 .................1

T(n)= 1+1 = 2= O(1) Or O(c)

M.Shoaib Farooq 3

If StatementIf Statemente.g.g

if (a>b) than……..........1a=2.......................1b b+3*t 1b= b+3*t...............1

elsex=x+2*3 ..............1x x 2 3 ..............1

T(n)= 1+2 = 3= O(1)

M.Shoaib Farooq4

LOOP

For n complete iterations- while ............n- for loop ..........n+1- repeat..until ....n

M.Shoaib Farooq

5

An Example: LOOP

i=0 .............................1while i<=n ..................n

i++ n-1i++ ........................n-1a=i ...........................1

T(n)=1+n+n-1+1( )= 2n+1= O(n)= O(n)

M.Shoaib Farooq 6

An Example: LOOPi=1 ...........................1while i<=n nwhile i<=n ................n

i++ ........................1a=i ............................1

T (n)= 1+n+1(n-1)+1 = 2n+1 = o(n)( ) ( ) ( )OR

T(n) = 1+ +1∑n

1T(n) = 1+ +1= 1+ n+1

∑=i 1

M.Shoaib Farooq

= O(n)7

An Example: LOOPpi=1 ...............................1while (i<=n) nwhile (i<=n)....................n

a=2+g........................n-1i i+1 1i=i+1 ............. ..........n-1

if (i<=n)........... .............12 1a=2 ..........................1

elsea=3............................1

T(n) = 1+ n+n-1+n-1+1+1 =3n+1 = O(n)

M.Shoaib Farooq 8

An Example: LOOPi=1..............................1while (i<=10)................10

i i 1 9i=i+1........................9i=1 ..............................1while (i<=n) nwhile (i<=n)..................n

a=2+g......................n-1i=i+1 .......................n-1

if (i<=n)........................1a=2 ..........................1 T ( n) = ?

elsea=3...........................1

M.Shoaib Farooq9

Linear SearchLinearSearch(A, Key)

i 1 1

while (A[i] !=key) and (i<=length[A])i 1

nn-1i 1

if ( i<=length[A]) 1

return trueelse

1

return false 1

M.Shoaib Farooq

T(n) = 1 + n + n-1 + 1 +1 = 2n+2

10

Linear Search- Best-caseLinear Search Best caseBest case occurs when key element occurs at first location of Array then loop executes only once

T(n) = 1 + n + (n-1) + 1 +1T(n) 1 n (n 1) 1 1= 1+1+0+1+1

O(1)= O(1)

M.Shoaib Farooq 11

Linear Search- Worst-caseLinear Search Worst caseWorst case occurs when key element is the last element in array or is not there at all loop executes n timesT(n) = 1 + n + (n-1) + 1 +1

= 2n +2 2n 2= O(n)

M.Shoaib Farooq12

Linear Search- Average-caseLinear Search Average caseKey element is equally likely to occur at any position in the array. The number of comparisons can be any of the numbers 1,2,3…..n and each number occurs with probability p=1/nT(n) = 1.1/n + 2. 1/n +……….+ n.1/n

= (1+2+3+……+n).1/n (1 2 3 …… n).1/n= [n(n+1)/2] 1/n = n+1/2

O( )= O(n) M.Shoaib Farooq 13

Binary Search BinarySearch(A, key)1. mid (low + high)/22. while (A[mid]!=key) and ( low<=high)3. if (key>A[mid])4 low mid +14. low mid +15. else6 high mid - 16. high mid 17. mid (low + high)/28. if (low<=high) T(n)=1+k+(k-1)+(k-1)+(k-1)+1+1

4K 4 l O(l )( g )

9. return true10. else

= 4K= 4 log2n O(log2n)

n/2 + n/4 + n/8 + n/2k

M.Shoaib Farooq

11. return false n/2 + n/4 + n/8 ……..+ n/2k

2K<=n k=log2n14

An Example: Insertion SortAn Example: Insertion SortInsertionSort(A, n)( , )1. for i = 2 to n 2. key = A[i] 3. j = i – 14. while (j > 0) and (A[j] > key)

j 1 j5. A[j+1] = A[j]6. j = j – 17 A[j+1] = key7. A[j+1] = key

15M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort30 10 40 20 i = ∅ j = ∅ key = ∅

A[j] = ∅ A[j+1] = ∅

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = ∅ A[j+1] = ∅

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

16M.Shoaib Farooq

An Example: Insertion Sort30 10 40 20 i = 2 j = 1 key = 10

A[j] = 30 A[j+1] = 10

An Example: Insertion Sort

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 30 A[j+1] = 10

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

17M.Shoaib Farooq

An Example: Insertion Sort30 30 40 20 i = 2 j = 1 key = 10

A[j] = 30 A[j+1] = 30

An Example: Insertion Sort

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 30 A[j+1] = 30

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

18M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort30 30 40 20 i = 2 j = 1 key = 10

A[j] = 30 A[j+1] = 30

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 30 A[j+1] = 30

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

19M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort30 30 40 20 i = 2 j = 0 key = 10

A[j] = ∅ A[j+1] = 30

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = ∅ A[j+1] = 30

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

20M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort30 30 40 20 i = 2 j = 0 key = 10

A[j] = ∅ A[j+1] = 30

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = ∅ A[j+1] = 30

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

21M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 40 20 i = 2 j = 0 key = 10

A[j] = ∅ A[j+1] = 10

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = ∅ A[j+1] = 10

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

22M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 40 20 i = 3 j = 0 key = 10

A[j] = ∅ A[j+1] = 10

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = ∅ A[j+1] = 10

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

23M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 40 20 i = 3 j = 2 key = 40

A[j] = 30 A[j+1] = 40

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 30 A[j+1] = 40

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

24M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 40 20 i = 4 j = 2 key = 40

A[j] = 30 A[j+1] = 40

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 30 A[j+1] = 40

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

25M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 40 20 i = 4 j = 2 key = 20

A[j] = 30 A[j+1] = 40

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 30 A[j+1] = 40

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

26M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 40 20 i = 4 j = 2 key = 20

A[j] = 30 A[j+1] = 40

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 30 A[j+1] = 40

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

27M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 40 20 i = 4 j = 3 key = 20

A[j] = 40 A[j+1] = 20

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 40 A[j+1] = 20

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

28M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 40 20 i = 4 j = 3 key = 20

A[j] = 40 A[j+1] = 20

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 40 A[j+1] = 20

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

29M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 40 40 i = 4 j = 3 key = 20

A[j] = 40 A[j+1] = 40

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 40 A[j+1] = 40

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

30M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 40 40 i = 4 j = 3 key = 20

A[j] = 40 A[j+1] = 40

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 40 A[j+1] = 40

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

31M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 40 40 i = 4 j = 3 key = 20

A[j] = 40 A[j+1] = 40

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 40 A[j+1] = 40

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

32M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 40 40 i = 4 j = 2 key = 20

A[j] = 30 A[j+1] = 40

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 30 A[j+1] = 40

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

33M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 40 40 i = 4 j = 2 key = 20

A[j] = 30 A[j+1] = 40

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 30 A[j+1] = 40

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

34M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 30 40 i = 4 j = 2 key = 20

A[j] = 30 A[j+1] = 30

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 30 A[j+1] = 30

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

35M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 30 40 i = 4 j = 2 key = 20

A[j] = 30 A[j+1] = 30

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 30 A[j+1] = 30

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

36M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 30 30 40 i = 4 j = 1 key = 20

A[j] = 10 A[j+1] = 30

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 10 A[j+1] = 30

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

37M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 20 30 40 i = 4 j = 1 key = 20

A[j] = 10 A[j+1] = 20

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 10 A[j+1] = 20

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}}

38M.Shoaib Farooq

An Example: Insertion SortAn Example: Insertion Sort10 20 30 40 i = 4 j = 1 key = 20

A[j] = 10 A[j+1] = 20

InsertionSort(A, n) {f i 2 t {

1 2 3 4A[j] = 10 A[j+1] = 20

for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}} Done!} Done!

39

M.Shoaib Farooq

Analysis: Insertion SortAnalysis: Insertion Sort

M.Shoaib Farooq40

Worst Case

Analysis: Insertion SortAnalysis: Insertion Sort

tj = 1

n

∑=

−=n

jn

211

T(n)= c1n + c2(n - 1) + c4(n - 1) + c5(n - 1) + c8(n - 1)

= (c1 + c2 + c4 + c5 + c8)n - (c2+ c4 + c5 + c8).

M.Shoaib Farooq41

Best Case

Selection Sort SelectionSort(A)

f i 1 1 dfor i 1 to n-1 dosmall ifor j i+1 to n do

if (A[small]>A[j])if (A[small]>A[j])small=j

A[i] A[ ll]A[i] A[small]

M.Shoaib Farooq 42

Sort (Which one?)Sort (Which one?)

SORT ( A)SORT ( A)for i 1 to n-1 do

f j i 1 t dfor j i+1 to n doif ( A[i]>A[j])

A[i] A[j]

M.Shoaib Farooq 43

Bubble Sort BubbleSort (A)for i 1 to n-1 do

for j 1 to n-1 dofor j 1 to n 1 do if ( A[j] > A[j+1])

A[j] A[j+1]A[j] A[j+1]

M.Shoaib Farooq 44

An Example: LOOPAnalysis (Bottom Up Approach)

T(n)= ?

M.Shoaib Farooq

45

Bottom Up Analysisp y• Bottom while Loop (line 5 and 6)

∑j

while(j) =

• Inner For Loop (line 3 and 4)∑=

+=k

j0

11

for(i) = ∑ ∑= =

+=i

j

i

j

jjwhile2

1

2

1

1)(

i i2 2

= = 2i(2i+1)/2 + 2i∑ ∑= =

+i

j

i

jj

2

1

2

11

= 2i2 + I + 2i 2i2 +3i

M.Shoaib Farooq 46

Bottom Up Analysis

Outer For Loop (Line 1 )T( )

n nT(n) = = ∑=i

ifor1

)( ∑=

+n

i

ii1

2 )32(

∑∑nn

2

== 2 [(2n3+3n2+n)/6] + 3 [n(n+1)/2]

∑∑==

+ii

ii11

2 32 Quadratic Series

= 2 [(2n +3n +n)/6] + 3 [n(n+1)/2]= O (n3)

M.Shoaib Farooq 47