Complexity Analysis (Part I)

• Introduction to Algorithms and Complexity Analysis.

• Motivations for Complexity Analysis.

• Big-O Notation: An Introduction.

• Simple Complexity Analysis Examples.

• Average, Best, and Worst Cases.

Introduction to Algorithm & Complexity Analysis

• Let’s consider the execution time of some of the sorting

algorithms.

Introduction (Contd.)

• From a theoretical point of view, the execution time of the

tested algorithms is insufficient.

• So we need to define sound complexity metrics to give us a

complete view about the algorithm performance.

• Computer scientists have defined for such purposes several

• efficiency indices such as:– Machine efficiency. – User efficiency. – Maintenance efficiency.– Algorithm complexity.– Coding efficiency.

Introduction (contd.)

• We will focus on the algorithm complexity measure. The main concern is to highlight the relation between the size of the processed input data and the algorithm complexity.

• The curves shown below indicate the number of operations required to implement the algorithm given the size of the data (n).

Introduction (contd.)

• Let's inspect the algorithms with 2n and 10n complexities, respectively.

Big-O Notation: an Introduction

• With the above concept of algorithm complexity, we can cast algorithms into different complexity classes. Some common classes are shown below:

Complexity ClassBig-O Notation

ConstantO(1)

LogarithmicO(log n)

LinearO(n)

QuadraticO(n^2)

Exponential O(2^n)

Big-O Notation (contd.)

• Question: Why do we need this notation?• Answer: Let’s consider the following code fragment to perform the

sum of the integer elements in an array A. The array A has n elements.

1 public int sum(int[] A){

2 int sm = 0;

3 for(int i=0; i < A.length; i++)

4 sm += A[i] ;

5 return(sm);6}

• The number of operations performed in this method is equal to:

1+1+n+1+n(2+2)+1=5n+4

Big-O Notation (contd.)

• Let’s consider another code fragment to perform the sum of the integer elements in a two-dimensional array B.

1 public int sum_all(int[][] B) }

2 int sm = 0;

3 for(int i=0; i < B.length; i++)

4 for(int j=0; j < B[0].length; j++)

5 sm += B[i][j];

6 return(sm);

7 }

• The number of operations performed in this method is equal to:

1+1+n+1+n(1+n+1+n(2+2)+2)+1=5n^2+5n+4

Big-O Notation (contd.)

• Now, we will consider a case of exponential class complexity.• We will investigate matrix multiplication operation.

Number of columns in matrix A EQUAL to the number of rows in matrix B.

B

3100

3102

3-216

4321

8075

2301

5423

x

A

=2381445

72819

4717956

Big-O Notation (contd.)

1 public int[][] matrixMult(int[][] A, int[][] B) \}

2 int r_a = A.length, c_a = A[0].length;

3 int r_b = B.length, c_b = B[0].length;

4 if(c_a != r_b) \}

5 System.out.pritln("ERROR! Matrix dimensions mismatch.");

6 System.exit(-1);

7 }

8 else \}

9 int[][] C = new int[r_a][c_b];

9 for(int i = 0; i < r_a; i++) \}

10 for(int j = 0; j < c_b; j++) \}

11 C[i][j] = 0;

12 for(int k = 0; k < r_b; k++)

13 C[i][j] += A[i][k] * B[k][j];

14 }

15 }

16 return(C);

17 }

18 } // End of matrixMult()

Average, Worst, and Best Cases

• An algorithm may run faster on certain data sets than others.• Finding the average case can be very difficult, so typically

algorithms are measured in the worst case time complexity.• Also, in certain application domains (e.g., air traffic control, medical,

etc.) knowing the worst case time complexity is of crucial importance.

Average, Worst, and Best Cases (contd.)

• The analysis of the linear search algorithm illustrates the difficulties in defining the algorithm complexity.

2

-1

0

3

12

-4

9

Key ValueCounterCost

211 Op

-122 Op

977 Op

1177 Op & Search {FAILED}

Average, Worst, and Best Cases (contd.)

• From the previous example, the linear search algorithm can have different complexity cases.

• The best case requires 1 comparison operation.• The worst case requires n comparison operations (with possible

failure!).• The average case is the average of all the possible cases over the

number of possibilities, i.e.,

O(n) =

O(n)= ∑ i = 1+2+…+n =

1+2+…+n

n=

∑ ii -1

n

n=

n+1

n

n

i=1

n(n+1)

2

Drill Questions

• Find the number of operations performed by the following code fragment:

sum = 0;

for(i = 1; i <= n; i++)

sum += A[i][i];

• Find the number of operations performed by the following code fragment:

for(cnt1 = 0, i = 1; i <= n; i++)

for(j = 1; j <= n; j++)

cnt1++;

• Find the number of operations performed by the following code fragment:

for(cnt2 = 0, i = 1; i <= n; i *= 2)

for(j = 1; j <= n; j++)

cnt2++;