1

Chapter 3

3.3 Complexity of Algorithms– Time Complexity– Understanding the complexity of Algorithms

2

Complexity of Algorithm

• Computational Complexity (of the Algorithm)• Time Complexity: Analysis of the time required.• Space Complexity: Analysis of the memory

required.

3

Time Complexity• Example 1: Describe the time complexity of Algorithm

1 of section 3.1 for finding the maximum element in a set (in terms of number of comparisons).

• Algorithm 1: Finding the maximum element in a finite sequence.

procedure max(a1, a2, . . . ,an: integers)

max := a1

for i: =2 to n if max < ai then max := ai

{max is the largest element}

4

• Example 2: Describe the time complexity of the linear search algorithm.

• Algorithm 2 : the linear search algorithm procedure linear search (x: integer, a1, a2, …,an: distinct integers)

i :=1; while ( i ≤n and x ≠ ai)

i := i + 1If i ≤ n then location := iElse location := 0{location is the subscript of the term that equals x , or is 0 if x isnot found}

5

• Example 3: Describe the time complexity of the binary search algorithm in terms of the number of comparisons used .

(and ignoring the time required to compute m= in each iteration of the loop in the algorithm)

• Algorithm 3: the binary search algorithmProcedure binary search (x: integer, a1, a2, …,an: increasing integers)

i :=1 { i is left endpoint of search interval} j :=n { j is right endpoint of search interval}While i < jbegin m := if x > am then i := m+1

else j := mendIf x = ai then location := I

else location :=0{location is the subscript of the term equal to x, or 0 if x is not found}

2/)( ji

2/)( ji

6

• Example 4: Describe the average-case performance of the linear search algorithm, assuming that the element x is in the list.

• Example 5: What is the worst-case complexity of the bubble sort in terms of the number of comparisons made?

• ALGORITHM 4: The Bubble Sortprocedure bubble sort (a1, a2, …,an: real numbers with n ≥2)

for i := 1 to n-1 for j := 1 to n- i if aj > aj+1 then interchange aj and aj+1

{a1, a2, …,an is in increasing order}

7

• Example 6: What is the worst-case complexity of the insertion sort in terms of the number of comparisons made?

• Algorithm 5: The Insertion Sortprocedure insertion sort (a1, a2, …,an: real numbers with n ≥2)

for j := 2 to nbegin i := 1 while aj > ai

i := i + 1 m := aj

for k :=0 to j-i-1 aj-k := a j-k-1

ai := m

end {a1, a2, …,an are sorted}

8

Understanding the complexity of Algorithms

9

• Solvable (in polynomial time, or in exponential time)

• Tractable: A problem that is solvable using an algorithm with polynomial worst-case complexity.

• Intractable: The situation is much worse for problems that cannot be solved using an algorithm with worst-case polynomial time complexity. The problems are called intractable.

• NP problem.• NP-complete problem.• Unsolvable problem: no algorithm to solve them.

10

• Big-O estimate on the time complexity of an algorithm provides an upper, but not a lower, bound on the worst-case time required for the algorithm as a function of the input size.

• Table 2 displays the time needed to solve problems of various sizes with an algorithm using the indicated number of bit operations. Every bit operation takes nanosecond. Times of more than 10100 years are indicated with an asterisk.