Chapter 9

Efficiency of Algorithms

9.3

Efficiency of Algorithms

Efficiency of Algorithms

• Two aspects of algorithm efficiency are important: – 1. the amount of time required for execution– 2. amount of memory space needed when it runs.

• Algorithms are usually analyzed in terms of their best case, worst case, and average.

• How can the time efficiency of an algorithm be calculated?– Have to factor in the input size to the algorithm– Nature of the input data

Efficiency of Algorithms

• Because the time efficiency can be influenced by many physical parameters of a computing device, i.e. processor speed, memory size, multi-core, etc., a method of analysis must be used that is not a factor of the processing platform.

• Evaluation of algorithms can be based on the number of elementary operations required by the algorithm.– Elementary operations are addition, subtraction, multiplication,

division and comparison.– All elementary ops are viewed as taking 1 time unit on any

system.

Example

• Consider algorithms A & B designed to accomplish the same task. For input size of n– A requires 10n to 20n elementary operations.– B requires 2n2 to 4n2 elementary operations.

• Which algorithm is more efficient?– for n ≤ 10, 2n2 < 20n, and hence, B is better– for n > 10, 20n < 2n2, and in this case A is better

• The answer is dependent on the size of the input. For a small n B is better, but for a larger inputs A wins. It is important to understand the constraints on the order.

Definition

• Let A be an algorithm1. Suppose the number of elementary ops performed when A is

executed for an input of size n depends on n alone and not on the nature of the input data; say it equals f(n). If f(n) is Θ(g(n)) then, A is of order g(n).

2. Suppose the number of elementary operations performed when A is executed for an input of size n depends on the nature of the input data as well as on n.

1. Let b(n) be the minimum number of elementary operations required to execute A for all possible input sets of size n. If b(n) is Θ(g(n)), we say A has a best case order of g(n).

2. Let w(n) be the maximum number of elementary operations required to execute A for all possible input sets of size n. If w(n) is Θ(g(n)), we say A has a worst case order of g(n).

Time Comparisons of Algorithm Orders

Example• Consider the following algorithm segment

p =0, x=2for i = 2 to n

p = (p + i) * xnext I

• Compute the actual number of elementary ops?– 1 multi, 1 add for each iteration. – 2 ops per iteration. – num of iteration = n – 2 + 1 = n-1.– num of elementary ops = 2(n-1)

• Find an order from among the set of power functions– by theorem on polynomial orders, 2n – 2 is Θ(n)– and thus, segment is Θ(n).

Example• Consider a segment with a nested loop (loop inside of a loop)

s=0for i=1 to n for j= 1 to i s=s + j * (i-j+1) next jnext i

• Compute the number of elementary ops.– 2 adds, 1 multi, and 1 minus (4 ops) for each iteration– inside loop (j) iterates i=1 1, i=2 2, i=3 3 … i=n n times– inside loop: 1 + 2 + 3 + … + n = n(n+1)/2 – num of ops = 4 * n(n+1)/2 = 2*n(n+1) = 2n2 + 2n

• Find the order among the set of power functions– 2n(n+1) = 2n2 + 2n is Θ(n2),– hence, segment is Θ(n2)

Example

• Consider the segment where the floor function is used.for i = n/2 to n⎣ ⎦ a = n – Inext I

• Compute the actual number of subtractions performed.– 1 subtraction for each iteration.– loop iterates n - n/2 + 1 times.⎣ ⎦

• n is even: n/2 = n/2⎣ ⎦• n – n/2 + 1 = (2n – n + 2)/2 = (n+2)/2• n is odd: n/2 = (n-1)/2⎣ ⎦• n – (n-1)/2 + 1 = [2n – (n-1) + 2]/2 = (n+3)/2

• Find an order for this segment– (n+2)/2 is Θ(n) and hence, segment is Θ(n)

Sequential Search

• Sequential search occurs when every element in the list is compared to a particular x until either a match is found or the end of the list.

Example

• Find the best and worst case orders for sequential search.– Best case: the best case occurs when the first

element of the list is the item that is being pursued (searched for). The search takes 1 comparison. Θ(1)

– Worst case: the element being searched for is the last element or not in the last (these two situations are equal). The search takes n comparisons. Θ(n)