CSC 2300Data Structures & Algorithms

January 30, 2007

Chapter 2. Algorithm Analysis

Today

Binary Search Euclid’s Algorithm Efficient Exponentiation Recursion and recurrences

Binary Search

Given an integer X and integers A0, A1, …, AN-1, which are presorted and already in memory, find i such that Ai = X, or return i=-1 if X is not in the input.

What is an obvious solution? Scan through the list from left to right. Time? O(N). Better strategy? Check if X is the middle element. If yes, we get the answer. If X is

smaller, we apply the same strategy to the sorted array to the left of the middle element. If X is larger, we look to the right half.

Binary Search

What is the running time? O(logN).

Analyzing Binary Search

Example

We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=9.

low high #pts mid a[mid] return

0 15 16 7 15

0 6 7 3 7

4 6 3 5 11

4 4 1 4 9 4

Example

We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=8.

low high #pts mid a[mid] return

0 15 16 7 15

0 6 7 3 7

4 6 3 5 11

4 4 1 4 9

4 3 0 -1

Example

We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=1.

low high #pts mid a[mid] return

0 15 16 7 15

0 6 7 3 7

0 2 3 1 3

0 0 1 0 1 0

Example

We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=31.

low high #pts mid a[mid] return

0 15 16 7 15

8 15 8 11 23

12 15 4 13 27

14 15 2 14 29

15 15 1 15 31 15

Example

We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=33.

low high #pts mid a[mid] return

0 15 16 7 15

8 15 8 11 23

12 15 4 13 27

14 15 2 14 29

15 15 1 15 31 15

16 15 0 -1

Euclid’s Algorithm

This algorithm computes gcd(m,n), assuming m≥n.

The algorithm works by computing remainders until 0 is reached.

The last nonzero remainder is the answer.

Example, M=1989 and N=1590. The sequence of remainders is

399, 393, 6, 3, 0. Therefore, gcd(1989,1590)=3.

Running Time Analysis

The sequence of remainders is 399, 393, 6, 3, 0. The remainder does not decrease by a constant factor in

one iteration. What to do? We can prove that after two iterations, the remainder is

at most half of its original value. This will show that the number of iterations is at most

2 logN = O(logN). Theorem 2.1. If M>N, then M mod N < M/2. Worst case is 2 logN? No, 1.44 logN, when M and N are consecutive Fibonacci

numbers.

Efficient Exponentiation

The obvious way to compute XN uses N-1 multiplications.

A recursive algorithm can do better. What are the base cases? N=0 and N=1. Otherwise, if N is even, we compute

XN = XN/2 XN/2 ,and if N is odd,

XN = XN/2 XN/2 X. How many multiplications are needed? At most 2 logN. Why 2? If N is odd.

Example

Compute X62. The algorithm performs these calculations:

X3 = (X2)X,X7 = (X3)2X,X15 = (X7)2X,X31 = (X15)2X,X62 = (X31)2,

How many multiplications? Nine. In general, at most 2 logN.

Number of Multiplications

Does it always require more work to compute Xi than Xj if i > j? No. Consider X64 (versus X62). The algorithm performs these calculations:

X2 = (X)2,X4 = (X2)2,X8 = (X4)2,X16 = (X8)2,X32 = (X16)2,X64 = (X32)2,

How many multiplications? Six (versus nine for X62).

Precise Number of Multiplies

Can we find the precise number of multiplies used by the algorithm?

Consider X65. The algorithm performs these calculations:

X2 = (X)2,X4 = (X2)2,X8 = (X4)2,X16 = (X8)2,X32 = (X16)2,X65 = (X32)2 X,

How many multiplications? Seven (versus six for X64).

Another Example

Consider X63. The algorithm performs these calculations:

X3 = (X)2 X,X7 = (X3)2 X,X15 = (X7)2 X,X31 = (X15)2 X,X63 = (X31)2 X,

How many multiplications? Ten (versus six for X64). What does the number depend on?

Precise Number of Multiplies

Compute XN. What is the number of multiplies when N=0 or N=1? Zero.

What is the important feature on the value of N? Its binary representation. Let b(N) represent the number of ones in the binary representation

of N. What is the formula for #mult? [ log2N ] + b(N) - 1

N 0 1 62 63 64 65

#mult 0 0 9 10 6 7

Fast Exponentiation

Is algorithm optimal? No. Compute X62. Fast exponentiation requires nine multiplications. Can you calculate X62 using only 8 multiplies? How? Compute X2, X4, X8, …, X62. You are asked to solve this problem in Homework 2.

Recursion and Recurrences

Methods for Recurrences

Two straightforward methods for solving recurrences are

1. Repeated expansion accompanied by summation

2. Forming telescopic sums

Method 1

Repeated expansion Recurrence: T(n) = T(n-1) + 1

Method 2

Telescoping sum Recurrence: T(n) = T(n-1) + 1

Telescoping Sum

Recall Algorithm 3 to find Maximum Subsequence Sum. Recurrence: T(1) = 1, T(N) = 2 T(N/2) + N. Divide recurrence by N: T(N)/N = T(N/2)/(N/2) + 1. The recurrence is valid for any N that is a power of 2:

T(N)/N = T(N/2)/(N/2) + 1T(N/2)/(N/2) = T(N/4)/(N/4) + 1T(N/4)/(N/4) = T(N/8)/(N/8) + 1

…T(4)/4 = T(2)/2 + 1T(2)/2 = T(1)/1 + 1

Telescoping the sum, we getT(N)/N = T(1)/(1) + logN.

So, T(N) = N logN + N = O(N logN).