Slide 1

Slide 2

Analysis of Recursive Algorithms October 29, 2014 1

Slide 3

Recursive Algorithms Analysis becomes a bit more complicated for recursive algorithms. Recurrence relations is helpful for such analyses. A recursive step can be expressed as a recurrence relation, showing how earlier terms in a sequence are related to later terms. 2

Slide 4

Recurrence Relation A recurrence relation for a sequence a 1, a 2, a 3, is a formula that relates each term a k to its certain predecessors a k-1, a k-2, , a k-i where i is a fixed integer and k is any integer greater than i. The initial conditions for such a recurrence relation specify the values of a 1, a 2, , a i-1. 3

Slide 5

Example For the Tower of Hanoi problem: F 1 = 1 F n = 2F n-1 + 1 (It can be shown with a complex proof that 2 n -1 is the optimal number of steps.) Fibonacci Sequence: F 0 = 0 F 1 = 1 F n = F n-1 + F n-2 4

Slide 6

Class Discussion Can you set up some recurrence relations to compute f(n) where f(n) is the number of 0-1 bit strings with n bits and with no consecutive 1s, i.e., no 11 patterns? Can you set up a recurrence relation to count the number of ways to parenthesize x 1 x 2 x n ? E.g., when n = 3, there are two ways, ((x 1 x 2 ) x 3 ) and (x 1 (x 2 x 3 )). 5 f(n) = f 0 (n)+f 1 (n); f 0 (n) = f(n-1); f 1 (n) = f 0 (n-1); f 0 (1) = f 1 (1) = 1

Slide 7

Repeated Substitution Solving recurrence relation is the key to analyze recursive algorithms. One direct method of solving recurrence relations is called repeated substitution. This is done by repeatedly substitute the formulae for the earlier terms in the recurrence relation. 6

Slide 8

Repeated Substitution For example: a 0 = 1 a k = a k-1 + 2 We start by: a k = a k-1 + 2 = a k-2 + 2 + 2 = a k-3 + 2 + 2 + 2 = a 0 + 2k = 1 + 2k 7

Slide 9

Induction We can also start with a guess of the closed form solution. Then, prove its correctness with induction. 8

Slide 10

Induction For example: a 0 = 1 a k = a k-1 + 2 Suppose we want to show that a k =1+2k: Base case: k = 0, a 0 = 1 = 1 + 2 0 Inductive case: Assume that a k = 1+2k for k 0 a k+1 = a k + 2 = 1 + 2k + 2 = 1 + 2(k+1) Therefore, we can conclude that a k =1+2k (Structural Induction) 9

Slide 11

Class Discussion Give a closed form solution to the following recurrence relation: a 0 = 5 a n = na n-1 10

Slide 12

Recursive Algorithms Solving recurrence relations is the key to analyze recursive algorithms. A recurrence relation can be used to express the running time of a recursive algorithm, showing how many recursive calls are made and how much work is done at each level. We can then solve the recurrence relations to determine an upper bound to the worst case running time. 11

Slide 13

Example Consider the following simple example: 1. factorial(n) 2. if (n 1) 3. return(1) 4. else 5. return(n factorial(n-1) Let T(n) be the running time of factorial(n): T(1) = a T(n) = T(n-1) + b 12

Slide 14

Example Lets try to solve T(n): T(n) = T(n-1) + b = T(n-2) + 2b = T(n-3) + 3b = T(n-(n-1)) + (n-1)b = a + (n-1)b = O(n) 13

Slide 15

More Example Consider the following: 1. SelectionSort(A[ ], i, n) 2. if (i < n) 3. small = i 4. for (j = i + 1, j n, j++) 5. if (A[j] < A[small]) 6. small = j 7. t = A[small] 8. A[small] = A[i] 9. A[i] = t 10. SelectionSort(A, i + 1, n) 14

Slide 16

More Example Let T(m) denote the running time of SelectSort(A,n-m+1,n). T(1) = 0 T(m) = T(m-1) + O(m) T(m-1) + C m T(m-2) + C (m-1) + C m T(1) + C 2 + C 3 + + C m C (m+2)(m-1)/2 = O(m 2 ) Therefore, the run time of SelectSort(A,1,n) is O(n 2 ) 15

Slide 17

Class Discussion Consider MergeSort(): 1. MergeSort(A[ ]) 2. If (length of A[ ] > 1) 3. Split A[ ] into two halves, A 1 [ ] and A 2 [ ] 4. MergeSort(A 1 ) 5. MergeSort(A 2 ) 6. Merge A 1 and A 2 into one sorted list A What is the running time of MergeSort on an array with n elements? 16 T(n) 2T(n/2) + Cn 2(2T(n/4) + C(n/2)) + Cn 2lgnT(1) + lgn Cn = O(nlgn)

Slide 18

Master Theorem Let f be an increasing function that satisfies the recurrence relation: f(n) = af(n/b) + cn d (assume n = b k ), where k is a positive integer, a 1, b is an integer greater than 1, c is a positive real number and d is a non-negative real number, then: 17

Slide 19

Class Discussion Use the Master Theorem to find the Big-O of MergeSort() 18

Slide 20

More Theorems Let f be a function that satisfies the recurrence relation: f(n) = af(n-1) + bf(n-2) for n 2 and f(0) and f(1) have given values. Let r 1 and r 2 be the roots of the equation: x 2 = ax + b Then, if r 1 r 2, f(n) = c 1 r 1 n + c 2 r 2 n where n 0 for some constants c 1 and c 2. If r 1 = r 2 = r, f(n) = (c 1 + c 2 n)r n where n 0 for some constants c 1 and c 2. 19

Slide 21

Class Discussion Give a Big-O of the following recurse(): 1. recurse(n) 2. if (n == 0) 3. return 1 4. for (i = 0; i < 4; i++) 5. total += recurse(n-2) 6. return total 20