Solving Recurrence

Lecture 19: Nov 25

Some Recursive Programming (Optional?)

Test

int hello(int n){

if (n==0)return 0;

elseprintf(“Hello World %d\n”,n);hello(n-1);

}

1. What would the program do if I call hello(10)?

2. What if I call hello(-1)?

3. What if the order of printf() and hello() is reversed?

Computing Sum of Arithmetic Progression

int AP(int n){

if (n==0)return 0;

elsereturn (n+AP(n-1));

}

Many programs can be written in a recursive way.

The way of thinking is different.

Always try to reduce it to smaller problems.

Computing Exponential Function

int EX(int n){

if (n==0)return 1;

elsereturn (EX(n-1)+EX(n-1));

}

How many function calls if I run EX(n)?

This function is to compute 2n.

2n times.

If we replace the last line by return 2EX(n-1),

then the program will compute the same thing,

but there will be only n function calls.

Post #1 Post #2 Post #3

Move1,2(n)::= Move1,3(n-1);

biggest disk 12;

Move3,2(n-1)

Tower of Hanoi

To move the biggest disk,we must first move the disks on top to another post.

Tower of Hanoi

T11(a,b)

{

T10(a,c);

printf(“Move Disk 11 from a to b\n”);

T10(c,b);

}

Suppose we already have a function T10(a,b) to move 10 disks from pole a to pole b.

It is then easy to write a program for T11(a,b)

In general you can write exactly the same program for Tn,

if we are given a program for Tn-1.

Instead of writing one program for each n,

we can take n as an input and write a recursive program.

Tower of Hanoi

Tower_of_Hanoi(int origin, int destination, int buffer, int number)

{

if (n==0)

return;

Tower_of_Hanoi(origin, buffer, destination, number-1);

printf(“Move Disk #%d from %d to %d\n”, number, origin, destination);

Tower_of_Hanoi(buffer, destination, origin, number-1);

}

This is the power of recursive thinking.

The program is very short,

yet it is not easy to write it without recursion

Tower of Hanoi

T(A,C,B,3)

Tower_of_Hanoi(origin, destination, buffer, number)

Move 3 from A to C.

T(A,B,C,2)

T(B,C,A,2)

move 2 from A to B

move 2 from B to C

T(A,C,B,1)

T(C,B,A,1)

T(B,A,C,1)

T(A,C,B,1)

move 1 from A to C

move 1 from C to B

move 1 from B to A

move 1 from A to C

1

2

3

4

5

6

7

Generate Strings

Can you write a program to generate all n-bit strings with k ones?

There are many ways to do it.

(1)Generate all n-bit strings and output those strings with k ones.

- Too slow

(2) Write k for-loops.

- That’s okay if k is known, but what if k is part of the input?

(3) Write a program to write a program with k for loops.

- That’s okay, but can we do it more elegantly?

(4) Write a recursive program for it.

Generate Strings

The idea of the recursive program is simple.

First we generate all strings which begin with 1,

and then generate all strings which begin with 0.

Can you write a program to generate all n-bit strings with k ones?

For strings which begin with 1, we still have n-1 bits and k-1 ones left.

For strings which begin with 0, we still have n-1 bits and k ones left.

Let’s say the program is called gen(n,k).

gen(n-1,k-1)

gen(n-1,k)Sounds familiar?

Generate Strings

Can you write a program to generate all N-bit strings with K ones?

int solution[N]; (an array holding an N-bit string, from solution[0] to solution[n-1])

gen(int n, int k) (n is # bits left, and k is # ones left){

if (n==0) (no more bits left)print solution; (write a for loop to print out the N-bits in solution)return;

if (k > 0)solution[N-n] = 1; (generate the strings beginning with one)gen(n-1,k-1);

if (n > k) (do it only if there are enough places for the ones)solution[N-n]=0; (generate the strings beginning with zero)gen(n-1,k);

}

Programming Exercises

Can you write a recursive program to generate all permutations of n elements?

Can you write a recursive program for merge-sort?

Solving Recurrence

Warm Up

a0=1, ak = ak-1 + 2

a1 = a0 + 2

a2 = a1 + 2 = (a0 + 2) + 2 = a0 + 4

a3 = a2 + 2 = (a0 + 4) + 2 = a0 + 6

a4 = a3 + 2 = (a0 + 6) + 2 = a0 + 8

See the pattern is ak = a0 + 2k = 2k+1

You can verify by induction.

Solving Hanoi Sequence

a1=1, ak = 2ak-1 + 1

a2 = 2a1 + 1 = 3

a3 = 2a2 + 1 = 2(2a1 + 1) + 1 = 4a1 + 3 = 7

a4 = 2a3 + 1 = 2(4a1 + 3) + 1 = 8a1 + 7 = 15

a5 = 2a4 + 1 = 2(8a1 + 7) + 1 = 16a1 + 15 = 31

a6 = 2a5 + 1 = 2(16a1 + 15) + 1 = 32a1 + 31 = 63

Guess the pattern is ak = 2k-1

You can verify by induction.

Solving Merge Sort Recurrence

T2k = 2Tk + 2k

If we could guess that Tk is k log2k,

then we can prove that T2k is 2k log2(2k).

This is because T2k = 2Tk + 2k

= 2klog2k + 2k

= 2k(log2k + 1)

= 2k(log2k + log22)

= 2klog22k

Solving Merge Sort Recurrence

T2k = 2Tk + 2k

How could we guess Tk = k log2k in the first place?

If there are n numbers there are log2n levels.

In each level i we need to solve 2i-1 merge problems.

Each merge problem in level i has two subseqences

of n/2i numbers, and so can be solved in n/2i-1

steps.

So each level requires a total of (2i-1)(n/2i-1)=n steps.

Since there are log2n levels,

the total number of steps is at most nlog2n.

Level 3

Level 1

Level 2

Solving Fibonacci Sequence

a0=0, a1=1, ak = ak-1 + ak-2

a2 = a1 + a0 = 1

a3 = a2 + a1 = 2a1 + a0 = 2

a4 = a3 + a2 = 2a2 + a1 = 3a1 + 2a0 = 3

a5 = a4 + a3 = 2a3 + a2 = 3a2 + 2a1 = 5a1 + 3a0 = 5

a6 = a5 + a4 = 2a4 + a3 = 3a3 + 2a2 = 5a2 + 3a1 = 8a1 + 5a0 = 8

a7 = a6 + a5 = 2a5 + a4 = 3a4 + 2a3 = 5a3 + 3a2 = 8a2 + 5a1 = 13a1 + 8a0 = 13

See the pattern an = an-kak+1 + an-k-1ak

but this does not give a formula for computing an

Second Order Recurrence Relation

In the textbook it is called “second-order linear homogeneous recurrence

relation with constant coefficients”.

ak = Aak-1 + Bak-2

A and B are real numbers and B≠0

For example, Fibonacci sequence is when A=B=1.

To solve this equation, first we will find some solutions,

and later show that they “generate” all possible solutions.

Geometric-Sequence Solution

ak = Aak-1 + Bak-2

Find solutions of the form (1, t, t2, t3, t4, …, tn, …)

tk = Atk-1 + Btk-2

That is, suppose ak=tk

t2 = At + B

t2 - At – B = 0

So t is a root of the quadratic equation t2 - At – B = 0.

Example

ak = ak-1 + 2ak-2

Find solutions of the form (1, t, t2, t3, t4, …, tn, …)

So t must be a root of the quadratic equation t2 - t – 2 = 0.

This implies that t=2 or t=-1.

So solutions of the form (1, t, t2, t3, t4, …, tn, …) are:

(i) (1,2,4,8,16,32,64,…)

(ii) (1,-1,1,-1,1,-1,…)

Example

ak = ak-1 + 2ak-2

So solutions of the form (1, t, t2, t3, t4, …, tn, …) are:

(i) (1,2,4,8,16,32,64,…)

(ii) (1,-1,1,-1,1,-1,1,…)

Are there other solutions?

Try (2,1,5,7,17,31,65,…)

(0,3,3,9,15,33,63,…)

(4,5,13,23,49,95,193,…)

How to obtain these solutions?

Linear Combination of Two Solutions

If (r0,r1,r2,r3,…) and (s0,s1,s2,s3,…) are solutions to ak = Aak-1 + Bak-2,

then the sequence (a0,a1,a2,a3,…) defined by the formula

ak = Crk + Dsk

also satisfies the same recurrence relation for any C and D.

This says that any linear combination of two solutions for

the recurrence relation is also a solution for the recurrence.

(This is easy to check.)

Linear Combination of Two Solutions

Are there other solutions than ak = Crk + Dsk?

Any solution to ak = Aak-1 + Bak-2 is determined by the first two terms a0 and a1

Suppose we already have two solutions (r0,r1,r2,r3,…) and (s0,s1,s2,s3,…).

If (r0,r1) and (s0,s1) are linearly independent,

then any possible (a0,a1) is a linear combination of (r0,r1) and (s0,s1),

and thus any solution to the recurrence relation ak = Aak-1 + Bak-2

is a linear combination of (r0,r1,r2,r3,…) and (s0,s1,s2,s3,…).

Distinct-Roots Theorem

Suppose a sequence (a0,a1,a2,a3,…) satisfies a recurrence relation

ak = Aak-1 + Bak-2

If t2 - At – B = 0 has two distinct roots r and s,

then an = Crn + Dsn for some C and D.

If we are given a0 and a1, then C and D are uniquely determined.

The theorem says that all the solutions of the recurrence relation

are a linear combination of the two series (1,r,r2,r3,r4,…,rn,…)

and (1,s,s2,s3,s4,…,sn,…) defined by the distinct roots of t2 - At – B = 0,

because the two series are linearly independent.

Solving Fibonacci Sequence

a0=0, a1=1, ak = ak-1 + ak-2

First solve the quadratic equation t2 - t – 1 = 0.

So the distinct roots are:

Solving Fibonacci Sequence

a0=0, a1=1, ak = ak-1 + ak-2

By the distinct-roots theorem, the solutions satisfy the formula:

To figure out C and D, we substitute the value of a0 and a1:

Solving these two equations, we get:

Therefore:

Solving Fibonacci Sequence

Quick Summary

Recursion is a very useful and powerful technique in computer science.

It is very important to learn to think recursively,

by reducing the problem into smaller problems.

This is a necessary skill to acquire to become a professional programmer.

Make sure you have more practices in setting up recurrence relations.