lecture 2 maths for computer science recurrences

Lecture 2 – Maths for Computer Science Recurrences

Lecture 2 – Maths for Computer ScienceRecurrences

Denis TRYSTRAMLecture notes MoSIG1

sept. 16, 2020

1 / 1


Recall of the principle

Objective

The objective of this lecture is to present induction, which is thebasic principle of recurrence and to show its application on severalexamples.

This lecture is devoted to deriving and solving a variety of types ofrecurrences (linear recurrence, multiple steps recurrences, etc.).

We will show how to use various ways for solving problems byinduction.

2 / 1



Brief overview of this sequence

Recall of the induction principle.

A first example (direct application)

Take care of the first steps!

Sum of cubes

Master Theorem

Linear and bilinear recurrences:Hanoi’s towers and The token game

3 / 1



An example of induction: Factorial

The classical first example of the recurrent mode of computinginvolves the factorial function Fact (of a nonnegative integer).The “direct” mode of computing Fact at an argument n is:

Fact(n) = 1× 2× · · · × n.

The recurrent mode of computing Fact(n) is more compact—andit better exposes the inherent structure of the function.

Fact(n) =

n × Fact(n − 1) if n > 1

1 if n = 1

4 / 1



Proof by recurrence

Goal: proving that a statement P(n) involving integer n is trueusing the induction principle.

Basis. Solve the statement for the small values of n.

Induction step. Prove the statement for n assuming it iscorrect for k ≤ n − 1.

5 / 1



ExampleProposition. ∀n, the nth perfect square is the sum of the first nodd integers.

n2 = 1 + 3 + 5 + · · ·+ (2n − 3) + (2n − 1)

Proof. For every positive integer m, let P(m) denote the assertion

m2 = 1 + 3 + 5 + · · ·+ (2m − 1)

Let us proceed by the standard format of an inductive argument:

Basis. Because 12 = 1, proposition P(1) is true.

Induction step. Let us assume, for the sake of induction,that assertion P(m) is true for all positive integers strictlysmaller than n.

6 / 1



Consider now the summation

1 + 3 + 5 + · · ·+ (2n − 3) + (2n − 1)

Because P(n − 1) is true, we know that

1 + 3 + · · ·+ (2n − 1) =(1 + 3 + · · ·+ (2n − 3)

)+ (2n − 1)

=(1 + 3 + · · ·+ (2(n − 1)− 1)

)+ (2n − 1)

= (n − 1)2 + (2n − 1)

By –easy– direct calculation, we now find that

(n − 1)2 + (2n − 1) = (n2 − 2n + 1) + (2n − 1) = n2

The Principle of (Finite) Induction tells us that P(n) is true for allinteger n.

7 / 1



Starting an induction

Let us (mis)use the method of Finite Induction to craft a fallaciousproof of the following absurd “fact”.

Proposition. All horses are the same color.

The base case. If there is only a single horse in the set, then P istrue (all horses in the set are the same color).

Inductive hypothesis. in every set of n horses, all horses in theset are the same color.

8 / 1



Extension to n + 1 horses.

Let us be given a set of n + 1 horses.Remove one horse from the set, then the remaining set, call it Shas n horses.By our inductive hypothesis, all of the horses in S have the samecolor.Now remove one horse from S and replace it with the horse thatwas removed from the (n + 1)-horse set. We now have a newn-horses set S ′.Once again we invoke the inductive hypothesis to conclude that allhorses in S ′ have the same color. If we now reunite all of thehorses, the transitivity of the relation “have the same color”guarantees that all of the horses in the (n + 1)-horses set have thesame color.

9 / 1



What’s wrong?

We all know that all horses do not share the same color.

The base case was not adequate for the “proof”

When we remove first one horse from the set of n + 1 horses andthen another horse from that set, and we still have a horse left tocompare those two horses to, we must have started with at leastthree horses!This means that n + 1 must be no smaller than 3, so n must be nosmaller than 2. The base of the induction must, therefore, be setsthat contain 2 horses—and the same-color “proposition” is, ofcourse, absurd for such sets!

This brings us to the critical issue of how to select the “small”cases that comprise the base of our induction.

10 / 1



What’s wrong?

We all know that all horses do not share the same color.The base case was not adequate for the “proof”

When we remove first one horse from the set of n + 1 horses andthen another horse from that set, and we still have a horse left tocompare those two horses to, we must have started with at leastthree horses!This means that n + 1 must be no smaller than 3, so n must be nosmaller than 2. The base of the induction must, therefore, be setsthat contain 2 horses—and the same-color “proposition” is, ofcourse, absurd for such sets!

This brings us to the critical issue of how to select the “small”cases that comprise the base of our induction.

10 / 1


Sum of cubes

Sum of n first cubes

Cn =∑n

k=1 k3

A straightforward upper bound is n4 since each of the n terms ofthe sum k3 is less than n3.This bound may be refined using the analogy of integrals andRiemann’s sum, which leads to a value of order n4

4 .

Let us determine precisely this expression.

11 / 1


Sum of cubes

The first ranks give us an idea:C1 = 1C2 = 1 + 8 = 9 = 32

C3 = 1 + 8 + 27 = 36 = 62

C4 = 1 + 8 + 27 + 64 = 100 = 102, ...

All these values are perfect squares.A more attentive observation evidences a link with the triangularnumbers (1,3,6,10, ...):Cn = ∆2

n

This is a guess, not a proof!We can do a simple recurrence...

12 / 1


Sum of cubes


C3 = 1 + 8 + 27 = 36 = 62

C4 = 1 + 8 + 27 + 64 = 100 = 102, ...

All these values are perfect squares.

A more attentive observation evidences a link with the triangularnumbers (1,3,6,10, ...):Cn = ∆2

n


12 / 1


Sum of cubes


C3 = 1 + 8 + 27 = 36 = 62

C4 = 1 + 8 + 27 + 64 = 100 = 102, ...

All these values are perfect squares.A more attentive observation evidences a link with the triangularnumbers (1,3,6,10, ...):Cn = ∆2

n


12 / 1


Sum of cubes

Link with summing the odd numbers

We proved that summing the odd numbers is equal to a perfectsquare.

The figure below shows an organization of the odd numbers (left)where each row corresponds to a cube of the number of elementsin the row (right).

1"3" 5"

7" 9" 11"13" 15" 17" 19"

1"3" 5"

7" 9" 11"13" 15" 17" 19"

13"

23"

33"

43"

There are k numbers in row k.

13 / 1


Sum of cubes

A little more effort...

The summation over each column, which makes the cubes is notobvious.

1"3" 5"

7" 9" 11"13" 15" 17" 19"

13"

23"

33"

43"

Row k contains the sum of odds from ∆k−1 + 1 to ∆k .

= (k(k+1)2 )2 − (k(k−1)2 )2 = k2

4 ((k + 1)2 − (k − 1)2) = k2

4 (4k) = k3

14 / 1


Sum of cubes

A structural evidence?

Let us look at row k more carefully.

13 5

7 9 1113 15 17 19

31 33 35 37 39 4121 23 25 27 29

15 / 1


Sum of cubes

13 5

7 32 1113 15 17 19

31 33 35 37 39 4121 23 52 27 29

22

42

62

16 / 1


Sum of cubes

Using the double counting principle

An alternative proof is to determine this result directly by usingmultiplicative tables (each row/column is multiplied by the nextinteger).

12345

246810

3691215

48121620

510152025

x5x4x3x2

+

17 / 1


Sum of cubes

Using classical multiplication tables

The sum of all the elements in column k of this matrix is equal tok times the triangular number ∆n. Thus, the global sum is:1.∆n + 2.∆n + ...+ n.∆n = (1 + 2 + ...+ n).∆n = ∆2

n.

18 / 1


Sum of cubes

Using Fubini double counting principle, it is easy to remark thatthe same global sum can also be obtained by summing the nquadrants:

12.45

24.810

.

.

.1215

48121620

510152025

2+4+2=233+6+9+6+3=33

+

Each of these partial sums at rank k is k times theascending/descending triangular numbers:k .(1 + 2 + ...+ (k−1) +k + (k−1) + ...+ 2 + 1) = k .(∆k + ∆k−1).From the previous result ∆k + ∆k−1 = k2, the sum of theelements in quadrant k is k3.Thus, the global sum is equal to

∑nk=1 k

3.19 / 1


Sum of cubes

Alternative geometrical proof

Another way of determining the sum of cubes by means oftriangular numbers.The idea is that the cube of k is written as k times the squares kby k .

Summation of the first two cubes:

20 / 1


Sum of cubes

Rearranging 23

21 / 1


Sum of cubes

Adding the term 33

3

2

1

22 / 1


Sum of cubes

General iteration of the summation of cubes

Δnn+1

(n+1)/2

(n+1)/2

23 / 1


Sum of cubes

General iteration of the summation of cubes

We let the reader the task to complete the formal proof.

It corresponds to translate the previous picture into the rightwords...

24 / 1


Sum of cubes

Beyond the standard proofs

Let us study two families of recurrences.

25 / 1


Master Theorem

Master Theorem

The cost analysis of the divide-and-conquer paradigm leads tothe following recurrence equation. a ≥ 1 et b > 1.

f (n) = Θ(1) if n ≤ n0

f (n) = a.f (nb ) + c(n) if n > n0

We give below the general formulation for solving this equation:

1 if c(n) ∈ O(nlogba−ε) then f (n) ∈ Θ(nlogba)

2 if c(n) ∈ Θ(nlogba) then f (n) ∈ Θ(nlogbalog(n))

3 if c(n) ∈ Ω(nlogba+ε) and if a.c(n/b) ≤ kf (n) for someconstant k < 1 then, f (n) ∈ Θ(c(n))

where ε is a positive real number.

26 / 1


Master Theorem

Analysis of two simplified cases

We assume that n is a power of b (in other words, it can beperfectly divided by b until reaching the value 1).

f (1) = 1

f (n) = a.f (nb ) + c (c is a positive constant)

f (1) = 1

f (n) = a.f (nb ) + n

27 / 1


Master Theorem

Analysis of the simplest case

Let the function f be specified by the simple linear recurrence.Then the value of f on any argument n is given by

f (n) = (1 + logb n) · c if a = 1

=1− alogb n

1− a· c ≈ c

1− aif a < 1

=alogb n − 1

a− 1· c if a > 1

(1)

28 / 1


Master Theorem

proof

We start by expanding the specified computation, replacingoccurrences of f (·).Once we discern the pattern, we jump to the general form.

f (n) = af (n/b) + c= a

(af (n/b2) + c

)+ c

= a2f (n/b2) + (a + 1)c= a2

(af (n/b3) + c

)+ (a + 1)c

= a3f (n/b3) + (a2 + a + 1)c...

...

=(alogb n + · · ·+ a2 + a + 1

)c

(2)

29 / 1


Master Theorem

Pictorial view for a = b = 2

30 / 1


Master Theorem

Extension: linear cost instead of constant

We simplify the problem in two ways, in order to avoidcalculational complications (such as floors and ceilings) that canmask the principles that govern our analysis.

1 We employ a very simple function g : We focus on the caseg(n) = n.We assume that the argument n to functions f and g is apower of b1.

2 We consider c = 1.

Removing these assumptions would significantly complicate ourcalculations, but it would not change the reasoning!

1This allows us to concentrate on the general unfolding of the recurrencewithout worrying about floors and ceilings

31 / 1


Master Theorem

proofLet the function f be specified by the general linear recurrence.Then, the value of f on any argument n is given by

f (n) = alogb nf (1) +

logb(n)−1∑i=0

(a/b)i

n

When a > b, the behavior of f (n) is dominated by the first term:

alogb n · f (1) = nlogb a

When a < b, the behavior of f (n) is dominated by the second term:

n ·logb(n)−1∑

i=0

(a/b)i =

(1 − (a/b)logb(n)

)1− (a/b)

· n ≈ b

b − a· n

32 / 1


Master Theorem

proof

We expose the algebraic pattern created by the recurrence“unfolding”. As before, once we discern this pattern, we jump tothe general form (which can be verified via induction).

f (n) = af (n/b) + n= a

(af (n/b2) + n/b

)+ n

= a2f (n/b2) + (an/b + n)= a2

(af (n/b3) + n/b2

)+ (a/b + 1)n

= a3f (n/b3) + (a2/b2 + a/b + 1)n...

...

= alogb nf (1) +

logb(n)−1∑i=0

(a/b)i

n

33 / 1


Master Theorem

proof

We thus see that solving the more general recurrence requires onlyaugmenting the solution to the simple recurrence by “appending”to the simple solution a geometric summation whose base is theratio a/b.

34 / 1


Master Theorem

Pictorial view for any a and bf (n) = a.f (nb ) + c(n)Solve this problem for a (positive) multiplicative function c definedon the successive powers of b.

35 / 1


Master Theorem

Interpretation of the master Theorem

Intuitively, the theorem tells us that f (n) is determined by thedominance of one of the functions:granularity of the partitioning and merge process.

if nlogba dominates, then, the solution is in Θ(nlogba), if this is thecontrary, the solution is in Θ(f (n)).if it is well-balanced, the solution is in Θ(f (n).log(n))

36 / 1


Linear and bilinear recurrences

Linear and Bilinear recurrences

37 / 1



Example Hanoi’s Towers

A set of n disks of decreasing diameters initially stacked on one ofthree pegs.Problem definition. The goal is to transfer the entire tower froma given peg to another fixed one, moving only one disk at a timeand never moving a larger disk on top of a smaller one.

D I A

The main question is to determine the best way to realize thisoperation (which means in a minimum number of moves).

38 / 1



Lower Bound

Let call Hn the number of moves required to solve the puzzle withn disks. What is the minimum number of moves?When there is only one disk, there is only one move: H1 = 1, withtwo disks, at least 3 moves are necessary: H2 = 3.

Let us prove Hn ≥ 2Hn−1 + 1. Indeed, looking at the largest disk,the n − 1 others must be on a single peg, which required Hn−1 toput them here. Then, the largest disk should be moved, and againmove the n − 1 others on the target peg.Thus, we have the following recurrence to solve: Hn = 2Hn−1 + 1(n ≥ 2) with H1 = 1.

There exists a closed formula: The first ranks give us an insight ofthe solution (1, 3, 7, 15, 31, ...).We guess Hn = 2n − 1 for n ≥ 1.

39 / 1



Compute the cost

Basis is straightforward since H1 = 21 − 1 = 1.

Induction stepHn = 2Hn−1 + 1 where Hn−1 = 2n−1 − 1,thus, Hn = 2(2n−1 − 1) + 1 = 2n − 1 and we are done.

Notice that this expression can also be obtained directly as thesum of a geometric series:

Hn = 2Hn−1 + 1= 2(2Hn−2 + 1) + 1= . . .

=∑n−1

j=0 2j = 2n − 1

40 / 1



The classical (recursive) solutionThe natural method is recursive. It consists in moving the n − 1top disks on the intermediate peg, then, put the largest one on thetarget peg, and moving again the n − 1 disks on top of it.

41 / 1



formal algorithm

input: an integer n, three pegs indexed as D (departure), A(arrival) and I (intermediate). All the disks are stacked on peg D.output: The disks are stacked on peg A.

If (n 6= 0) then

Hanoi(n-1,D,I,A)

move disk from D to A

Hanoi(n-1,I,A,D)

Analysis. It is easy to compute the number of moves, using thesame recurrence equation as for the lower bound:Hn = 2n − 1.

42 / 1



Bilinear equations

43 / 1



Another case study: The Token GameConsider a bank with n circle positions numbered from 1 to n andn tokens. Initially, the bank is empty.

The game consists in determining the process to fill the bank withthe n tokens, putting or removing one token at a time according toone of the two following constraints.

Rule 1. Position 1: Put a token if it is empty or remove it.

Rule 2. Position next to the first empty position (i.e. on theright): Put a token if the position is empty or remove it.

44 / 1



Figure: Rule 1: Position 1 contains a token, thus, remove it.

Figure: Rule 2: The position next to the first idle position (i.e. position 3here) is idle, thus, put a remaining token.

45 / 1



AnalysisHow the player should choose successive moves, with the goal offilling the bank as quickly as possible?

One can garner some strategic observations about how to play theGame by looking at small instances.

When n = 1, the player should simply fill the slot using aType-1 move.

When n = 2, the player must first play a Type-2, then aType-1 move.

As the Game proceeds, observation suggests that the playershould begin with a Type-1 move when n is odd and with aType-2 move when n is even.

Another easy observation is:The player should not play two successive moves of the sametype, because the second one just undoes the first.

46 / 1



Analysis (2)

A strategy is beginning to emerge:

1 Choose the initial move based on the parity of n.

2 Subsequently, alternate between the two types of moves.

This strategy is pleasingly simple, but:(a) Does it lead us to the required terminal state?(b) What is the cost of a (successful) play using this strategy?

The answers can be discovered if we reformulate the strategy to arecursive form:We can then use recurrences to prove that the proposed strategy isoptimal and to evaluate the cost of an optimal play of the order-ngame.

47 / 1



Analysis (3)

A token can be placed into the last bank-slot (i.e., the nth) via theType-2 move

move token from pile into bank-slot n

In order for this move to be eligible, the bank must be in thefollowing configuration:[tokens in slots 1, 2, . . . , n−2

],[no token in n−1

],[no token in n

]This configuration requires that the first n − 2 slots have beenfilled.2

Once the player has achieved this configuration and executed themandated move, the player henceforth ignores the token inbank-slot n. Here comes the recursion!

2Remark that n − 2 has the same parity as n.48 / 1



Analysis (4)the Game can be played by recursively executing the super-stepsdepicted below, on successively smaller banks and piles.

Figure: A schematic of the recursive play of the Game.

49 / 1



Analysis (4)the Game can be played by recursively executing the super-stepsdepicted below, on successively smaller banks and piles.

Figure: A schematic of the recursive play of the Game.49 / 1



Analysis (5)

The cost of the recursive solution is:

f (n) =

1 if n = 12 if n = 2f (n − 2) + 1 + f −1(n − 2) + f (n − 1) if n > 2

(3)

Let f −1 denote the bank-emptying operation.If one thinks in terms of mirror-image operations, then onediscovers a recursive solution for emptying the bank:

f −1(n) =

1 if n = 12 if n = 2f −1(n − 1) + f (n − 2) + 1 + f −1(n − 2) if n > 2

(4)

50 / 1



Analysis (6)

Taking the difference between expressions (??) and (??), we findthat the costs of f (n) and f −1(n) are equal!

f (n) − f −1(n) = f (n − 2) + 1 + f −1(n − 2) + f (n − 1)

− f −1(n − 1) − f (n − 2) − 1 − f −1(n − 2)

= f (n − 1) − f −1(n − 1)

· · ·= f (1) − f −1(1)

= 0

51 / 1



Analysis (7)

The final equation is:

f (n) =

1 if n = 12 if n = 2f (n − 1) + 2f (n − 2) + 1 if n > 2

(5)

52 / 1



Solving the recurrence

We can dramatically simplify the previous recurrence by focusingon the function

g(n) = f (n) + f (n − 1) for n ≥ 2

instead of on f .Elementary calculation shows that g(n) satisfies the recurrence

g(n) =

3 if n = 22g(n − 1) + 1 if n > 2

(6)

53 / 1



We have, thereby, replaced the initial bilinear recurrence by the(singly) linear recurrence (??).

We know how to solve geometric summations.

g(n) = 2n−1 + 2n−2 + · · ·+ 22 + 2 + 1 = 2n − 1 (7)

54 / 1



We can now return to evaluating f (n) in the light of our analysisof g(n).

f (n) =

1 if n = 1g(n)− f (n − 1) = (2n − 1)− f (n − 1) if n > 1

(8)We begin to solve the singly linear recurrence (??) for f (n) usingthe strategy we developed for simple geometric sums.We expand the recurrence in order to discern its pattern and thenanalyze the summation that the pattern leads to. In this case, weobserve:

f (n) = 2n − 2n−1 + f (n − 2) + 1− 1

= 2n − 2n−1 + 2n−2 − f (n − 3)− 1

= 2n − 2n−1 + 2n−2 − 2n−3 + f (n − 4) + 1− 1...

55 / 1



We have now reached the penultimate step in finding the value off (n); specifically, we have derived the following parity-specifiedsummations.

For even values of n : f (n) =n∑

k=1

(−1)k 2k (9)

For odd values of n : f (n) =n∑

k=0

(−1)k+1 2k (10)

56 / 1



We first gather the positive and negative terms in summation Wethereby find that, for odd values of n:

f (n) =(2n + 2n−2 + · · ·+ 2

)−(2n−1 + 2n−3 + · · ·+ 1

)= 2n−1 + 2n−3 + · · ·+ 1

= 2n−1 ·(

1 +1

4+

1

16+ · · ·+ 1

2n−1

)= 2n−1 ·

(1 +

1

4+

1

16+ · · ·+ 1

4(n−1)/2

)= 2n−1 · 4

3·

(1 −

(1

4

)(n+1)/2)

=2n+1

3·(

1 − 1

2n+1

)=

1

3

(2n+1 − 1

)57 / 1



The final touch

For each n, f (n) is given by:

f (n) =

(2n+1 − 1

)/3 if n is odd(

2n+1 − 2)/3 if n is even

(11)

58 / 1



A geometric approach for the case of odd n.Remark that 2n−1 is a perfect square whenever n is odd, we setout to represent f (n) as the aggregated area of a shrinkingsequence of squares, of successive dimensions

2(n−1)/2×2(n−1)/2, 2(n−3)/2×2(n−3)/2, 2(n−5)/2×2(n−5)/2, . . . , 1×1

59 / 1



We can now use a geometric construction to evaluate f (n) forarbitrary odd n. We take three copies of the cascade.

Figure: Evaluating f (n) for odd n by “almost” filling a large square60 / 1

lecture 2 maths for computer science recurrences

Documents