discrete structures1 one, two, three, we’re… counting

Discrete Structures 1

One, two, three, we’re…One, two, three, we’re…

CountingCounting


Basic Counting PrinciplesBasic Counting Principles

Counting problemsCounting problems are of the following kind: are of the following kind:

““How manyHow many different 8-letter passwords are different 8-letter passwords are there?”there?”

““How manyHow many possible ways are there to pick 11 possible ways are there to pick 11 soccer players out of a 20-player team?”soccer players out of a 20-player team?”

Most importantly, counting is the basis for Most importantly, counting is the basis for computing computing probabilities of discrete eventsprobabilities of discrete events..

(“What is the probability of winning the (“What is the probability of winning the lottery?”) lottery?”)


Basic Counting PrinciplesBasic Counting PrinciplesThe sum rule:The sum rule:If a task can be done in nIf a task can be done in n11 ways and a second ways and a second task in ntask in n22 ways, and if these two tasks cannot ways, and if these two tasks cannot be done at the same time, then there are nbe done at the same time, then there are n11 + + nn22 ways to do either task. ways to do either task.

Example:Example: The department will award a free computer to The department will award a free computer to either a CS student or a CS professor. either a CS student or a CS professor. How many different choices are there, if there How many different choices are there, if there are 530 students and 15 professors?are 530 students and 15 professors?

There are 530 + 15 = 545 choices.There are 530 + 15 = 545 choices.



Generalized sum rule:Generalized sum rule:

If we have tasks TIf we have tasks T11, T, T22, …, T, …, Tmm that can be done that can be done in nin n11, n, n22, …, n, …, nmm ways, respectively, and no two ways, respectively, and no two of these tasks can be done at the same time, of these tasks can be done at the same time, then there are nthen there are n11 + n + n22 + … + n + … + nmm ways to do ways to do one of these tasks.one of these tasks.



The product rule:The product rule:

Suppose that a procedure can be broken down Suppose that a procedure can be broken down into two successive tasks. If there are ninto two successive tasks. If there are n11 ways ways to do the first task and nto do the first task and n22 ways to do the ways to do the second task after the first task has been done, second task after the first task has been done, then there are nthen there are n11nn22 ways to do the procedure. ways to do the procedure.


Basic Counting PrinciplesBasic Counting PrinciplesExample:Example:How many different license plates are there How many different license plates are there that contain exactly three English letters ?that contain exactly three English letters ?

Solution:Solution:

There are 26 possibilities to pick the first There are 26 possibilities to pick the first letter, then 26 possibilities for the second one, letter, then 26 possibilities for the second one, and 26 for the last one. and 26 for the last one.

So there are 26So there are 26262626 = 17576 different 26 = 17576 different license plates.license plates.



Generalized product rule:Generalized product rule:

If we have a procedure consisting of If we have a procedure consisting of sequential tasks Tsequential tasks T11, T, T22, …, T, …, Tmm that can be done that can be done in nin n11, n, n22, …, n, …, nmm ways, respectively, then there ways, respectively, then there are nare n11 n n22 … … n nmm ways to carry out the ways to carry out the procedure.procedure.


Basic Counting PrinciplesBasic Counting PrinciplesThe sum and product rules can also be phrased The sum and product rules can also be phrased in terms of in terms of set theoryset theory..

Sum rule:Sum rule: Let A Let A11, A, A22, …, A, …, Amm be disjoint sets. be disjoint sets. Then the number of ways to choose any element Then the number of ways to choose any element from one of these sets is |Afrom one of these sets is |A11 A A22 … … A Amm | = | =|A|A11| + |A| + |A22| + … + |A| + … + |Amm|.|.

Product rule:Product rule: Let A Let A11, A, A22, …, A, …, Amm be finite sets. be finite sets. Then the number of ways to choose one element Then the number of ways to choose one element from each set in the order Afrom each set in the order A11, A, A22, …, A, …, Am m is is |A|A11 A A22 … … A Amm | = |A | = |A11| | |A |A22| | … … |A |Amm|.|.


Inclusion-ExclusionInclusion-ExclusionHow many bit strings of length 8 either start with How many bit strings of length 8 either start with a 1 or end with 00?a 1 or end with 00?

Task 1:Task 1: Construct a string of length 8 that starts Construct a string of length 8 that starts with a 1.with a 1.

There is one way to pick the first bit (1), There is one way to pick the first bit (1), two ways to pick the second bit (0 or 1),two ways to pick the second bit (0 or 1),two ways to pick the third bit (0 or 1),two ways to pick the third bit (0 or 1),......two ways to pick the eighth bit (0 or 1).two ways to pick the eighth bit (0 or 1).

Product rule:Product rule: Task 1 can be done in 1 Task 1 can be done in 12277 = 128 = 128 ways.ways.


Inclusion-ExclusionInclusion-ExclusionTask 2:Task 2: Construct a string of length 8 that ends Construct a string of length 8 that ends with 00.with 00.

There are two ways to pick the first bit (0 or 1), There are two ways to pick the first bit (0 or 1), two ways to pick the second bit (0 or 1),two ways to pick the second bit (0 or 1),......two ways to pick the sixth bit (0 or 1),two ways to pick the sixth bit (0 or 1),one way to pick the seventh bit (0), andone way to pick the seventh bit (0), andone way to pick the eighth bit (0).one way to pick the eighth bit (0).

Product rule:Product rule: Task 2 can be done in 2 Task 2 can be done in 266 = 64 = 64 ways.ways.


Inclusion-ExclusionInclusion-Exclusion

Since there are 128 ways to do Task 1 and 64 ways Since there are 128 ways to do Task 1 and 64 ways to do Task 2, does this mean that there are 192 bit to do Task 2, does this mean that there are 192 bit strings either starting with 1 or ending with 00 ?strings either starting with 1 or ending with 00 ?

No, because here Task 1 and Task 2 can be done No, because here Task 1 and Task 2 can be done at at the same timethe same time..

When we carry out Task 1 and create strings When we carry out Task 1 and create strings starting with 1, some of these strings end with 00.starting with 1, some of these strings end with 00.

Therefore, we sometimes do Tasks 1 and 2 at the Therefore, we sometimes do Tasks 1 and 2 at the same time, so same time, so the sum rule does not applythe sum rule does not apply..


Inclusion-ExclusionInclusion-Exclusion

If we want to use the sum rule in such a case, If we want to use the sum rule in such a case, we have to subtract the cases when Tasks 1 we have to subtract the cases when Tasks 1 and 2 are done at the same time.and 2 are done at the same time.

How many cases are there, that is, how many How many cases are there, that is, how many strings start with 1 strings start with 1 andand end with 00? end with 00?

There is one way to pick the first bit (1), There is one way to pick the first bit (1), two ways for the second, …, sixth bit (0 or 1),two ways for the second, …, sixth bit (0 or 1),one way for the seventh, eighth bit (0).one way for the seventh, eighth bit (0).

Product rule:Product rule: In 2 In 255 = 32 cases, Tasks 1 and 2 = 32 cases, Tasks 1 and 2 are carried out at the same time.are carried out at the same time.


Inclusion-ExclusionInclusion-ExclusionSince there are 128 ways to complete Task 1 and Since there are 128 ways to complete Task 1 and 64 ways to complete Task 2, and in 32 of these 64 ways to complete Task 2, and in 32 of these cases Tasks 1 and 2 are completed at the same cases Tasks 1 and 2 are completed at the same time, there aretime, there are

128 + 64 – 32 = 160 ways to do either task.128 + 64 – 32 = 160 ways to do either task.

In set theory, this corresponds to sets AIn set theory, this corresponds to sets A11 and A and A22 that are that are notnot disjoint. Then we have: disjoint. Then we have:

|A|A11 A A22| = |A| = |A11| + |A| + |A22| - |A| - |A11 A A22||

This is called the This is called the principle of inclusion-principle of inclusion-exclusion.exclusion.


Tree DiagramsTree DiagramsHow many bit strings of length four do not have How many bit strings of length four do not have two consecutive 1s?two consecutive 1s?

Task 1Task 1 Task 2Task 2 Task 3Task 3 Task 4Task 4(1(1stst bit) bit) (2(2ndnd bit) bit) (3(3rdrd bit) bit) (4(4thth bit) bit)

00

0000

00

1111

0011 00 00

11

11 0000 00

1111

00There are 8 strings.There are 8 strings.


The Pigeonhole PrincipleThe Pigeonhole PrincipleThe pigeonhole principle:The pigeonhole principle: If (k + 1) or more If (k + 1) or more objects are placed into k boxes, then there is objects are placed into k boxes, then there is at at leastleast one box containing two or more of the one box containing two or more of the objects.objects.

Example 1:Example 1: If there are 11 players in a soccer If there are 11 players in a soccer team that wins 12-0, there must be at least one team that wins 12-0, there must be at least one player in the team who scored at least twice.player in the team who scored at least twice.

Example 2:Example 2: If you have 6 classes from Monday If you have 6 classes from Monday to Friday, there must be at least one day on to Friday, there must be at least one day on which you have at least two classes.which you have at least two classes.


The Pigeonhole PrincipleThe Pigeonhole Principle

The generalized pigeonhole principle:The generalized pigeonhole principle: If N If N objects are placed into k boxes, then there is objects are placed into k boxes, then there is at at leastleast one box containing at least one box containing at least N/kN/k of the of the objects.objects.

Example 1:Example 1: In our 60-student class, at least 12 In our 60-student class, at least 12 students will get the same letter grade (A, B, C, students will get the same letter grade (A, B, C, D, or F).D, or F).


The Pigeonhole PrincipleThe Pigeonhole Principle

Example 2:Example 2: Assume you have a drawer Assume you have a drawer containing a random distribution of a dozen containing a random distribution of a dozen brown socks and a dozen black socks. It is dark, brown socks and a dozen black socks. It is dark, so how many socks do you have to pick to be so how many socks do you have to pick to be sure that among them there is a matching pair?sure that among them there is a matching pair?

There are two types of socks, so if you pick at There are two types of socks, so if you pick at least 3 socks, there must be either at least two least 3 socks, there must be either at least two brown socks or at least two black socks.brown socks or at least two black socks.(in other words: objects = socks, boxes = 2 (in other words: objects = socks, boxes = 2 colors)colors)Generalized pigeonhole principle: Generalized pigeonhole principle: 3/23/2 = 2. = 2.


Permutations and Permutations and CombinationsCombinations

How many ways are there to pick a set of 3 How many ways are there to pick a set of 3 people from a group of 6?people from a group of 6?

There are 6 choices for the first person, 5 for There are 6 choices for the first person, 5 for the second one, and 4 for the third one, so the second one, and 4 for the third one, so there arethere are66554 = 120 ways to do this.4 = 120 ways to do this.

This is not the correct answer!This is not the correct answer!

For example, picking person C, then person A, For example, picking person C, then person A, and then person E leads to the and then person E leads to the same groupsame group as as first picking E, then C, and then A.first picking E, then C, and then A.However, these cases are counted However, these cases are counted separatelyseparately in the above computation.in the above computation.



So how can we compute how many different So how can we compute how many different subsets of people can be picked (that is, we subsets of people can be picked (that is, we want to disregard the order of picking) ?want to disregard the order of picking) ?

To find out about this, we need to look at To find out about this, we need to look at permutations.permutations.

A A permutationpermutation of a set of distinct objects is an of a set of distinct objects is an ordered arrangement of these objects.ordered arrangement of these objects.

An ordered arrangement of r elements of a set An ordered arrangement of r elements of a set is called an is called an r-permutationr-permutation..



Example:Example: Let S = {1, 2, 3}. Let S = {1, 2, 3}.The arrangement 3, 1, 2 is a permutation of S.The arrangement 3, 1, 2 is a permutation of S.The arrangement 3, 2 is a 2-permutation of S.The arrangement 3, 2 is a 2-permutation of S.

The number of r-permutations of a set with n The number of r-permutations of a set with n distinct elements is denoted by distinct elements is denoted by P(n, r).P(n, r).

We can calculate P(n, r) with the product rule:We can calculate P(n, r) with the product rule:

P(n, r) = nP(n, r) = n(n – 1)(n – 1)(n – 2) (n – 2) ……(n – r + 1).(n – r + 1).

(n choices for the first element, (n – 1) for the (n choices for the first element, (n – 1) for the second one, (n – 2) for the third one…)second one, (n – 2) for the third one…)



Example:Example:

P(8, 3) = 8P(8, 3) = 8776 = 3366 = 336 = (8= (87766554433221)/(51)/(54433221)1)

General formula:General formula:

P(n, r) = n!/(n – r)!P(n, r) = n!/(n – r)!

Knowing this, we can return to our initial Knowing this, we can return to our initial question:question:How many ways are there to pick a set of 3 How many ways are there to pick a set of 3 people from a group of 6 (disregarding the people from a group of 6 (disregarding the order of picking)?order of picking)?



An An r-combinationr-combination of elements of a set is an of elements of a set is an unordered selection of r elements from the set.unordered selection of r elements from the set.Thus, an r-combination is simply a subset of the Thus, an r-combination is simply a subset of the set with r elements.set with r elements.

Example:Example: Let S = {1, 2, 3, 4}. Let S = {1, 2, 3, 4}.Then {1, 3, 4} is a 3-combination from S.Then {1, 3, 4} is a 3-combination from S.

The number of r-combinations of a set with n The number of r-combinations of a set with n distinct elements is denoted by C(n, r).distinct elements is denoted by C(n, r).

Example:Example: C(4, 2) = 6, since, for example, the C(4, 2) = 6, since, for example, the 2-combinations of a set {1, 2, 3, 4} are {1, 2}, 2-combinations of a set {1, 2, 3, 4} are {1, 2}, {1, 3}, {1, 4}, {2, 3}, {2, 4}, {3, 4}.{1, 3}, {1, 4}, {2, 3}, {2, 4}, {3, 4}.



How can we calculate C(n, r)?How can we calculate C(n, r)?

Consider that we can obtain the r-permutation Consider that we can obtain the r-permutation of a set in the following way:of a set in the following way:

First,First, we form all the r-combinations of the set we form all the r-combinations of the set(there are C(n, r) such r-combinations).(there are C(n, r) such r-combinations).

Then,Then, we generate all possible orderings in we generate all possible orderings in each of these r-combinations (there are P(r, r) each of these r-combinations (there are P(r, r) such orderings in each case).such orderings in each case).

Therefore, we have:Therefore, we have:

P(n, r) = C(n, r)P(n, r) = C(n, r)P(r, r)P(r, r)



C(n, r) = P(n, r)/P(r, r)C(n, r) = P(n, r)/P(r, r) = n!/(n – r)!/(r!/(r – r)!)= n!/(n – r)!/(r!/(r – r)!) = n!/(r!(n – r)!)= n!/(r!(n – r)!)

Now we can answer our initial question:Now we can answer our initial question:

How many ways are there to pick a set of 3 How many ways are there to pick a set of 3 people from a group of 6 (disregarding the people from a group of 6 (disregarding the order of picking)?order of picking)?

C(6, 3) = 6!/(3!C(6, 3) = 6!/(3!3!) = 720/(63!) = 720/(66) = 720/36 = 206) = 720/36 = 20

There are 20 different ways, that is, 20 different There are 20 different ways, that is, 20 different groups to be picked.groups to be picked.



Corollary:Corollary: Let n and r be nonnegative integers with r Let n and r be nonnegative integers with r n. n.Then C(n, r) = C(n, n – r).Then C(n, r) = C(n, n – r).

Note that Note that “picking a group of r people from “picking a group of r people from a group of n people”a group of n people” is the same as is the same as “splitting a group of n people into a group “splitting a group of n people into a group of r people and another group of (n – r) of r people and another group of (n – r) people”.people”.

Please also look at proof on page 323.Please also look at proof on page 323.



Example:A soccer club has 8 senior and 7 junior members. For today’s match, the coach wants to have 6 senior and 5 junior players on the grass. How many possible configurations are there?

C(8, 6) C(7, 5) = 8!/(6! C(7, 5) = 8!/(6!2!) 2!) 7!/(5! 7!/(5!2!)2!) = 28= 2821 21 = 588= 588


CombinationsCombinationsWe also saw the following:We also saw the following:

),(!)!(

!

)]!([)!(

!),( rnC

rrn

n

rnnrn

nrnnC

This symmetry is intuitively plausible. For This symmetry is intuitively plausible. For example, let us consider a set containing six example, let us consider a set containing six elements (n = 6).elements (n = 6).

Picking twoPicking two elements and elements and leaving fourleaving four is is essentially the same as essentially the same as picking fourpicking four elements elements and and leaving twoleaving two..

In either case, our number of choices is the In either case, our number of choices is the number of possibilities to number of possibilities to dividedivide the set into the set into one set containing two elements and another one set containing two elements and another set containing four elements.set containing four elements.


CombinationsCombinations

Pascal’s Identity:Pascal’s Identity:

Let n and k be positive integers with n Let n and k be positive integers with n k. k.Then C(n + 1, k) = C(n, k – 1) + C(n, k).Then C(n + 1, k) = C(n, k – 1) + C(n, k).

How can this be explained?How can this be explained?

What is it good for?What is it good for?


CombinationsCombinationsImagine a set S containing n elements and a set Imagine a set S containing n elements and a set T containing (n + 1) elements, namely all T containing (n + 1) elements, namely all elements in S plus a new element elements in S plus a new element aa..

Calculating C(n + 1, k) is equivalent to answering Calculating C(n + 1, k) is equivalent to answering the question: How many subsets of T containing k the question: How many subsets of T containing k items are there?items are there?

Case I:Case I: The subset contains (k – 1) elements of S The subset contains (k – 1) elements of S plus the element plus the element aa: C(n, k – 1) choices.: C(n, k – 1) choices.

Case II:Case II: The subset contains k elements of S and The subset contains k elements of S and does not contain does not contain aa: C(n, k) choices.: C(n, k) choices.

Sum Rule:Sum Rule: C(n + 1, k) = C(n, k – 1) + C(n, k). C(n + 1, k) = C(n, k – 1) + C(n, k).


Pascal’s TrianglePascal’s TriangleIn Pascal’s triangle, each number is the sum of In Pascal’s triangle, each number is the sum of the numbers to its upper left and upper right:the numbers to its upper left and upper right:

11

11 11

11 22 11

11 33 33 11

11 44 66 44 11

…… …… …… …… …… ……


Pascal’s TrianglePascal’s TriangleSince we have C(n + 1, k) = C(n, k – 1) + C(n, k) andSince we have C(n + 1, k) = C(n, k – 1) + C(n, k) andC(0, 0) = 1, we can use Pascal’s triangle to simplify C(0, 0) = 1, we can use Pascal’s triangle to simplify the computation of C(n, k):the computation of C(n, k):

C(0, 0) = C(0, 0) = 11

C(1, 0) = C(1, 0) = 11C(1, 1) = C(1, 1) = 11

C(2, 0) = C(2, 0) = 11C(2, 1) = C(2, 1) = 22C(2, 2) = C(2, 2) = 11

C(3, 0) = C(3, 0) = 11C(3, 1) = C(3, 1) = 33C(3, 2) = C(3, 2) = 33C(3, 3) = C(3, 3) = 11

C(4, 0) = C(4, 0) = 11C(4, 1) = C(4, 1) = 44C(4, 2) = C(4, 2) = 66C(4, 3) = C(4, 3) = 44C(4, 4) = C(4, 4) = 11

kk

nn


Binomial CoefficientsBinomial CoefficientsExpressions of the form C(n, k) are also called Expressions of the form C(n, k) are also called binomial coefficientsbinomial coefficients..How come?How come?A A binomial expressionbinomial expression is the sum of two is the sum of two terms, such as (a + b).terms, such as (a + b).Now consider (a + b)Now consider (a + b)2 2 = (a + b)(a + b).= (a + b)(a + b).When expanding such expressions, we have When expanding such expressions, we have to form all possible products of a term in the to form all possible products of a term in the first factor and a term in the second factor:first factor and a term in the second factor:(a + b)(a + b)22 = a·a + a·b + b·a + b·b = a·a + a·b + b·a + b·bThen we can sum identical terms:Then we can sum identical terms:(a + b)(a + b)22 = a = a22 + 2ab + b + 2ab + b22


Binomial CoefficientsBinomial CoefficientsFor (a + b)For (a + b)3 3 = (a + b)(a + b)(a + b) we have= (a + b)(a + b)(a + b) we have

(a + b)(a + b)3 3 = = aaa + aab + aba + abb + baa + bab + aaa + aab + aba + abb + baa + bab + bba + bbbbba + bbb

(a + b)(a + b)3 3 = a= a33 + 3a + 3a22b + 3abb + 3ab22 + b + b33

There is only one term a3, because there is only one possibility to form it: Choose a from all three factors: C(3, 3) = 1.There is three times the term a2b, because there are three possibilities to choose a from two out of the three factors: C(3, 2) = 3.Similarly, there is three times the term ab2 (C(3, 1) = 3) and once the term b3 (C(3, 0) = 1).


Binomial CoefficientsBinomial Coefficients

This leads us to the following formula:This leads us to the following formula:

jn

j

jnn bajnCba

0

),()(

With the help of Pascal’s triangle, this formula With the help of Pascal’s triangle, this formula can considerably simplify the process of can considerably simplify the process of expanding powers of binomial expressions.expanding powers of binomial expressions.

For example, the fifth row of Pascal’s triangleFor example, the fifth row of Pascal’s triangle(1 – 4 – 6 – 4 – 1) helps us to compute (a + (1 – 4 – 6 – 4 – 1) helps us to compute (a + b)b)44::

(a + b)(a + b)44 = a = a44 + 4a + 4a33b + 6ab + 6a22bb22 + 4ab + 4ab33 + b + b44

(Binomial (Binomial Theorem)Theorem)


Now it’s Time for…Now it’s Time for…

RecurrenRecurrencece

RelationsRelations


Recurrence RelationsRecurrence Relations

A A recurrence relationrecurrence relation for the sequence {a for the sequence {ann} } is an equation that expresses ais an equation that expresses ann is terms of is terms of one or more of the previous terms of the one or more of the previous terms of the sequence, namely, asequence, namely, a00, a, a11, …, a, …, an-1n-1, for all , for all integers n with integers n with n n n n00, where n, where n00 is a nonnegative integer. is a nonnegative integer.

A sequence is called a A sequence is called a solutionsolution of a of a recurrence relation if it terms satisfy the recurrence relation if it terms satisfy the recurrence relation.recurrence relation.



In other words, a recurrence relation is like a In other words, a recurrence relation is like a recursively defined sequence, but recursively defined sequence, but without without specifying any initial values (initial specifying any initial values (initial conditions)conditions)..

Therefore, the same recurrence relation can Therefore, the same recurrence relation can have (and usually has) have (and usually has) multiple solutionsmultiple solutions..

If If bothboth the initial conditions and the the initial conditions and the recurrence relation are specified, then the recurrence relation are specified, then the sequence is sequence is uniquely uniquely determined.determined.



Example:Example: Consider the recurrence relation Consider the recurrence relation aann = 2a = 2an-1n-1 – a – an-2n-2 for n = 2, 3, 4, … for n = 2, 3, 4, …

Is the sequence {aIs the sequence {ann} with a} with ann=3n a solution of =3n a solution of this recurrence relation?this recurrence relation?

For n For n 2 we see that 2 we see that 2a2an-1n-1 – a – an-2n-2 = 2(3(n – 1)) – 3(n – 2) = 3n = a = 2(3(n – 1)) – 3(n – 2) = 3n = ann..

Therefore, {aTherefore, {ann} with a} with ann=3n is a solution of the =3n is a solution of the recurrence relation.recurrence relation.



Is the sequence {aIs the sequence {ann} with a} with ann=5 a solution of =5 a solution of the same recurrence relation?the same recurrence relation?

For n For n 2 we see that 2 we see that 2a2an-1n-1 – a – an-2n-2 = 2 = 25 - 5 = 5 = a5 - 5 = 5 = ann..

Therefore, {aTherefore, {ann} with a} with ann=5 is also a solution of =5 is also a solution of the recurrence relation.the recurrence relation.


Modeling with Recurrence Modeling with Recurrence RelationsRelations

Example:Example:

Someone invests 10,000 SR in a business Someone invests 10,000 SR in a business yielding a profit of 15% per year . With an yielding a profit of 15% per year . With an initial deposit of 1000 SR, how much money initial deposit of 1000 SR, how much money will be in his account after 30 years?will be in his account after 30 years?

Solution:Solution:

Let PLet Pnn denote the amount in the account after denote the amount in the account after n years.n years.

How can we determine PHow can we determine Pnn on the basis of P on the basis of Pn-1n-1??



We can derive the following We can derive the following recurrence relationrecurrence relation::

PPnn = P = Pn-1n-1 + 0.15P + 0.15Pn-1n-1 = 1.15P = 1.15Pn-1n-1..

The initial condition is PThe initial condition is P00 = 1000. = 1000.Then we have:Then we have:

PP11 = 1.15P = 1.15P00

PP22 = 1.15P = 1.15P11 = (1.15) = (1.15)22PP00

PP33 = 1.15P = 1.15P22 = (1.15) = (1.15)33PP00

……

PPnn = 1.15P = 1.15Pn-1n-1 = (1.15) = (1.15)nnPP00

We now have a We now have a formulaformula to calculate P to calculate Pnn for any for any natural number n and can avoid the iteration.natural number n and can avoid the iteration.



Let us use this formula to find PLet us use this formula to find P3030 under the under the

initial condition Pinitial condition P00 = 1000: = 1000:

PP3030 = (1.15) = (1.15)30301000 = 662111000 = 66211

After 30 years, the account contains 66211 After 30 years, the account contains 66211 SR.SR.



Another example:Another example:

Let aLet ann denote the number of bit strings of denote the number of bit strings of length n that do not have two consecutive 0s length n that do not have two consecutive 0s (“valid strings”). Find a recurrence relation (“valid strings”). Find a recurrence relation and give initial conditions for the sequence and give initial conditions for the sequence {a{ann}.}.

Solution:Solution:

Idea: The number of valid strings equals the Idea: The number of valid strings equals the number of valid strings ending with a 0 plus number of valid strings ending with a 0 plus the number of valid strings ending with a 1.the number of valid strings ending with a 1.



Let us assume that n Let us assume that n 3, so that the string 3, so that the string contains at least 3 bits.contains at least 3 bits.

Let us further assume that we know the Let us further assume that we know the number anumber an-1n-1 of valid strings of length (n – 1). of valid strings of length (n – 1).

Then how many valid strings of length n are Then how many valid strings of length n are there, if the string ends with a 1?there, if the string ends with a 1?

There are aThere are an-1n-1 such strings, namely the set of such strings, namely the set of valid strings of length (n – 1) with a 1 valid strings of length (n – 1) with a 1 appended to them.appended to them.

Note:Note: Whenever we append a 1 to a valid Whenever we append a 1 to a valid string, that string remains valid.string, that string remains valid.



Now we need to know: How many valid strings of Now we need to know: How many valid strings of length n are there, if the string ends with a length n are there, if the string ends with a 00??

Valid strings of length n ending with a 0 Valid strings of length n ending with a 0 must must have a 1 as their (n – 1)st bithave a 1 as their (n – 1)st bit (otherwise they (otherwise they would end with 00 and would not be valid).would end with 00 and would not be valid).

And what is the number of valid strings of length And what is the number of valid strings of length (n – 1) that end with a 1?(n – 1) that end with a 1?

We already know that there are aWe already know that there are an-1n-1 strings of strings of length n that end with a 1.length n that end with a 1.

Therefore, there are aTherefore, there are an-2n-2 strings of length (n – 1) strings of length (n – 1) that end with a 1.that end with a 1.



So there are aSo there are an-2n-2 valid strings of length n that valid strings of length n that end with a 0 (all valid strings of length (n – 2) end with a 0 (all valid strings of length (n – 2) with 10 appended to them).with 10 appended to them).

As we said before, the number of valid strings As we said before, the number of valid strings is the number of valid strings ending with a 0 is the number of valid strings ending with a 0 plus the number of valid strings ending with a plus the number of valid strings ending with a 1.1.

That gives us the following That gives us the following recurrence recurrence relationrelation::

aann = a = an-1n-1 + a + an-2n-2



What are the What are the initial conditionsinitial conditions??

aa11 = 2 (0 and 1) = 2 (0 and 1)

aa22 = 3 (01, 10, and 11) = 3 (01, 10, and 11)

aa33 = a = a22 + a + a11 = 3 + 2 = 5 = 3 + 2 = 5

aa44 = a = a33 + a + a22 = 5 + 3 = 8 = 5 + 3 = 8

aa55 = a = a44 + a + a33 = 8 + 5 = 13 = 8 + 5 = 13

……

This sequence satisfies the same recurrence This sequence satisfies the same recurrence relation as the relation as the Fibonacci sequenceFibonacci sequence..

Since aSince a11 = f = f33 and a and a22 = f = f44, we have a, we have ann = f = fn+2n+2..


Solving Recurrence RelationsSolving Recurrence Relations

In general, we would prefer to have an In general, we would prefer to have an explicit formulaexplicit formula to compute the value of a to compute the value of ann rather than conducting n iterations.rather than conducting n iterations.

For one class of recurrence relations, we can For one class of recurrence relations, we can obtain such formulas in a systematic way.obtain such formulas in a systematic way.

Those are the recurrence relations that Those are the recurrence relations that express the terms of a sequence as express the terms of a sequence as linear linear combinationscombinations of previous terms. of previous terms.



Definition:Definition: A linear homogeneous recurrence A linear homogeneous recurrence relation of degree k with constant coefficients relation of degree k with constant coefficients is a recurrence relation of the form:is a recurrence relation of the form:

aann = c = c11aan-1n-1 + c + c22aan-2n-2 + … + c + … + ckkaan-kn-k,,

Where cWhere c11, c, c22, …, c, …, ckk are real numbers, and c are real numbers, and ckk 0. 0.

A sequence satisfying such a recurrence A sequence satisfying such a recurrence relation is uniquely determined by the relation is uniquely determined by the recurrence relation and the k initial conditionsrecurrence relation and the k initial conditions

aa00 = C = C00, a, a11 = C = C11, a, a22 = C = C22, …, a, …, ak-1k-1 = C = Ck-1k-1..



Examples:Examples:

The recurrence relation PThe recurrence relation Pnn = (1.15)P = (1.15)Pn-1n-1

is a linear homogeneous recurrence relation of is a linear homogeneous recurrence relation of degree onedegree one..

The recurrence relation fThe recurrence relation fnn = f = fn-1n-1 + f + fn-2n-2

is a linear homogeneous recurrence relation of is a linear homogeneous recurrence relation of degree twodegree two..

The recurrence relation aThe recurrence relation ann = a = an-5n-5

is a linear homogeneous recurrence relation of is a linear homogeneous recurrence relation of degree fivedegree five..


Solving Recurrence RelationsSolving Recurrence RelationsBasically, when solving such recurrence Basically, when solving such recurrence relations, we try to find solutions of the form relations, we try to find solutions of the form aann = r = rnn, where r is a constant., where r is a constant.

aann = r = rnn is a solution of the recurrence relation is a solution of the recurrence relationaann = c = c11aan-1n-1 + c + c22aan-2n-2 + … + c + … + ckkaan-kn-k if and only if if and only if

rrnn = c = c11rrn-1 n-1 + c+ c22rrn-2 n-2 + … + c+ … + ckkrrn-kn-k..

Divide this equation by rDivide this equation by rn-kn-k and subtract the and subtract the right-hand side from the left:right-hand side from the left:

rrkk - c - c11rrk-1 k-1 - c- c22rrk-2 k-2 - … - c- … - ck-1k-1r - cr - ckk = 0 = 0

This is called the This is called the characteristic equationcharacteristic equation of of the recurrence relation.the recurrence relation.


Solving Recurrence RelationsSolving Recurrence RelationsThe solutions of this equation are called the The solutions of this equation are called the characteristic rootscharacteristic roots of the recurrence relation. of the recurrence relation.

Let us consider linear homogeneous recurrence Let us consider linear homogeneous recurrence relations of relations of degree twodegree two..

Theorem:Theorem: Let c Let c11 and c and c22 be real numbers. Suppose be real numbers. Suppose that rthat r22 – c – c11r – cr – c22 = 0 has two distinct roots r = 0 has two distinct roots r11 and r and r22..

Then the sequence {aThen the sequence {ann} is a solution of the } is a solution of the recurrence relation arecurrence relation ann = c = c11aan-1n-1 + c + c22aan-2n-2 if and only if if and only if aann = = 11rr11

nn + + 22rr22nn for n = 0, 1, 2, …, where for n = 0, 1, 2, …, where 11 and and

22 are constants. are constants.

See pp. 321 and 322 for the proof.See pp. 321 and 322 for the proof.



Example:Example: What is the solution of the recurrence What is the solution of the recurrence relation arelation ann = a = an-1n-1 + 2a + 2an-2n-2 with a with a00 = 2 and a = 2 and a11 = 7 ? = 7 ?

Solution:Solution: The characteristic equation of the The characteristic equation of the recurrence relation is rrecurrence relation is r22 – r – 2 = 0. – r – 2 = 0.

Its roots are r = 2 and r = -1.Its roots are r = 2 and r = -1.

Hence, the sequence {aHence, the sequence {ann} is a solution to the } is a solution to the recurrence relation if and only if:recurrence relation if and only if:

aann = = 1122n n + + 22(-1)(-1)nn for some constants for some constants 1 1 and and 22..



Given the equation aGiven the equation ann = = 1122n n + + 22(-1)(-1)nn and the and the initial conditions ainitial conditions a00 = 2 and a = 2 and a11 = 7, it follows that = 7, it follows that

aa00 = 2 = = 2 = 1 1 + + 22

aa11 = 7 = = 7 = 112 + 2 + 2 2 (-1)(-1)

Solving these two equations gives usSolving these two equations gives us11 = 3 and = 3 and 22 = -1. = -1.

Therefore, the solution to the recurrence relation Therefore, the solution to the recurrence relation and initial conditions is the sequence {aand initial conditions is the sequence {ann} with} with

aann = 3 = 322nn – (-1) – (-1)nn. .


Solving Recurrence RelationsSolving Recurrence Relationsaann = r = rnn is a solution of the linear homogeneous is a solution of the linear homogeneous recurrence relationrecurrence relationaann = c = c11aan-1n-1 + c + c22aan-2n-2 + … + c + … + ckkaan-kn-k

if and only ifif and only if

rrnn = c = c11rrn-1 n-1 + c+ c22rrn-2 n-2 + … + c+ … + ckkrrn-kn-k..

Divide this equation by rDivide this equation by rn-kn-k and subtract the and subtract the right-hand side from the left:right-hand side from the left:

rrkk - c - c11rrk-1 k-1 - c- c22rrk-2 k-2 - … - c- … - ck-1k-1r - cr - ckk = 0 = 0

This is called the This is called the characteristic equationcharacteristic equation of of the recurrence relation.the recurrence relation.


Solving Recurrence RelationsSolving Recurrence RelationsThe solutions of this equation are called the The solutions of this equation are called the characteristic rootscharacteristic roots of the recurrence relation. of the recurrence relation.

Let us consider linear homogeneous recurrence Let us consider linear homogeneous recurrence relations of relations of degree twodegree two..

Theorem:Theorem: Let c Let c11 and c and c22 be real numbers. Suppose be real numbers. Suppose that rthat r22 – c – c11r – cr – c22 = 0 has two distinct roots r = 0 has two distinct roots r11 and r and r22..

Then the sequence {aThen the sequence {ann} is a solution of the } is a solution of the recurrence relation arecurrence relation ann = c = c11aan-1n-1 + c + c22aan-2n-2 if and only if if and only if aann = = 11rr11

nn + + 22rr22nn for n = 0, 1, 2, …, where for n = 0, 1, 2, …, where 11 and and

22 are constants. are constants.

See pp. 321 and 322 for the proof.See pp. 321 and 322 for the proof.



Example:Example: Give an explicit formula for the Give an explicit formula for the Fibonacci numbers.Fibonacci numbers.

Solution:Solution: The Fibonacci numbers satisfy the The Fibonacci numbers satisfy the recurrence relation frecurrence relation fnn = f = fn-1n-1 + f + fn-2n-2 with initial with initial conditions fconditions f00 = 0 and f = 0 and f11 = 1. = 1.

The characteristic equation is rThe characteristic equation is r22 – r – 1 = 0. – r – 1 = 0.

Its roots areIts roots are

2

51,

2

5121

rr


Solving Recurrence RelationsSolving Recurrence RelationsTherefore, the Fibonacci numbers are given byTherefore, the Fibonacci numbers are given by

nn

nf

2

51

2

5121

for some constants for some constants 11 and and 22..

We can determine values for these constants We can determine values for these constants so that the sequence meets the conditions fso that the sequence meets the conditions f00

= 0 and f= 0 and f11 = 1: = 1: 0210 f

12

51

2

51211

f



The unique solution to this system of two The unique solution to this system of two equations and two variables isequations and two variables is

5

1,

5

121

So finally we obtained an explicit formula for So finally we obtained an explicit formula for the Fibonacci numbers:the Fibonacci numbers:

nn

nf

2

51

5

1

2

51

5

1



But what happens if the characteristic equation But what happens if the characteristic equation has only one root?has only one root?

How can we then match our equation with the How can we then match our equation with the initial conditions ainitial conditions a00 and a and a1 1 ??

Theorem:Theorem: Let c Let c11 and c and c22 be real numbers with be real numbers with cc22 0. Suppose that r 0. Suppose that r22 – c – c11r – cr – c22 = 0 has only one = 0 has only one root rroot r00. . A sequence {aA sequence {ann} is a solution of the recurrence } is a solution of the recurrence relation arelation ann = c = c11aan-1n-1 + c + c22aan-2n-2 if and only if if and only if aann = = 11rr00

nn + + 22nrnr00nn, for n = 0, 1, 2, …, where , for n = 0, 1, 2, …, where 11

and and 22 are constants. are constants.


Solving Recurrence RelationsSolving Recurrence RelationsExample:Example: What is the solution of the recurrence What is the solution of the recurrence relation arelation ann = 6a = 6an-1n-1 – 9a – 9an-2n-2 with a with a00 = 1 and a = 1 and a11 = 6? = 6?

Solution:Solution: The only root of r The only root of r22 – 6r + 9 = 0 is r – 6r + 9 = 0 is r00 = 3. = 3.Hence, the solution to the recurrence relation isHence, the solution to the recurrence relation is

aann = = 1133nn + + 22n3n3nn for some constants for some constants 11 and and 22..

To match the initial condition, we needTo match the initial condition, we need

aa00 = 1 = = 1 = 11

aa11 = 6 = = 6 = 113 + 3 + 2233

Solving these equations yields Solving these equations yields 11 = 1 and = 1 and 22 = 1. = 1.

Consequently, the overall solution is given byConsequently, the overall solution is given by

aann = 3 = 3nn + n3 + n3nn..

Discrete Structures 62Discrete Structures 62

TheoremTheorem• Let cLet c11, c, c22, …, c, …, ckk be real numbers. Suppose that the be real numbers. Suppose that the• characteristic equation characteristic equation • rrk k - c- c11rrk-1 k-1 - … - c- … - ckk = 0 has d (d = 0 has d (d k) distinct roots r k) distinct roots r11, r, r22, …, r, …, rdd

• with multiplicities mwith multiplicities m11, m, m22, …, m, …, mdd, respectively, , respectively, • so that mso that mii 1 and m 1 and m1 1 + m+ m2 2 + …+ m+ …+ md d = k.= k.

• Then, a sequence {aThen, a sequence {ann} is a solution of the recurrence } is a solution of the recurrence relation:relation:

• aann = c = c11aan-1 n-1 + c+ c22aan-2 n-2 + … + c+ … + ckkaan-kn-k

• iffiff• aann = ( = (1,01,0 + + 1,11,1n + … + n + … + 1,m1,m11-1-1nnmm

11-1-1) r) r11

nn

• + (+ (2,02,0 + + 2,12,1n + … + n + … + 2,m2,m22-1-1nnmm22

-1-1) r) r22nn

• +...+ (+...+ (d,0d,0 + + d,1d,1n + … + n + … + d,md,mdd-1-1nnmm

dd-1-1) r) rdd

n n

• for n=0,1,2,… where for n=0,1,2,… where i,ji,j are constants 1 are constants 1 i i d, 0 d, 0 j j m mi-i-

11

•


Non-homogeneousNon-homogeneous Linear Linear Recurrence RelationsRecurrence Relations

• These are of the form:These are of the form:• aann = c = c11aan-1 n-1 + c+ c22aan-2 n-2 + … + c+ … + ckkaan-k n-k + F(n) + F(n)

[NRL][NRL]• where F(n) is a function that depends only on n. where F(n) is a function that depends only on n. • The recurrence relation:The recurrence relation:• aann = c = c11aan-1 n-1 + c+ c22aan-2 n-2 + … + c+ … + ckkaan-k n-k

[HRL][HRL]• is the associated is the associated homogeneoushomogeneous recurrence relation. recurrence relation.

• TheoremTheorem• If {If {aan n

(p)(p)} is a particular solution of the non-homogeneous recurrence } is a particular solution of the non-homogeneous recurrence • relation relation [NRL][NRL], then every solution is of the form {, then every solution is of the form {aan n

(p)(p) + + aan n (h)(h)}, where}, where

• {{aan n (h)(h)} is a solution of the associated homogeneous recurrence } is a solution of the associated homogeneous recurrence

relationrelation• [HRL][HRL]..

• Note:Note: We know how to solve We know how to solve [HRL][HRL], we need only find one particular solution , we need only find one particular solution for for

• [NRL][NRL], but how? See next slide for some hints., but how? See next slide for some hints.


Finding a particular solutionFinding a particular solution• Suppose that {aSuppose that {ann} satisfies the linear non-homogeneous } satisfies the linear non-homogeneous

recurrencerecurrence• relation relation • aann = c = c11aan-1 n-1 + c+ c22aan-2 n-2 + … + c+ … + ckkaan-k n-k + F(n)+ F(n)• where cwhere cii , 1 , 1 i i k, are real numbers and k, are real numbers and • F(n) = (bF(n) = (bssnnss

+ b+ bs-1s-1nns-1s-1 + … + b+ … + b11n + bn + b00) z) znn

• where bwhere bii , 0 , 0 i i s, and z are real numbers. s, and z are real numbers.

• When z When z is notis not a root of the characteristic equation of the a root of the characteristic equation of the• associated homogeneous recurrence relation, there is a associated homogeneous recurrence relation, there is a

particularparticular• solution of the form:solution of the form:• (p(pssnnss

+ p+ ps-1s-1nns-1s-1 + … + p+ … + p11n + pn + p00)z)znn

• When z When z isis a root of the characteristic equation and its a root of the characteristic equation and its multiplicity multiplicity

• is m, there is a particular solution of the form:is m, there is a particular solution of the form:• nnmm(p(pssnnss

+ p+ ps-1s-1nns-1s-1 + … + p+ … + p11n + pn + p00)z)znn

• where pwhere pii, 0 , 0 i i s, are real numbers. s, are real numbers.


An ExampleAn Example

• aann = 7a = 7an-1 n-1 - 16a- 16an-2 n-2 + 12a+ 12an-3 n-3 + n4+ n4nn, with a, with a00 = −2, a = −2, a11 = 0 and a = 0 and a22 = 5. = 5.

• This is a non-homogeneous recurrence relation, so by lastThis is a non-homogeneous recurrence relation, so by last• theorem the solution is the sum of the solution to the associated theorem the solution is the sum of the solution to the associated • homogeneous recurrence relation and the particular solution.homogeneous recurrence relation and the particular solution.

• We begin by solving the associated homogeneous recurrence We begin by solving the associated homogeneous recurrence relation:relation:

• aann = 7a = 7an-1 n-1 - 16a- 16an-2 n-2 + 12a+ 12an-3 n-3 [HRL], which has the characteristic [HRL], which has the characteristic equation:equation:

• (r − 2)(r − 2)22(r − 3) = 0.(r − 3) = 0.

• Thus, [HRL] has a solution of the form: aThus, [HRL] has a solution of the form: ann(h)(h) = = 1122nn + + 22n2n2nn + + 3333nn

• We now need a particular solution to the non-homogeneous relation.We now need a particular solution to the non-homogeneous relation.• We have that F (n) = n4We have that F (n) = n4nn. Thus, the particular solution is of the form:. Thus, the particular solution is of the form:• aann

(p)(p) = (bn + c) 4 = (bn + c) 4nn. (4 is not a root of the equation above!). (4 is not a root of the equation above!)


Example (cont’d)Example (cont’d)• We now solve for b and c by plugging this into the initial equation:We now solve for b and c by plugging this into the initial equation:• ((Note:Note: Our method works because the equality holds for all n Our method works because the equality holds for all n 3 3))• aann = 7a = 7an-1 n-1 - 16a- 16an-2 n-2 + 12a+ 12an-3 n-3 + n4+ n4nn

• (bn + c) 4(bn + c) 4nn = 7(b(n − 1) + c) 4 = 7(b(n − 1) + c) 4n-1n-1 − 16(b(n − 2) + c) 4 − 16(b(n − 2) + c) 4n-2n-2 + 12(b(n − 3) + c) 4 + 12(b(n − 3) + c) 4n-3n-3 + + n4n4nn

• This leads after some algebra to:This leads after some algebra to:• (b − 16)n +(5b+c) = 0n + 0, which gives us the following system:(b − 16)n +(5b+c) = 0n + 0, which gives us the following system:• b − 16 = 0b − 16 = 0• 5b + c = 05b + c = 0• Hence, b = 16 and c = −80. Thus, the particular solution has the form:Hence, b = 16 and c = −80. Thus, the particular solution has the form:• aann

(p)(p) = (16n − 80)4 = (16n − 80)4nn = (n − 5)4 = (n − 5)4n+2n+2..

• And the general solution to the recurrence relation is:And the general solution to the recurrence relation is:• aann = a = ann

(h)(h) + a + ann(p)(p) = = 1122nn + + 22n2n2nn + + 3333n n + (n − 5)4+ (n − 5)4n+2n+2

• We can now use the initial conditions to solve for the We can now use the initial conditions to solve for the ii as usual. We as usual. We find:find:

• aann = a = ann(h)(h) + a + ann

(p)(p) = (17)2 = (17)2nn + (39/2)n2 + (39/2)n2nn + (61)3 + (61)3nn + (n − 5)4 + (n − 5)4n+2n+2


Check your understandingCheck your understanding

• Solve the recurrence relations Solve the recurrence relations (roots are given for simplicity):(roots are given for simplicity):• aann = 3a = 3an-1n-1 + 2n + 2n with awith a11 = 1 = 1 (r(r11 = 3) = 3)• aann = 5a = 5an-1n-1 - 6a - 6an-2n-2 + 7n + 7n (r(r11 = 3, r = 3, r22 = 2) = 2)

• What is the form of a particular solution for:What is the form of a particular solution for:

• aann = 6a = 6an-1n-1 - 9a - 9an-2n-2 + F(n) + F(n) (r(r11 = 3, multiplicity 2) = 3, multiplicity 2)

• When:When:• F(n) = 3F(n) = 3nn

• F(n) = n3F(n) = n3nn

• F(n) = nF(n) = n2222nn

• F(n) = (nF(n) = (n2 2 + 1)3+ 1)3nn


A final remarkA final remark

• Other methods exist for solving linear non-Other methods exist for solving linear non-homogeneous recurrence relations (e.g. method homogeneous recurrence relations (e.g. method of generating functions).of generating functions).

• And what about And what about non-linearnon-linear recurrence relations? recurrence relations? – Example: Time complexity for binary search:Example: Time complexity for binary search:

T(1) = aT(1) = a

T(n) = b + T(n/2) for nT(n) = b + T(n/2) for n 1 1

where a and b are algorithm-specific where a and b are algorithm-specific constants.constants.

– See Data Structures and Algorithms Course for more See Data Structures and Algorithms Course for more details.details.

discrete structures1 one, two, three, we’re… counting

Documents