discrete mathematics and its applications -...

98
Discrete Mathematics and Its Applications Lecture 4: Advanced Counting Techniques: Divide-and-Conquer and Generating Functions MING GAO DaSE@ ECNU (for course related communications) [email protected] Dec. 5, 2017

Upload: phungdieu

Post on 21-Aug-2018

227 views

Category:

Documents


0 download

TRANSCRIPT

Discrete Mathematics and Its ApplicationsLecture 4: Advanced Counting Techniques: Divide-and-Conquer and

Generating Functions

MING GAO

DaSE@ ECNU(for course related communications)

[email protected]

Dec. 5, 2017

Outline

1 Divide-and-Conquer Recurrence RelationsDefinition and Examples of DCR2

Master Theorem

2 Generating FunctionsUseful Facts About Power SeriesExtended Binomial CoefficientCounting Problems and Generating FunctionsUsing Generating Functions to Solve Recurrence RelationsProving Identities via Generating Functions

3 Take-aways

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 2 / 43

Divide-and-Conquer Recurrence Relations

Divide-and-Conquer strategy

The divide-and-conquer strategy solves a problem P by:

1 Breaking P into subproblems that are themselves smallerinstances of the same type of problem (Divide step);

2 Recursively solving these subproblems (Solve step);

3 Appropriately combining their answers (Conquer step).

The real work to implement Divide-and-Conquer strategy is donepiecemeal, where the key works lay in three different places:

1 How to partition problem into subproblems;

2 At the very tail end of the recursion, how to solve the smallestsubproblems outright;

3 How to glue together the partial answers.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 3 / 43

Divide-and-Conquer Recurrence Relations

Divide-and-Conquer strategy

The divide-and-conquer strategy solves a problem P by:

1 Breaking P into subproblems that are themselves smallerinstances of the same type of problem (Divide step);

2 Recursively solving these subproblems (Solve step);

3 Appropriately combining their answers (Conquer step).

The real work to implement Divide-and-Conquer strategy is donepiecemeal, where the key works lay in three different places:

1 How to partition problem into subproblems;

2 At the very tail end of the recursion, how to solve the smallestsubproblems outright;

3 How to glue together the partial answers.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 3 / 43

Divide-and-Conquer Recurrence Relations

Mergesort example

The classic divide andconquer recurrence isMerge-sort’sT (n) = 2T (n/2) + O(n),which divides the data intoequal-sized halves andspends linear time mergingthe halves after they aresorted.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 4 / 43

Divide-and-Conquer Recurrence Relations

Mergesort example

The classic divide andconquer recurrence isMerge-sort’sT (n) = 2T (n/2) + O(n),which divides the data intoequal-sized halves andspends linear time mergingthe halves after they aresorted.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 4 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Definition of DCR2

Suppose that a recursive algorithm divides a problem of size n intoa subproblems, where each subproblem is of size n/b (for simplicity,assume that n is a multiple of b; in reality, the smaller problems areoften of size equal to the nearest integers either less than or equal to,or greater than or equal to, n/b). Also, suppose that a total of g(n)extra operations are required in the conquer step of the algorithmto combine the solutions of the subproblems into a solution of theoriginal problem. Then, if f (n) represents the number of operationsrequired to solve the problem of size n, it follows that f satisfies therecurrence relation

f (n) = af (n/b) + g(n).

This is called a divide-and-conquer recurrence relation (shortedin DCR2).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 5 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Binary search

Problem: We have an orderedsequence of numbers,a1, a2, · · · , an. Given x , decidewhether x is in the sequence ornot. For example, x = 17 in theright tree.

Solution: This binary searchalgorithm reduces the search foran element in a search sequenceof size n to the binary search forthis element in a search sequenceof size n/2, when n is even.

Hence, the problem of size n has been reduced to one problem ofsize n/2. The DCR2 for binary search is T (n) = T (n/2) + 2.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 6 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Binary search

Problem: We have an orderedsequence of numbers,a1, a2, · · · , an. Given x , decidewhether x is in the sequence ornot. For example, x = 17 in theright tree.Solution: This binary searchalgorithm reduces the search foran element in a search sequenceof size n to the binary search forthis element in a search sequenceof size n/2, when n is even.

Hence, the problem of size n has been reduced to one problem ofsize n/2. The DCR2 for binary search is T (n) = T (n/2) + 2.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 6 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Binary search

Problem: We have an orderedsequence of numbers,a1, a2, · · · , an. Given x , decidewhether x is in the sequence ornot. For example, x = 17 in theright tree.Solution: This binary searchalgorithm reduces the search foran element in a search sequenceof size n to the binary search forthis element in a search sequenceof size n/2, when n is even.

Hence, the problem of size n has been reduced to one problem ofsize n/2. The DCR2 for binary search is T (n) = T (n/2) + 2.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 6 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Find the maximum and minimum of a sequence

Let T (n) be the totalnumber of comparisonsneeded to find themaximum and minimumelements of the sequencewith n elements.

We have shown that a

problem of size n can be

reduced into two problems

of size n/2, when n is even.

Using two comparisons, One compares the maxima of the two se-quences, and the other compares the minima of the two sequences.This gives the recurrence relation T (n) = 2T (n/2) + 2, when n iseven.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 7 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Find the maximum and minimum of a sequence

Let T (n) be the totalnumber of comparisonsneeded to find themaximum and minimumelements of the sequencewith n elements.

We have shown that a

problem of size n can be

reduced into two problems

of size n/2, when n is even.

Using two comparisons, One compares the maxima of the two se-quences, and the other compares the minima of the two sequences.This gives the recurrence relation T (n) = 2T (n/2) + 2, when n iseven.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 7 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Find the maximum and minimum of a sequence

Let T (n) be the totalnumber of comparisonsneeded to find themaximum and minimumelements of the sequencewith n elements.

We have shown that a

problem of size n can be

reduced into two problems

of size n/2, when n is even.

Using two comparisons, One compares the maxima of the two se-quences, and the other compares the minima of the two sequences.

This gives the recurrence relation T (n) = 2T (n/2) + 2, when n iseven.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 7 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Find the maximum and minimum of a sequence

Let T (n) be the totalnumber of comparisonsneeded to find themaximum and minimumelements of the sequencewith n elements.

We have shown that a

problem of size n can be

reduced into two problems

of size n/2, when n is even.

Using two comparisons, One compares the maxima of the two se-quences, and the other compares the minima of the two sequences.This gives the recurrence relation T (n) = 2T (n/2) + 2, when n iseven.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 7 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Multiplication for integers

Suppose x and y are two n-bit integers, and assume for conveniencethat n is a power of 2.

As a first step toward multiplying x and y , we split each of them intotheir left and right halves, which are n/2 bits long, i.e.,

x = (a2n−1a2n−2 · · · an︸ ︷︷ ︸xL

an−1an−2 · · · a0︸ ︷︷ ︸xR

)2,

y = (b2n−1b2n−2 · · · bn︸ ︷︷ ︸yL

bn−1bn−2 · · · b0︸ ︷︷ ︸yR

)2.

That is x = 2nxL + xR and y = 2nyL + yR .Thus, we have xy = 22nxLyL + 2n(xLyR + xRyL) + xRyR . The sig-nificant operations are the four n/2-bit multiplications; these we canhandle by four recursive calls, then evaluates the preceding expressionin O(n) time. Thus, recurrence relation is T (n) = 4T (n/2) + O(n),when n is even.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 8 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Multiplication for integers

Suppose x and y are two n-bit integers, and assume for conveniencethat n is a power of 2.As a first step toward multiplying x and y , we split each of them intotheir left and right halves, which are n/2 bits long, i.e.,

x = (a2n−1a2n−2 · · · an︸ ︷︷ ︸xL

an−1an−2 · · · a0︸ ︷︷ ︸xR

)2,

y = (b2n−1b2n−2 · · · bn︸ ︷︷ ︸yL

bn−1bn−2 · · · b0︸ ︷︷ ︸yR

)2.

That is x = 2nxL + xR and y = 2nyL + yR .Thus, we have xy = 22nxLyL + 2n(xLyR + xRyL) + xRyR . The sig-nificant operations are the four n/2-bit multiplications; these we canhandle by four recursive calls, then evaluates the preceding expressionin O(n) time.

Thus, recurrence relation is T (n) = 4T (n/2) + O(n),when n is even.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 8 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Multiplication for integers

Suppose x and y are two n-bit integers, and assume for conveniencethat n is a power of 2.As a first step toward multiplying x and y , we split each of them intotheir left and right halves, which are n/2 bits long, i.e.,

x = (a2n−1a2n−2 · · · an︸ ︷︷ ︸xL

an−1an−2 · · · a0︸ ︷︷ ︸xR

)2,

y = (b2n−1b2n−2 · · · bn︸ ︷︷ ︸yL

bn−1bn−2 · · · b0︸ ︷︷ ︸yR

)2.

That is x = 2nxL + xR and y = 2nyL + yR .Thus, we have xy = 22nxLyL + 2n(xLyR + xRyL) + xRyR . The sig-nificant operations are the four n/2-bit multiplications; these we canhandle by four recursive calls, then evaluates the preceding expressionin O(n) time. Thus, recurrence relation is T (n) = 4T (n/2) + O(n),when n is even.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 8 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Using Gauss’s trick

Three multiplications xLyL, (xL + xR)(yL + yR), and xRyR are sufficesince

xLyR + xRyL = (xL + xR)(yL + yR)− xLyL − xRyR .

Thus, recurrence relation is T (n) = 3T (n/2)+O(n), when n is even.

The algorithm’s recursive calls form a tree structure;

At each successive level the subproblems get halved in size;

At the log2 nth level, the subproblems get down to size 1, andso the recursion ends, i.e., the height is log2 n;

The branching factor is 3: each problem recursively producesthree smaller ones, with the result that at depth k in the treethere are 3k subproblems, each of size n/2k .

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 9 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Using Gauss’s trick

Three multiplications xLyL, (xL + xR)(yL + yR), and xRyR are sufficesince

xLyR + xRyL = (xL + xR)(yL + yR)− xLyL − xRyR .

Thus, recurrence relation is T (n) = 3T (n/2)+O(n), when n is even.

The algorithm’s recursive calls form a tree structure;

At each successive level the subproblems get halved in size;

At the log2 nth level, the subproblems get down to size 1, andso the recursion ends, i.e., the height is log2 n;

The branching factor is 3: each problem recursively producesthree smaller ones, with the result that at depth k in the treethere are 3k subproblems, each of size n/2k .

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 9 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Using Gauss’s trick

For each subproblem, the total time therefore spends at depth k inthe tree is 3k × O( n

2k) = ( 3

2 )k × O(n).

At the very top level, when k = 0, we need O(n);

At the bottom, when k = log2 n, it is

O(3log2 n) = O(nlog2 3);

Between these two endpoints, the work done increasesgeometrically from O(n) to O(nlog2 3), by a factor of 3

2 perlevel.

The sum of any increasing geometric series is, within a constant fac-tor, simply the last term of the series. Therefore the overall runningtime is

O(nlog2 3) ≈ O(n1.59).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 10 / 43

Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2

Using Gauss’s trick

For each subproblem, the total time therefore spends at depth k inthe tree is 3k × O( n

2k) = ( 3

2 )k × O(n).

At the very top level, when k = 0, we need O(n);

At the bottom, when k = log2 n, it is

O(3log2 n) = O(nlog2 3);

Between these two endpoints, the work done increasesgeometrically from O(n) to O(nlog2 3), by a factor of 3

2 perlevel.

The sum of any increasing geometric series is, within a constant fac-tor, simply the last term of the series. Therefore the overall runningtime is

O(nlog2 3) ≈ O(n1.59).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 10 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Solution for DCR2

Theorem

Let f be an increasing function that satisfies the recurrence relation

f (n) = af (n/b) + c

whenever n is divisible by b, where a ≥ 1, b is an integer greaterthan 1, and c is a positive real number. Then

f (n) is

{O(nlogb a), if a > 1;O(log n), if a = 1.

Furthermore, when n = bk and a 6= 1, where k is a positive integer,

f (n) = C1nlogb a + C2,

where C1 = f (1) + c/(a− 1) and C2 = −c/(a− 1).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 11 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Solution for DCR2

Theorem

Let f be an increasing function that satisfies the recurrence relation

f (n) = af (n/b) + c

whenever n is divisible by b, where a ≥ 1, b is an integer greaterthan 1, and c is a positive real number. Then

f (n) is

{O(nlogb a), if a > 1;O(log n), if a = 1.

Furthermore, when n = bk and a 6= 1, where k is a positive integer,

f (n) = C1nlogb a + C2,

where C1 = f (1) + c/(a− 1) and C2 = −c/(a− 1).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 11 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Example

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 12 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Proof I

Suppose that f satisfies this recurrence relation whenever n is divisibleby b. Let n = bk , where k is a positive integer.

Then

f (n) = af (n/b) + c

= a2f (n/b2) + ac + c

= a3f (n/b3) + a2c + ac + c

= · · ·

= ak f (n/bk) +k−1∑i=0

aic

Because n/bk = 1, it follows that

f (n) = ak f (1) +k−1∑i=0

aic .

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 13 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Proof I

Suppose that f satisfies this recurrence relation whenever n is divisibleby b. Let n = bk , where k is a positive integer. Then

f (n) = af (n/b) + c

= a2f (n/b2) + ac + c

= a3f (n/b3) + a2c + ac + c

= · · ·

= ak f (n/bk) +k−1∑i=0

aic

Because n/bk = 1, it follows that

f (n) = ak f (1) +k−1∑i=0

aic .

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 13 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Proof I

Suppose that f satisfies this recurrence relation whenever n is divisibleby b. Let n = bk , where k is a positive integer. Then

f (n) = af (n/b) + c

= a2f (n/b2) + ac + c

= a3f (n/b3) + a2c + ac + c

= · · ·

= ak f (n/bk) +k−1∑i=0

aic

Because n/bk = 1, it follows that

f (n) = ak f (1) +k−1∑i=0

aic .

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 13 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Proof II

Case I a = 1: We have f (n) = ak f (1) +∑k−1

i=0 c .

Because n = bk , we have k = logb n. Hence,

f (n) = f (1) + c logb n.

When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k . Because f is increasing, it follows that f (n) ≤ f (bk+1) =f (1) + c(k + 1) = (f (1) + c) + ck ≤ (f (1) + c) + c logb n.Therefore, in both cases, f (n) is O(log n) when a = 1.Case II a > 1 : First assume that n = bk , where k is a positiveinteger. It follows that

f (n) = ak f (1) + c(ak − 1)/(a− 1)

= ak [f (1) + c/(a− 1)]− c/(a− 1)

= C1nlogb a + C2,

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 14 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Proof II

Case I a = 1: We have f (n) = ak f (1) +∑k−1

i=0 c .Because n = bk , we have k = logb n. Hence,

f (n) = f (1) + c logb n.

When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k . Because f is increasing, it follows that f (n) ≤ f (bk+1) =f (1) + c(k + 1) = (f (1) + c) + ck ≤ (f (1) + c) + c logb n.Therefore, in both cases, f (n) is O(log n) when a = 1.Case II a > 1 : First assume that n = bk , where k is a positiveinteger. It follows that

f (n) = ak f (1) + c(ak − 1)/(a− 1)

= ak [f (1) + c/(a− 1)]− c/(a− 1)

= C1nlogb a + C2,

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 14 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Proof II

Case I a = 1: We have f (n) = ak f (1) +∑k−1

i=0 c .Because n = bk , we have k = logb n. Hence,

f (n) = f (1) + c logb n.

When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k . Because f is increasing, it follows that f (n) ≤ f (bk+1) =f (1) + c(k + 1) = (f (1) + c) + ck ≤ (f (1) + c) + c logb n.

Therefore, in both cases, f (n) is O(log n) when a = 1.Case II a > 1 : First assume that n = bk , where k is a positiveinteger. It follows that

f (n) = ak f (1) + c(ak − 1)/(a− 1)

= ak [f (1) + c/(a− 1)]− c/(a− 1)

= C1nlogb a + C2,

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 14 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Proof II

Case I a = 1: We have f (n) = ak f (1) +∑k−1

i=0 c .Because n = bk , we have k = logb n. Hence,

f (n) = f (1) + c logb n.

When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k . Because f is increasing, it follows that f (n) ≤ f (bk+1) =f (1) + c(k + 1) = (f (1) + c) + ck ≤ (f (1) + c) + c logb n.Therefore, in both cases, f (n) is O(log n) when a = 1.

Case II a > 1 : First assume that n = bk , where k is a positiveinteger. It follows that

f (n) = ak f (1) + c(ak − 1)/(a− 1)

= ak [f (1) + c/(a− 1)]− c/(a− 1)

= C1nlogb a + C2,

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 14 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Proof II

Case I a = 1: We have f (n) = ak f (1) +∑k−1

i=0 c .Because n = bk , we have k = logb n. Hence,

f (n) = f (1) + c logb n.

When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k . Because f is increasing, it follows that f (n) ≤ f (bk+1) =f (1) + c(k + 1) = (f (1) + c) + ck ≤ (f (1) + c) + c logb n.Therefore, in both cases, f (n) is O(log n) when a = 1.Case II a > 1 : First assume that n = bk , where k is a positiveinteger. It follows that

f (n) = ak f (1) + c(ak − 1)/(a− 1)

= ak [f (1) + c/(a− 1)]− c/(a− 1)

= C1nlogb a + C2,

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 14 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Proof III

Because

ak = alogb n = blogb alogb n

= b(logb n)·(logb a) = blogb nlogb a

= nlogb a,

where C1 = f (1) + c/(a− 1) and C2 = −c/(a− 1).

When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k. Because f is increasing, it follows that

f (n) ≤ f (bk+1) = C1ak+1 + C2

≤ (C1a)alogb n + C2

≤ (C1a)nlogb a + C2

Because k ≤ logb n < k + 1. Hence, we have f (n) is O(nlogb a).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 15 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Proof III

Because

ak = alogb n = blogb alogb n

= b(logb n)·(logb a) = blogb nlogb a

= nlogb a,

where C1 = f (1) + c/(a− 1) and C2 = −c/(a− 1).When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k. Because f is increasing, it follows that

f (n) ≤ f (bk+1) = C1ak+1 + C2

≤ (C1a)alogb n + C2

≤ (C1a)nlogb a + C2

Because k ≤ logb n < k + 1. Hence, we have f (n) is O(nlogb a).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 15 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Proof III

Because

ak = alogb n = blogb alogb n

= b(logb n)·(logb a) = blogb nlogb a

= nlogb a,

where C1 = f (1) + c/(a− 1) and C2 = −c/(a− 1).When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k. Because f is increasing, it follows that

f (n) ≤ f (bk+1) = C1ak+1 + C2

≤ (C1a)alogb n + C2

≤ (C1a)nlogb a + C2

Because k ≤ logb n < k + 1. Hence, we have f (n) is O(nlogb a).MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 15 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Example I

Question: Letf (n) = 5f (n/2) + 3 and f (1) = 7. Find f (2k), wherek is a positive integer. Also, estimate f (n) if f is an increasingfunction.Solution: From the proof of the theorem, with a = 5, b = 2, andc = 3, we see that if n = 2k , then

f (n) = ak [f (1) + c/(a− 1)] + [−c/(a− 1)]

= 5k [7 + (3/4)]− 3/4

= 5k(31/4)− 3/4.

Also, if f (n) is increasing, the above theorem shows that f (n) isO(nlogb a) = O(nlog 5).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 16 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Example II

Binary search

Question: Give a big-O estimate for the number of comparisonsused by a binary search.Solution: We have known that f (n) = f (n/2) + 2 when n is even,where f is the number of comparisons required to perform a binarysearch on a sequence of size n.

Note that a = 1. Hence, from the above theorem, it follows thatf (n) is O(log n).

Search maximum and minimum elements

Question: Give a big-O estimate for the number of comparisonsused to locate the maximum and minimum elements in a sequence.Solution: We have already known that f (n) = 2f (n/2) + 2 witha = 2, b = 2, and c = 2.Hence, it follows that f (n) is O(nlog 2) = O(n).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 17 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Example II

Binary search

Question: Give a big-O estimate for the number of comparisonsused by a binary search.Solution: We have known that f (n) = f (n/2) + 2 when n is even,where f is the number of comparisons required to perform a binarysearch on a sequence of size n.Note that a = 1. Hence, from the above theorem, it follows thatf (n) is O(log n).

Search maximum and minimum elements

Question: Give a big-O estimate for the number of comparisonsused to locate the maximum and minimum elements in a sequence.Solution: We have already known that f (n) = 2f (n/2) + 2 witha = 2, b = 2, and c = 2.Hence, it follows that f (n) is O(nlog 2) = O(n).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 17 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Example II

Binary search

Question: Give a big-O estimate for the number of comparisonsused by a binary search.Solution: We have known that f (n) = f (n/2) + 2 when n is even,where f is the number of comparisons required to perform a binarysearch on a sequence of size n.Note that a = 1. Hence, from the above theorem, it follows thatf (n) is O(log n).

Search maximum and minimum elements

Question: Give a big-O estimate for the number of comparisonsused to locate the maximum and minimum elements in a sequence.Solution: We have already known that f (n) = 2f (n/2) + 2 witha = 2, b = 2, and c = 2.

Hence, it follows that f (n) is O(nlog 2) = O(n).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 17 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Example II

Binary search

Question: Give a big-O estimate for the number of comparisonsused by a binary search.Solution: We have known that f (n) = f (n/2) + 2 when n is even,where f is the number of comparisons required to perform a binarysearch on a sequence of size n.Note that a = 1. Hence, from the above theorem, it follows thatf (n) is O(log n).

Search maximum and minimum elements

Question: Give a big-O estimate for the number of comparisonsused to locate the maximum and minimum elements in a sequence.Solution: We have already known that f (n) = 2f (n/2) + 2 witha = 2, b = 2, and c = 2.Hence, it follows that f (n) is O(nlog 2) = O(n).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 17 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Master theorem

Theorem

Let T be an increasing function that satisfies the recurrence relation

T (n) = aT (n/b) + cnd

whenever n is divisible by b, where a ≥ 1, b is an integer greater than1, and c and d are real numbers with c positive and d nonnegative.Then

T (n) is

O(nd), if a < bd ;O(nd log n), if a = bd ;O(nlogb a), if a > bd ;

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 18 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Master theorem Cont’d

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 19 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Proof of Master theorem

At internal level k of the tree, the work done is

akc(n/bk)d

= cnd(a/bd)k

Therefore

T (n) = calogb n +

logb n−1∑k=0

cnd(a

bd)k

= calogb n + cndlogb n−1∑k=0

(a

bd)k

= cnlogb a + cndlogb n−1∑k=0

(a

bd)k

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 20 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Proof of Master theorem

At internal level k of the tree, the work done is

akc(n/bk)d = cnd(a/bd)k

Therefore

T (n) = calogb n +

logb n−1∑k=0

cnd(a

bd)k

= calogb n + cndlogb n−1∑k=0

(a

bd)k

= cnlogb a + cndlogb n−1∑k=0

(a

bd)k

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 20 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Case I proof

Case I

If a/bd = 1, that is a = bd and d = logb a.

T (n) = cnlogb a + cndlogb n−1∑k=0

(a

bd)k

= cnd + cndlogb n−1∑k=0

1 = cnd + cnd logb n

= O(nd log n)

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 21 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Case II proof

Case II

If a/bd < 1, that is a < bd and d > logb a.

T (n) = cnlogb a + cndlogb n−1∑k=0

(a

bd)k

< cnd + cndlogb n−1∑k=0

(a

bd)k

< cnd + cnd∞∑k=0

(a

bd)k = cnd

(1 +

1

1− a/bd)

= O(nd)

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 22 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Case III proof

Case III

If a/bd > 1, that is a > bd and d < logb a.

T (n) = cnlogb a + cndlogb n−1∑k=0

(a

bd)k = cnd + cnd

(a/bd)logb n − 1

(a/bd)− 1

< cnlogb a + cnd(a/bd)logb n1

(a/bd)− 1

= cnlogb a + cnd(a/bd)logb nΘ(1)

= cnlogb a + cnd(alogb n/bd logb n)Θ(1)

= cnlogb a + cnd(nlogb a/nd)Θ(1) = cnlogb a + cnlogb aΘ(1)

= O(nlogb a)

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 23 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Example I of Master theorem

wherea = 2, b = 2and d = 1,that isa = bd .Therefore,we have

T (n) = O(n logn).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 24 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Example II of Master theorem

where a = 1, b = 2 and d = 1,that is a < bd . Therefore, wehave

T (n) = O(nd) = O(n).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 25 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Example III

Question: Let # comparisons used by the Mergesort to sort a listof n elements be less than M(n), where M(n) = 2M(n/2) + n.Solution: By the master theorem, we find that M(n) is O(n log n).

Question: Let f (n) be # bit operations for multiplying two n-bitintegers, where f (n) = 3f (n/2) + Cn.Solution: Hence, from the master theorem, it follows that f (n) isO(nlog 3). Note that log 3 ≈ 1.6.

Question: Let f (n) be # multiplications and additions required tomultiply two n × n matrices, where f (n) = 7f (n/2) + 15n2/4.Solution: Hence, from the master theorem, it follows that f (n) isO(nlog 7). Note that log 7 ≈ 2.8.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 26 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Example III

Question: Let # comparisons used by the Mergesort to sort a listof n elements be less than M(n), where M(n) = 2M(n/2) + n.Solution: By the master theorem, we find that M(n) is O(n log n).

Question: Let f (n) be # bit operations for multiplying two n-bitintegers, where f (n) = 3f (n/2) + Cn.Solution: Hence, from the master theorem, it follows that f (n) isO(nlog 3). Note that log 3 ≈ 1.6.

Question: Let f (n) be # multiplications and additions required tomultiply two n × n matrices, where f (n) = 7f (n/2) + 15n2/4.Solution: Hence, from the master theorem, it follows that f (n) isO(nlog 7). Note that log 7 ≈ 2.8.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 26 / 43

Divide-and-Conquer Recurrence Relations Master Theorem

Example III

Question: Let # comparisons used by the Mergesort to sort a listof n elements be less than M(n), where M(n) = 2M(n/2) + n.Solution: By the master theorem, we find that M(n) is O(n log n).

Question: Let f (n) be # bit operations for multiplying two n-bitintegers, where f (n) = 3f (n/2) + Cn.Solution: Hence, from the master theorem, it follows that f (n) isO(nlog 3). Note that log 3 ≈ 1.6.

Question: Let f (n) be # multiplications and additions required tomultiply two n × n matrices, where f (n) = 7f (n/2) + 15n2/4.Solution: Hence, from the master theorem, it follows that f (n) isO(nlog 7). Note that log 7 ≈ 2.8.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 26 / 43

Generating Functions

Generating functions

Definition

The generating function for sequence a0, a1, · · · , ak , · · · of real num-bers is the infinite series

G (x) = a0 + a1x + · · ·+ akxk + · · · =

∞∑k=0

akxk .

Examples

{ak} g(x)

ak = 3∑∞

k=0 3xk

ak = k + 1∑∞

k=0(k + 1)xk

ak = 2k∑∞

k=0 2kxk

ak = 1(k = 0, 1, · · · , 5)∑5

k=0 xk = x6−1

x−1

ak = C (m, k)(k = 0, 1, · · · ,m)∑m

k=0 C (m, k)xk = (1 + x)m

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 27 / 43

Generating Functions

Generating functions

Definition

The generating function for sequence a0, a1, · · · , ak , · · · of real num-bers is the infinite series

G (x) = a0 + a1x + · · ·+ akxk + · · · =

∞∑k=0

akxk .

Examples

{ak} g(x)

ak = 3∑∞

k=0 3xk

ak = k + 1∑∞

k=0(k + 1)xk

ak = 2k∑∞

k=0 2kxk

ak = 1(k = 0, 1, · · · , 5)∑5

k=0 xk = x6−1

x−1

ak = C (m, k)(k = 0, 1, · · · ,m)∑m

k=0 C (m, k)xk = (1 + x)m

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 27 / 43

Generating Functions Useful Facts About Power Series

Operations of generating functions

Theorem

Let f (x) =∑∞

k=0 akxk and g(x) =

∑∞k=0 bkx

k . Then

f (x) + g(x) =∞∑k=0

(ak + bk)xk , f (x)g(x) =∞∑k=0

( k∑j=0

ajbk−j)xk

Examples of power series

{ak} g(x)

ak = 1∑∞

k=0 xk = 1

1−x for |x | < 1

ak = ak∑∞

k=0 akxk = 1

1−ax for |ax | < 1

ak = k + 1∑∞

k=0

(∑kj=0 1

)xk = 1

(1−x)2

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 28 / 43

Generating Functions Useful Facts About Power Series

Operations of generating functions

Theorem

Let f (x) =∑∞

k=0 akxk and g(x) =

∑∞k=0 bkx

k . Then

f (x) + g(x) =∞∑k=0

(ak + bk)xk , f (x)g(x) =∞∑k=0

( k∑j=0

ajbk−j)xk

Examples of power series

{ak} g(x)

ak = 1∑∞

k=0 xk = 1

1−x for |x | < 1

ak = ak∑∞

k=0 akxk = 1

1−ax for |ax | < 1

ak = k + 1∑∞

k=0

(∑kj=0 1

)xk = 1

(1−x)2

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 28 / 43

Generating Functions Extended Binomial Coefficient

Extended binomial coefficient

Definition

Let u be a real number and k a nonnegative integer. Then theextended binomial coefficient

(uk

)is defined by(

u

k

)=

{u(u−1)···(u−k+1)

k! , if k > 0;1, if k = 0.

Examples

Question: Find the values of(−2

3

)and

(1/23

).

Solution: (−2

3

)=

(−2)(−3)(−4)

3!= −4;(

1/2

3

)=

(1/2)(1/2− 1)(1/2− 2)

3!=

1

16.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 29 / 43

Generating Functions Extended Binomial Coefficient

Extended binomial coefficient

Definition

Let u be a real number and k a nonnegative integer. Then theextended binomial coefficient

(uk

)is defined by(

u

k

)=

{u(u−1)···(u−k+1)

k! , if k > 0;1, if k = 0.

Examples

Question: Find the values of(−2

3

)and

(1/23

).

Solution: (−2

3

)=

(−2)(−3)(−4)

3!= −4;(

1/2

3

)=

(1/2)(1/2− 1)(1/2− 2)

3!=

1

16.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 29 / 43

Generating Functions Extended Binomial Coefficient

Corollary

Let n and r are two positive integers, the extended binomial coeffi-cient can be expressed as(

−nr

)= (−1)r

(n + r − 1

r

).

Proof: (−nr

)=−n(−n − 1) · · · (−n − r + 1)

r !

=(−1)rn(n + 1) · · · (n + r − 1)

r !

=(−1)r (n + r − 1)!

r !(n − 1)!

= (−1)r(n + r − 1

r

).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 30 / 43

Generating Functions Extended Binomial Coefficient

The extended binomial theorem

Let x be a real number with |x | < 1 and let u be a real number.Then

(1 + x)u =∞∑k=0

(u

k

)xk .

Examples

Find the generating functions for (1+x)−n and (1−x)−n for n ∈ Z+.Solution:

(1 + x)−n =∞∑k=0

(−nk

)xk =

∞∑k=0

(−1)k(n + k − 1

k

)xk .

Replacing x by −x , we find that

(1− x)−n =∞∑k=0

(n + k − 1

k

)xk .

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 31 / 43

Generating Functions Extended Binomial Coefficient

The extended binomial theorem

Let x be a real number with |x | < 1 and let u be a real number.Then

(1 + x)u =∞∑k=0

(u

k

)xk .

Examples

Find the generating functions for (1+x)−n and (1−x)−n for n ∈ Z+.Solution:

(1 + x)−n =∞∑k=0

(−nk

)xk =

∞∑k=0

(−1)k(n + k − 1

k

)xk .

Replacing x by −x , we find that

(1− x)−n =∞∑k=0

(n + k − 1

k

)xk .

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 31 / 43

Generating Functions Counting Problems and Generating Functions

Running example I

Question: Find the number of solutions of e1 + e2 + e3 = 17, wheree1, e2, and e3 are nonnegative integers with 2 ≤ e1 ≤ 5, 3 ≤ e2 ≤ 6,and 4 ≤ e3 ≤ 7.Solution: The number of solutions with the indicated constraints isthe coefficient of x17 in the expansion of

(x2 + x3 + x4 + x5)(x3 + x4 + x5 + x6)(x4 + x5 + x6 + x7).

Note thatWay # cases1: x5 · x6 · x6 12: x5 · x5 · x7 13: x4 · x6 · x7 1

It is not hard to see that the coefficient of x17 in this product is 3.Hence, there are three solutions.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 32 / 43

Generating Functions Counting Problems and Generating Functions

Running example I

Question: Find the number of solutions of e1 + e2 + e3 = 17, wheree1, e2, and e3 are nonnegative integers with 2 ≤ e1 ≤ 5, 3 ≤ e2 ≤ 6,and 4 ≤ e3 ≤ 7.Solution: The number of solutions with the indicated constraints isthe coefficient of x17 in the expansion of

(x2 + x3 + x4 + x5)(x3 + x4 + x5 + x6)(x4 + x5 + x6 + x7).

Note thatWay # cases1: x5 · x6 · x6 12: x5 · x5 · x7 13: x4 · x6 · x7 1

It is not hard to see that the coefficient of x17 in this product is 3.Hence, there are three solutions.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 32 / 43

Generating Functions Counting Problems and Generating Functions

Running example II

Question: In how many different ways can eight identical cookiesbe distributed among three distinct children if each child receives atleast two cookies and no more than four cookies?

Solution: Because each child receives at least two but no more thanfour cookies, for each child there is a factor equal to (x2 + x3 + x4)in the generating function for the sequence {cn}, where cn is thenumber of ways to distribute n cookies. Because there are threechildren, this generating function is

(x2 + x3 + x4)3.

We need the coefficient of x8 in this product.Computation shows that this coefficient equals 6. Hence, there aresix ways to distribute the cookies so that each child receives at leasttwo, but no more than four, cookies.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 33 / 43

Generating Functions Counting Problems and Generating Functions

Running example II

Question: In how many different ways can eight identical cookiesbe distributed among three distinct children if each child receives atleast two cookies and no more than four cookies?Solution: Because each child receives at least two but no more thanfour cookies, for each child there is a factor equal to (x2 + x3 + x4)in the generating function for the sequence {cn}, where cn is thenumber of ways to distribute n cookies.

Because there are threechildren, this generating function is

(x2 + x3 + x4)3.

We need the coefficient of x8 in this product.Computation shows that this coefficient equals 6. Hence, there aresix ways to distribute the cookies so that each child receives at leasttwo, but no more than four, cookies.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 33 / 43

Generating Functions Counting Problems and Generating Functions

Running example II

Question: In how many different ways can eight identical cookiesbe distributed among three distinct children if each child receives atleast two cookies and no more than four cookies?Solution: Because each child receives at least two but no more thanfour cookies, for each child there is a factor equal to (x2 + x3 + x4)in the generating function for the sequence {cn}, where cn is thenumber of ways to distribute n cookies. Because there are threechildren, this generating function is

(x2 + x3 + x4)3.

We need the coefficient of x8 in this product.

Computation shows that this coefficient equals 6. Hence, there aresix ways to distribute the cookies so that each child receives at leasttwo, but no more than four, cookies.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 33 / 43

Generating Functions Counting Problems and Generating Functions

Running example II

Question: In how many different ways can eight identical cookiesbe distributed among three distinct children if each child receives atleast two cookies and no more than four cookies?Solution: Because each child receives at least two but no more thanfour cookies, for each child there is a factor equal to (x2 + x3 + x4)in the generating function for the sequence {cn}, where cn is thenumber of ways to distribute n cookies. Because there are threechildren, this generating function is

(x2 + x3 + x4)3.

We need the coefficient of x8 in this product.Computation shows that this coefficient equals 6. Hence, there aresix ways to distribute the cookies so that each child receives at leasttwo, but no more than four, cookies.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 33 / 43

Generating Functions Counting Problems and Generating Functions

Running example III

Question: Please determine # ways to insert tokens worth 1,2, and$5 into a vending machine that costs r dollars not matter the orders.

Solution: The number of solutions with the indicated constraints isthe coefficient of x r in the expansion of

(1+x+x2 +x3 +· · · )(1+x2 +x4 +x6 +· · · )(1+x5 +x10 +x15 +· · · ).

When the order in which the tokens are inserted matters, the numberof ways to insert exactly n tokens to produce a total of r dollars isthe coefficient of x r in (x + x2 + x5)n,Because any number of tokens may be inserted, the number of waysto produce r dollars using $1, $2, or $5 tokens, when the order inwhich the tokens are inserted matters, is the coefficient of x r in

1 + (x + x2 + x5) + (x + x2 + x5)2 + · · · =1

1− (x + x2 + x5).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 34 / 43

Generating Functions Counting Problems and Generating Functions

Running example III

Question: Please determine # ways to insert tokens worth 1,2, and$5 into a vending machine that costs r dollars not matter the orders.Solution: The number of solutions with the indicated constraints isthe coefficient of x r in the expansion of

(1+x+x2 +x3 +· · · )(1+x2 +x4 +x6 +· · · )(1+x5 +x10 +x15 +· · · ).

When the order in which the tokens are inserted matters, the numberof ways to insert exactly n tokens to produce a total of r dollars isthe coefficient of x r in (x + x2 + x5)n,Because any number of tokens may be inserted, the number of waysto produce r dollars using $1, $2, or $5 tokens, when the order inwhich the tokens are inserted matters, is the coefficient of x r in

1 + (x + x2 + x5) + (x + x2 + x5)2 + · · · =1

1− (x + x2 + x5).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 34 / 43

Generating Functions Counting Problems and Generating Functions

Running example III

Question: Please determine # ways to insert tokens worth 1,2, and$5 into a vending machine that costs r dollars not matter the orders.Solution: The number of solutions with the indicated constraints isthe coefficient of x r in the expansion of

(1+x+x2 +x3 +· · · )(1+x2 +x4 +x6 +· · · )(1+x5 +x10 +x15 +· · · ).

When the order in which the tokens are inserted matters, the numberof ways to insert exactly n tokens to produce a total of r dollars isthe coefficient of x r in (x + x2 + x5)n,

Because any number of tokens may be inserted, the number of waysto produce r dollars using $1, $2, or $5 tokens, when the order inwhich the tokens are inserted matters, is the coefficient of x r in

1 + (x + x2 + x5) + (x + x2 + x5)2 + · · · =1

1− (x + x2 + x5).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 34 / 43

Generating Functions Counting Problems and Generating Functions

Running example III

Question: Please determine # ways to insert tokens worth 1,2, and$5 into a vending machine that costs r dollars not matter the orders.Solution: The number of solutions with the indicated constraints isthe coefficient of x r in the expansion of

(1+x+x2 +x3 +· · · )(1+x2 +x4 +x6 +· · · )(1+x5 +x10 +x15 +· · · ).

When the order in which the tokens are inserted matters, the numberof ways to insert exactly n tokens to produce a total of r dollars isthe coefficient of x r in (x + x2 + x5)n,Because any number of tokens may be inserted, the number of waysto produce r dollars using $1, $2, or $5 tokens, when the order inwhich the tokens are inserted matters, is the coefficient of x r in

1 + (x + x2 + x5) + (x + x2 + x5)2 + · · · =1

1− (x + x2 + x5).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 34 / 43

Generating Functions Counting Problems and Generating Functions

Running example IV

Question: Find # k-combinations of a set with n elements. Assumethat the binomial theorem has already been established.

Solution: Each of the n elements in the set contributes term (1 +x)to the generating function f (x) =

∑nk=0 akx

k . Here f (x) is thegenerating function for {ak}, where ak represents the number of k-combinations of a set with n elements. Hence, f (x) = (1 + x)n

But by the binomial theorem, we have

f (x) =n∑

k=0

(n

k

)xk .

Hence, C (n, k) is the number of k-combinations of a set with nelements.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 35 / 43

Generating Functions Counting Problems and Generating Functions

Running example IV

Question: Find # k-combinations of a set with n elements. Assumethat the binomial theorem has already been established.Solution: Each of the n elements in the set contributes term (1 +x)to the generating function f (x) =

∑nk=0 akx

k . Here f (x) is thegenerating function for {ak}, where ak represents the number of k-combinations of a set with n elements. Hence, f (x) = (1 + x)n

But by the binomial theorem, we have

f (x) =n∑

k=0

(n

k

)xk .

Hence, C (n, k) is the number of k-combinations of a set with nelements.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 35 / 43

Generating Functions Counting Problems and Generating Functions

Running example IV

Question: Find # k-combinations of a set with n elements. Assumethat the binomial theorem has already been established.Solution: Each of the n elements in the set contributes term (1 +x)to the generating function f (x) =

∑nk=0 akx

k . Here f (x) is thegenerating function for {ak}, where ak represents the number of k-combinations of a set with n elements. Hence, f (x) = (1 + x)n

But by the binomial theorem, we have

f (x) =n∑

k=0

(n

k

)xk .

Hence, C (n, k) is the number of k-combinations of a set with nelements.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 35 / 43

Generating Functions Counting Problems and Generating Functions

Running example IV

Question: Find # k-combinations of a set with n elements. Assumethat the binomial theorem has already been established.Solution: Each of the n elements in the set contributes term (1 +x)to the generating function f (x) =

∑nk=0 akx

k . Here f (x) is thegenerating function for {ak}, where ak represents the number of k-combinations of a set with n elements. Hence, f (x) = (1 + x)n

But by the binomial theorem, we have

f (x) =n∑

k=0

(n

k

)xk .

Hence, C (n, k) is the number of k-combinations of a set with nelements.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 35 / 43

Generating Functions Counting Problems and Generating Functions

Running example V

Question: Find # r -combinations from a set with n elements whenrepetition of elements is allowed.

Solution: Each of the n elements in the set contributes term (1 +x + x2 + · · · ) to the generating function f (x) =

∑∞k=0 akx

k . Heref (x) is the generating function for {ak}, where ak represents # r -combinations from a set with n elements with repetition. Hence,f (x) = (1 + x + x2 + · · · )n.As long as |x | < 1, we have 1 + x + x2 + · · · = 1

1−x . Thus f (x) =(1− x)−n.Applying the extended binomial theorem, it follows that

(1− x)−n =∞∑r=0

(−nr

)(−x)r .

Thus, we have(−n

r

)(−1)r = (−1)rC (n + r − 1, r)(−1)r =

(n+r−1r

).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 36 / 43

Generating Functions Counting Problems and Generating Functions

Running example V

Question: Find # r -combinations from a set with n elements whenrepetition of elements is allowed.Solution: Each of the n elements in the set contributes term (1 +x + x2 + · · · ) to the generating function f (x) =

∑∞k=0 akx

k . Heref (x) is the generating function for {ak}, where ak represents # r -combinations from a set with n elements with repetition. Hence,f (x) = (1 + x + x2 + · · · )n.

As long as |x | < 1, we have 1 + x + x2 + · · · = 11−x . Thus f (x) =

(1− x)−n.Applying the extended binomial theorem, it follows that

(1− x)−n =∞∑r=0

(−nr

)(−x)r .

Thus, we have(−n

r

)(−1)r = (−1)rC (n + r − 1, r)(−1)r =

(n+r−1r

).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 36 / 43

Generating Functions Counting Problems and Generating Functions

Running example V

Question: Find # r -combinations from a set with n elements whenrepetition of elements is allowed.Solution: Each of the n elements in the set contributes term (1 +x + x2 + · · · ) to the generating function f (x) =

∑∞k=0 akx

k . Heref (x) is the generating function for {ak}, where ak represents # r -combinations from a set with n elements with repetition. Hence,f (x) = (1 + x + x2 + · · · )n.As long as |x | < 1, we have 1 + x + x2 + · · · = 1

1−x . Thus f (x) =(1− x)−n.

Applying the extended binomial theorem, it follows that

(1− x)−n =∞∑r=0

(−nr

)(−x)r .

Thus, we have(−n

r

)(−1)r = (−1)rC (n + r − 1, r)(−1)r =

(n+r−1r

).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 36 / 43

Generating Functions Counting Problems and Generating Functions

Running example V

Question: Find # r -combinations from a set with n elements whenrepetition of elements is allowed.Solution: Each of the n elements in the set contributes term (1 +x + x2 + · · · ) to the generating function f (x) =

∑∞k=0 akx

k . Heref (x) is the generating function for {ak}, where ak represents # r -combinations from a set with n elements with repetition. Hence,f (x) = (1 + x + x2 + · · · )n.As long as |x | < 1, we have 1 + x + x2 + · · · = 1

1−x . Thus f (x) =(1− x)−n.Applying the extended binomial theorem, it follows that

(1− x)−n =∞∑r=0

(−nr

)(−x)r .

Thus, we have(−n

r

)(−1)r = (−1)rC (n + r − 1, r)(−1)r =

(n+r−1r

).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 36 / 43

Generating Functions Counting Problems and Generating Functions

Running example VI

Question: Find # ways to select r objects of n different kinds if wemust select at least one object of each kind.

Solution: Each of the n elements in the set contributes term (x +x2 + · · · ) to the generating function f (x) =

∑∞k=0 akx

k . Here f (x)is the generating function for {ak}, where ak represents the numberof r objects of n different kinds with at least one object of each kind.Hence, f (x) = (x + x2 + · · · )n.As long as |x | < 1, we have f (x) = xn(1 + x + x2 + · · · )n = xn

(1−x)n .Applying the extended binomial theorem, it follows that

f (x) = xn∞∑r=0

(−nr

)(−x)r =

∞∑t=n

(t − 1

t − n

)x t .

Hence, there are C (r−1, r−n) ways to select r objects of n differentkinds if we must select at least one object of each kind.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 37 / 43

Generating Functions Counting Problems and Generating Functions

Running example VI

Question: Find # ways to select r objects of n different kinds if wemust select at least one object of each kind.Solution: Each of the n elements in the set contributes term (x +x2 + · · · ) to the generating function f (x) =

∑∞k=0 akx

k . Here f (x)is the generating function for {ak}, where ak represents the numberof r objects of n different kinds with at least one object of each kind.Hence, f (x) = (x + x2 + · · · )n.

As long as |x | < 1, we have f (x) = xn(1 + x + x2 + · · · )n = xn

(1−x)n .Applying the extended binomial theorem, it follows that

f (x) = xn∞∑r=0

(−nr

)(−x)r =

∞∑t=n

(t − 1

t − n

)x t .

Hence, there are C (r−1, r−n) ways to select r objects of n differentkinds if we must select at least one object of each kind.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 37 / 43

Generating Functions Counting Problems and Generating Functions

Running example VI

Question: Find # ways to select r objects of n different kinds if wemust select at least one object of each kind.Solution: Each of the n elements in the set contributes term (x +x2 + · · · ) to the generating function f (x) =

∑∞k=0 akx

k . Here f (x)is the generating function for {ak}, where ak represents the numberof r objects of n different kinds with at least one object of each kind.Hence, f (x) = (x + x2 + · · · )n.As long as |x | < 1, we have f (x) = xn(1 + x + x2 + · · · )n = xn

(1−x)n .

Applying the extended binomial theorem, it follows that

f (x) = xn∞∑r=0

(−nr

)(−x)r =

∞∑t=n

(t − 1

t − n

)x t .

Hence, there are C (r−1, r−n) ways to select r objects of n differentkinds if we must select at least one object of each kind.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 37 / 43

Generating Functions Counting Problems and Generating Functions

Running example VI

Question: Find # ways to select r objects of n different kinds if wemust select at least one object of each kind.Solution: Each of the n elements in the set contributes term (x +x2 + · · · ) to the generating function f (x) =

∑∞k=0 akx

k . Here f (x)is the generating function for {ak}, where ak represents the numberof r objects of n different kinds with at least one object of each kind.Hence, f (x) = (x + x2 + · · · )n.As long as |x | < 1, we have f (x) = xn(1 + x + x2 + · · · )n = xn

(1−x)n .Applying the extended binomial theorem, it follows that

f (x) = xn∞∑r=0

(−nr

)(−x)r =

∞∑t=n

(t − 1

t − n

)x t .

Hence, there are C (r−1, r−n) ways to select r objects of n differentkinds if we must select at least one object of each kind.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 37 / 43

Generating Functions Using Generating Functions to Solve Recurrence Relations

Running example VI

Question: Solve recurrence relation ak = 3ak−1 for k = 1, 2, 3, · · ·and initial condition a0 = 2.

Solution: Let f (x) be the generating function for the sequence {ak},that is, f (x) =

∑∞k=0 akx

k . First note that

xf (x) =∞∑k=0

akxk+1 =

∞∑k=1

ak−1xk .

Using the recurrence relation, we see that

f (x)− 3xf (x) =∞∑k=0

akxk − 3

∞∑k=1

ak−1xk

= a0 +∞∑k=1

(ak − 3ak−1)xk = 2.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 38 / 43

Generating Functions Using Generating Functions to Solve Recurrence Relations

Running example VI

Question: Solve recurrence relation ak = 3ak−1 for k = 1, 2, 3, · · ·and initial condition a0 = 2.Solution: Let f (x) be the generating function for the sequence {ak},that is, f (x) =

∑∞k=0 akx

k . First note that

xf (x) =∞∑k=0

akxk+1 =

∞∑k=1

ak−1xk .

Using the recurrence relation, we see that

f (x)− 3xf (x) =∞∑k=0

akxk − 3

∞∑k=1

ak−1xk

= a0 +∞∑k=1

(ak − 3ak−1)xk = 2.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 38 / 43

Generating Functions Using Generating Functions to Solve Recurrence Relations

Running example VI

Question: Solve recurrence relation ak = 3ak−1 for k = 1, 2, 3, · · ·and initial condition a0 = 2.Solution: Let f (x) be the generating function for the sequence {ak},that is, f (x) =

∑∞k=0 akx

k . First note that

xf (x) =∞∑k=0

akxk+1 =

∞∑k=1

ak−1xk .

Using the recurrence relation, we see that

f (x)− 3xf (x) =∞∑k=0

akxk − 3

∞∑k=1

ak−1xk

= a0 +∞∑k=1

(ak − 3ak−1)xk = 2.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 38 / 43

Generating Functions Using Generating Functions to Solve Recurrence Relations

Running example VI Cont’d

Thus,f (x)− 3xf (x) = (1− 3x)f (x) = 2,

i.e.,f (x) = 2/(1− 3x).

Using the identity

1/(1− ax) =∞∑k=0

akxk ,

we havef (x) = 2

∞∑k=0

3kxk .

Consequently,ak = 2 · 3k .

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 39 / 43

Generating Functions Using Generating Functions to Solve Recurrence Relations

Running example VI Cont’d

Thus,f (x)− 3xf (x) = (1− 3x)f (x) = 2,

i.e.,f (x) = 2/(1− 3x).

Using the identity

1/(1− ax) =∞∑k=0

akxk ,

we havef (x) = 2

∞∑k=0

3kxk .

Consequently,ak = 2 · 3k .

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 39 / 43

Generating Functions Using Generating Functions to Solve Recurrence Relations

Running example VII

Question: Suppose that a valid codeword is an n-digit number indecimal notation containing an even number of 0s. Let an denotethe number of valid codewords of length n. Note that a1 = 9, andthe recurrence relation is

an = 8an−1 + 10n−1.

Solution: We extend this sequence by setting a0 = 1, then

f (x)− 1 =∞∑n=1

anxn =

∞∑n=1

(8an−1xn + 10n−1xn)

= 8x∞∑n=1

an−1xn−1 + x

∞∑n=1

10n−1xn−1

= 8x∞∑n=0

anxn + x

∞∑n=0

10nxn = 8xf (x) +x

1− 10x.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 40 / 43

Generating Functions Using Generating Functions to Solve Recurrence Relations

Running example VII

Question: Suppose that a valid codeword is an n-digit number indecimal notation containing an even number of 0s. Let an denotethe number of valid codewords of length n. Note that a1 = 9, andthe recurrence relation is

an = 8an−1 + 10n−1.

Solution: We extend this sequence by setting a0 = 1, then

f (x)− 1 =∞∑n=1

anxn =

∞∑n=1

(8an−1xn + 10n−1xn)

= 8x∞∑n=1

an−1xn−1 + x

∞∑n=1

10n−1xn−1

= 8x∞∑n=0

anxn + x

∞∑n=0

10nxn = 8xf (x) +x

1− 10x.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 40 / 43

Generating Functions Using Generating Functions to Solve Recurrence Relations

Running example VII Cont’d

That is,f (x)− 1 = 8xf (x) +

x

1− 10x

i.e.,

f (x) =1− 9x

(1− 8x)(1− 10x)=

1

2

( 1

1− 8x+

1

1− 10x

).

Furthermore,

f (x) =∞∑k=0

1

2(8k + 10k)xk .

Consequently,an =

1

2(8n + 10n).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 41 / 43

Generating Functions Using Generating Functions to Solve Recurrence Relations

Running example VII Cont’d

That is,f (x)− 1 = 8xf (x) +

x

1− 10x

i.e.,

f (x) =1− 9x

(1− 8x)(1− 10x)=

1

2

( 1

1− 8x+

1

1− 10x

).

Furthermore,

f (x) =∞∑k=0

1

2(8k + 10k)xk .

Consequently,an =

1

2(8n + 10n).

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 41 / 43

Generating Functions Proving Identities via Generating Functions

Running example VIII

Statement: Let n is a positive integer, using generating functionsto show that

n∑k=0

C (n, k)2 = C (2n, n).

Proof: First note that by the binomial theorem C (2n, n) is the co-efficient of xn in (1 + x)2n. However, we also have

(1 + x)2n = [(1 + x)n]2

= [C (n, 0) + C (n, 1)x + C (n, 2)x2 + · · ·+ C (n, n)xn]2

The coefficient of xn in this expression is∑n

k=0 C (n, k)C (n, n−k) =∑nk=0 C (n, k)2.

Because both C (2n, n) and∑n

k=0 C (n, k)2 represent the coefficientof xn in (1 + x)2n, they must be equal.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 42 / 43

Generating Functions Proving Identities via Generating Functions

Running example VIII

Statement: Let n is a positive integer, using generating functionsto show that

n∑k=0

C (n, k)2 = C (2n, n).

Proof: First note that by the binomial theorem C (2n, n) is the co-efficient of xn in (1 + x)2n.

However, we also have

(1 + x)2n = [(1 + x)n]2

= [C (n, 0) + C (n, 1)x + C (n, 2)x2 + · · ·+ C (n, n)xn]2

The coefficient of xn in this expression is∑n

k=0 C (n, k)C (n, n−k) =∑nk=0 C (n, k)2.

Because both C (2n, n) and∑n

k=0 C (n, k)2 represent the coefficientof xn in (1 + x)2n, they must be equal.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 42 / 43

Generating Functions Proving Identities via Generating Functions

Running example VIII

Statement: Let n is a positive integer, using generating functionsto show that

n∑k=0

C (n, k)2 = C (2n, n).

Proof: First note that by the binomial theorem C (2n, n) is the co-efficient of xn in (1 + x)2n. However, we also have

(1 + x)2n = [(1 + x)n]2

= [C (n, 0) + C (n, 1)x + C (n, 2)x2 + · · ·+ C (n, n)xn]2

The coefficient of xn in this expression is∑n

k=0 C (n, k)C (n, n−k) =∑nk=0 C (n, k)2.

Because both C (2n, n) and∑n

k=0 C (n, k)2 represent the coefficientof xn in (1 + x)2n, they must be equal.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 42 / 43

Generating Functions Proving Identities via Generating Functions

Running example VIII

Statement: Let n is a positive integer, using generating functionsto show that

n∑k=0

C (n, k)2 = C (2n, n).

Proof: First note that by the binomial theorem C (2n, n) is the co-efficient of xn in (1 + x)2n. However, we also have

(1 + x)2n = [(1 + x)n]2

= [C (n, 0) + C (n, 1)x + C (n, 2)x2 + · · ·+ C (n, n)xn]2

The coefficient of xn in this expression is∑n

k=0 C (n, k)C (n, n−k) =∑nk=0 C (n, k)2.

Because both C (2n, n) and∑n

k=0 C (n, k)2 represent the coefficientof xn in (1 + x)2n, they must be equal.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 42 / 43

Generating Functions Proving Identities via Generating Functions

Running example VIII

Statement: Let n is a positive integer, using generating functionsto show that

n∑k=0

C (n, k)2 = C (2n, n).

Proof: First note that by the binomial theorem C (2n, n) is the co-efficient of xn in (1 + x)2n. However, we also have

(1 + x)2n = [(1 + x)n]2

= [C (n, 0) + C (n, 1)x + C (n, 2)x2 + · · ·+ C (n, n)xn]2

The coefficient of xn in this expression is∑n

k=0 C (n, k)C (n, n−k) =∑nk=0 C (n, k)2.

Because both C (2n, n) and∑n

k=0 C (n, k)2 represent the coefficientof xn in (1 + x)2n, they must be equal.

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 42 / 43

Take-aways

Take-aways

Divide-and-Conquer Recurrence Relations

Definition and Examples of DCR2

Master Theorem

Generating Functions

Useful Facts About Power SeriesExtended Binomial CoefficientCounting Problems and Generating FunctionsUsing Generating Functions to Solve Recurrence RelationsProving Identities via Generating Functions

MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 43 / 43