discrete mathematics and its applications -...
TRANSCRIPT
Discrete Mathematics and Its ApplicationsLecture 4: Advanced Counting Techniques: Divide-and-Conquer and
Generating Functions
MING GAO
DaSE@ ECNU(for course related communications)
Dec. 5, 2017
Outline
1 Divide-and-Conquer Recurrence RelationsDefinition and Examples of DCR2
Master Theorem
2 Generating FunctionsUseful Facts About Power SeriesExtended Binomial CoefficientCounting Problems and Generating FunctionsUsing Generating Functions to Solve Recurrence RelationsProving Identities via Generating Functions
3 Take-aways
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 2 / 43
Divide-and-Conquer Recurrence Relations
Divide-and-Conquer strategy
The divide-and-conquer strategy solves a problem P by:
1 Breaking P into subproblems that are themselves smallerinstances of the same type of problem (Divide step);
2 Recursively solving these subproblems (Solve step);
3 Appropriately combining their answers (Conquer step).
The real work to implement Divide-and-Conquer strategy is donepiecemeal, where the key works lay in three different places:
1 How to partition problem into subproblems;
2 At the very tail end of the recursion, how to solve the smallestsubproblems outright;
3 How to glue together the partial answers.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 3 / 43
Divide-and-Conquer Recurrence Relations
Divide-and-Conquer strategy
The divide-and-conquer strategy solves a problem P by:
1 Breaking P into subproblems that are themselves smallerinstances of the same type of problem (Divide step);
2 Recursively solving these subproblems (Solve step);
3 Appropriately combining their answers (Conquer step).
The real work to implement Divide-and-Conquer strategy is donepiecemeal, where the key works lay in three different places:
1 How to partition problem into subproblems;
2 At the very tail end of the recursion, how to solve the smallestsubproblems outright;
3 How to glue together the partial answers.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 3 / 43
Divide-and-Conquer Recurrence Relations
Mergesort example
The classic divide andconquer recurrence isMerge-sort’sT (n) = 2T (n/2) + O(n),which divides the data intoequal-sized halves andspends linear time mergingthe halves after they aresorted.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 4 / 43
Divide-and-Conquer Recurrence Relations
Mergesort example
The classic divide andconquer recurrence isMerge-sort’sT (n) = 2T (n/2) + O(n),which divides the data intoequal-sized halves andspends linear time mergingthe halves after they aresorted.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 4 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Definition of DCR2
Suppose that a recursive algorithm divides a problem of size n intoa subproblems, where each subproblem is of size n/b (for simplicity,assume that n is a multiple of b; in reality, the smaller problems areoften of size equal to the nearest integers either less than or equal to,or greater than or equal to, n/b). Also, suppose that a total of g(n)extra operations are required in the conquer step of the algorithmto combine the solutions of the subproblems into a solution of theoriginal problem. Then, if f (n) represents the number of operationsrequired to solve the problem of size n, it follows that f satisfies therecurrence relation
f (n) = af (n/b) + g(n).
This is called a divide-and-conquer recurrence relation (shortedin DCR2).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 5 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Binary search
Problem: We have an orderedsequence of numbers,a1, a2, · · · , an. Given x , decidewhether x is in the sequence ornot. For example, x = 17 in theright tree.
Solution: This binary searchalgorithm reduces the search foran element in a search sequenceof size n to the binary search forthis element in a search sequenceof size n/2, when n is even.
Hence, the problem of size n has been reduced to one problem ofsize n/2. The DCR2 for binary search is T (n) = T (n/2) + 2.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 6 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Binary search
Problem: We have an orderedsequence of numbers,a1, a2, · · · , an. Given x , decidewhether x is in the sequence ornot. For example, x = 17 in theright tree.Solution: This binary searchalgorithm reduces the search foran element in a search sequenceof size n to the binary search forthis element in a search sequenceof size n/2, when n is even.
Hence, the problem of size n has been reduced to one problem ofsize n/2. The DCR2 for binary search is T (n) = T (n/2) + 2.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 6 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Binary search
Problem: We have an orderedsequence of numbers,a1, a2, · · · , an. Given x , decidewhether x is in the sequence ornot. For example, x = 17 in theright tree.Solution: This binary searchalgorithm reduces the search foran element in a search sequenceof size n to the binary search forthis element in a search sequenceof size n/2, when n is even.
Hence, the problem of size n has been reduced to one problem ofsize n/2. The DCR2 for binary search is T (n) = T (n/2) + 2.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 6 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Find the maximum and minimum of a sequence
Let T (n) be the totalnumber of comparisonsneeded to find themaximum and minimumelements of the sequencewith n elements.
We have shown that a
problem of size n can be
reduced into two problems
of size n/2, when n is even.
Using two comparisons, One compares the maxima of the two se-quences, and the other compares the minima of the two sequences.This gives the recurrence relation T (n) = 2T (n/2) + 2, when n iseven.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 7 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Find the maximum and minimum of a sequence
Let T (n) be the totalnumber of comparisonsneeded to find themaximum and minimumelements of the sequencewith n elements.
We have shown that a
problem of size n can be
reduced into two problems
of size n/2, when n is even.
Using two comparisons, One compares the maxima of the two se-quences, and the other compares the minima of the two sequences.This gives the recurrence relation T (n) = 2T (n/2) + 2, when n iseven.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 7 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Find the maximum and minimum of a sequence
Let T (n) be the totalnumber of comparisonsneeded to find themaximum and minimumelements of the sequencewith n elements.
We have shown that a
problem of size n can be
reduced into two problems
of size n/2, when n is even.
Using two comparisons, One compares the maxima of the two se-quences, and the other compares the minima of the two sequences.
This gives the recurrence relation T (n) = 2T (n/2) + 2, when n iseven.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 7 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Find the maximum and minimum of a sequence
Let T (n) be the totalnumber of comparisonsneeded to find themaximum and minimumelements of the sequencewith n elements.
We have shown that a
problem of size n can be
reduced into two problems
of size n/2, when n is even.
Using two comparisons, One compares the maxima of the two se-quences, and the other compares the minima of the two sequences.This gives the recurrence relation T (n) = 2T (n/2) + 2, when n iseven.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 7 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Multiplication for integers
Suppose x and y are two n-bit integers, and assume for conveniencethat n is a power of 2.
As a first step toward multiplying x and y , we split each of them intotheir left and right halves, which are n/2 bits long, i.e.,
x = (a2n−1a2n−2 · · · an︸ ︷︷ ︸xL
an−1an−2 · · · a0︸ ︷︷ ︸xR
)2,
y = (b2n−1b2n−2 · · · bn︸ ︷︷ ︸yL
bn−1bn−2 · · · b0︸ ︷︷ ︸yR
)2.
That is x = 2nxL + xR and y = 2nyL + yR .Thus, we have xy = 22nxLyL + 2n(xLyR + xRyL) + xRyR . The sig-nificant operations are the four n/2-bit multiplications; these we canhandle by four recursive calls, then evaluates the preceding expressionin O(n) time. Thus, recurrence relation is T (n) = 4T (n/2) + O(n),when n is even.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 8 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Multiplication for integers
Suppose x and y are two n-bit integers, and assume for conveniencethat n is a power of 2.As a first step toward multiplying x and y , we split each of them intotheir left and right halves, which are n/2 bits long, i.e.,
x = (a2n−1a2n−2 · · · an︸ ︷︷ ︸xL
an−1an−2 · · · a0︸ ︷︷ ︸xR
)2,
y = (b2n−1b2n−2 · · · bn︸ ︷︷ ︸yL
bn−1bn−2 · · · b0︸ ︷︷ ︸yR
)2.
That is x = 2nxL + xR and y = 2nyL + yR .Thus, we have xy = 22nxLyL + 2n(xLyR + xRyL) + xRyR . The sig-nificant operations are the four n/2-bit multiplications; these we canhandle by four recursive calls, then evaluates the preceding expressionin O(n) time.
Thus, recurrence relation is T (n) = 4T (n/2) + O(n),when n is even.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 8 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Multiplication for integers
Suppose x and y are two n-bit integers, and assume for conveniencethat n is a power of 2.As a first step toward multiplying x and y , we split each of them intotheir left and right halves, which are n/2 bits long, i.e.,
x = (a2n−1a2n−2 · · · an︸ ︷︷ ︸xL
an−1an−2 · · · a0︸ ︷︷ ︸xR
)2,
y = (b2n−1b2n−2 · · · bn︸ ︷︷ ︸yL
bn−1bn−2 · · · b0︸ ︷︷ ︸yR
)2.
That is x = 2nxL + xR and y = 2nyL + yR .Thus, we have xy = 22nxLyL + 2n(xLyR + xRyL) + xRyR . The sig-nificant operations are the four n/2-bit multiplications; these we canhandle by four recursive calls, then evaluates the preceding expressionin O(n) time. Thus, recurrence relation is T (n) = 4T (n/2) + O(n),when n is even.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 8 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Using Gauss’s trick
Three multiplications xLyL, (xL + xR)(yL + yR), and xRyR are sufficesince
xLyR + xRyL = (xL + xR)(yL + yR)− xLyL − xRyR .
Thus, recurrence relation is T (n) = 3T (n/2)+O(n), when n is even.
The algorithm’s recursive calls form a tree structure;
At each successive level the subproblems get halved in size;
At the log2 nth level, the subproblems get down to size 1, andso the recursion ends, i.e., the height is log2 n;
The branching factor is 3: each problem recursively producesthree smaller ones, with the result that at depth k in the treethere are 3k subproblems, each of size n/2k .
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 9 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Using Gauss’s trick
Three multiplications xLyL, (xL + xR)(yL + yR), and xRyR are sufficesince
xLyR + xRyL = (xL + xR)(yL + yR)− xLyL − xRyR .
Thus, recurrence relation is T (n) = 3T (n/2)+O(n), when n is even.
The algorithm’s recursive calls form a tree structure;
At each successive level the subproblems get halved in size;
At the log2 nth level, the subproblems get down to size 1, andso the recursion ends, i.e., the height is log2 n;
The branching factor is 3: each problem recursively producesthree smaller ones, with the result that at depth k in the treethere are 3k subproblems, each of size n/2k .
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 9 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Using Gauss’s trick
For each subproblem, the total time therefore spends at depth k inthe tree is 3k × O( n
2k) = ( 3
2 )k × O(n).
At the very top level, when k = 0, we need O(n);
At the bottom, when k = log2 n, it is
O(3log2 n) = O(nlog2 3);
Between these two endpoints, the work done increasesgeometrically from O(n) to O(nlog2 3), by a factor of 3
2 perlevel.
The sum of any increasing geometric series is, within a constant fac-tor, simply the last term of the series. Therefore the overall runningtime is
O(nlog2 3) ≈ O(n1.59).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 10 / 43
Divide-and-Conquer Recurrence Relations Definition and Examples of DCR2
Using Gauss’s trick
For each subproblem, the total time therefore spends at depth k inthe tree is 3k × O( n
2k) = ( 3
2 )k × O(n).
At the very top level, when k = 0, we need O(n);
At the bottom, when k = log2 n, it is
O(3log2 n) = O(nlog2 3);
Between these two endpoints, the work done increasesgeometrically from O(n) to O(nlog2 3), by a factor of 3
2 perlevel.
The sum of any increasing geometric series is, within a constant fac-tor, simply the last term of the series. Therefore the overall runningtime is
O(nlog2 3) ≈ O(n1.59).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 10 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Solution for DCR2
Theorem
Let f be an increasing function that satisfies the recurrence relation
f (n) = af (n/b) + c
whenever n is divisible by b, where a ≥ 1, b is an integer greaterthan 1, and c is a positive real number. Then
f (n) is
{O(nlogb a), if a > 1;O(log n), if a = 1.
Furthermore, when n = bk and a 6= 1, where k is a positive integer,
f (n) = C1nlogb a + C2,
where C1 = f (1) + c/(a− 1) and C2 = −c/(a− 1).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 11 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Solution for DCR2
Theorem
Let f be an increasing function that satisfies the recurrence relation
f (n) = af (n/b) + c
whenever n is divisible by b, where a ≥ 1, b is an integer greaterthan 1, and c is a positive real number. Then
f (n) is
{O(nlogb a), if a > 1;O(log n), if a = 1.
Furthermore, when n = bk and a 6= 1, where k is a positive integer,
f (n) = C1nlogb a + C2,
where C1 = f (1) + c/(a− 1) and C2 = −c/(a− 1).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 11 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Example
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 12 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Proof I
Suppose that f satisfies this recurrence relation whenever n is divisibleby b. Let n = bk , where k is a positive integer.
Then
f (n) = af (n/b) + c
= a2f (n/b2) + ac + c
= a3f (n/b3) + a2c + ac + c
= · · ·
= ak f (n/bk) +k−1∑i=0
aic
Because n/bk = 1, it follows that
f (n) = ak f (1) +k−1∑i=0
aic .
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 13 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Proof I
Suppose that f satisfies this recurrence relation whenever n is divisibleby b. Let n = bk , where k is a positive integer. Then
f (n) = af (n/b) + c
= a2f (n/b2) + ac + c
= a3f (n/b3) + a2c + ac + c
= · · ·
= ak f (n/bk) +k−1∑i=0
aic
Because n/bk = 1, it follows that
f (n) = ak f (1) +k−1∑i=0
aic .
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 13 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Proof I
Suppose that f satisfies this recurrence relation whenever n is divisibleby b. Let n = bk , where k is a positive integer. Then
f (n) = af (n/b) + c
= a2f (n/b2) + ac + c
= a3f (n/b3) + a2c + ac + c
= · · ·
= ak f (n/bk) +k−1∑i=0
aic
Because n/bk = 1, it follows that
f (n) = ak f (1) +k−1∑i=0
aic .
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 13 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Proof II
Case I a = 1: We have f (n) = ak f (1) +∑k−1
i=0 c .
Because n = bk , we have k = logb n. Hence,
f (n) = f (1) + c logb n.
When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k . Because f is increasing, it follows that f (n) ≤ f (bk+1) =f (1) + c(k + 1) = (f (1) + c) + ck ≤ (f (1) + c) + c logb n.Therefore, in both cases, f (n) is O(log n) when a = 1.Case II a > 1 : First assume that n = bk , where k is a positiveinteger. It follows that
f (n) = ak f (1) + c(ak − 1)/(a− 1)
= ak [f (1) + c/(a− 1)]− c/(a− 1)
= C1nlogb a + C2,
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 14 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Proof II
Case I a = 1: We have f (n) = ak f (1) +∑k−1
i=0 c .Because n = bk , we have k = logb n. Hence,
f (n) = f (1) + c logb n.
When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k . Because f is increasing, it follows that f (n) ≤ f (bk+1) =f (1) + c(k + 1) = (f (1) + c) + ck ≤ (f (1) + c) + c logb n.Therefore, in both cases, f (n) is O(log n) when a = 1.Case II a > 1 : First assume that n = bk , where k is a positiveinteger. It follows that
f (n) = ak f (1) + c(ak − 1)/(a− 1)
= ak [f (1) + c/(a− 1)]− c/(a− 1)
= C1nlogb a + C2,
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 14 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Proof II
Case I a = 1: We have f (n) = ak f (1) +∑k−1
i=0 c .Because n = bk , we have k = logb n. Hence,
f (n) = f (1) + c logb n.
When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k . Because f is increasing, it follows that f (n) ≤ f (bk+1) =f (1) + c(k + 1) = (f (1) + c) + ck ≤ (f (1) + c) + c logb n.
Therefore, in both cases, f (n) is O(log n) when a = 1.Case II a > 1 : First assume that n = bk , where k is a positiveinteger. It follows that
f (n) = ak f (1) + c(ak − 1)/(a− 1)
= ak [f (1) + c/(a− 1)]− c/(a− 1)
= C1nlogb a + C2,
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 14 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Proof II
Case I a = 1: We have f (n) = ak f (1) +∑k−1
i=0 c .Because n = bk , we have k = logb n. Hence,
f (n) = f (1) + c logb n.
When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k . Because f is increasing, it follows that f (n) ≤ f (bk+1) =f (1) + c(k + 1) = (f (1) + c) + ck ≤ (f (1) + c) + c logb n.Therefore, in both cases, f (n) is O(log n) when a = 1.
Case II a > 1 : First assume that n = bk , where k is a positiveinteger. It follows that
f (n) = ak f (1) + c(ak − 1)/(a− 1)
= ak [f (1) + c/(a− 1)]− c/(a− 1)
= C1nlogb a + C2,
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 14 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Proof II
Case I a = 1: We have f (n) = ak f (1) +∑k−1
i=0 c .Because n = bk , we have k = logb n. Hence,
f (n) = f (1) + c logb n.
When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k . Because f is increasing, it follows that f (n) ≤ f (bk+1) =f (1) + c(k + 1) = (f (1) + c) + ck ≤ (f (1) + c) + c logb n.Therefore, in both cases, f (n) is O(log n) when a = 1.Case II a > 1 : First assume that n = bk , where k is a positiveinteger. It follows that
f (n) = ak f (1) + c(ak − 1)/(a− 1)
= ak [f (1) + c/(a− 1)]− c/(a− 1)
= C1nlogb a + C2,
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 14 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Proof III
Because
ak = alogb n = blogb alogb n
= b(logb n)·(logb a) = blogb nlogb a
= nlogb a,
where C1 = f (1) + c/(a− 1) and C2 = −c/(a− 1).
When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k. Because f is increasing, it follows that
f (n) ≤ f (bk+1) = C1ak+1 + C2
≤ (C1a)alogb n + C2
≤ (C1a)nlogb a + C2
Because k ≤ logb n < k + 1. Hence, we have f (n) is O(nlogb a).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 15 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Proof III
Because
ak = alogb n = blogb alogb n
= b(logb n)·(logb a) = blogb nlogb a
= nlogb a,
where C1 = f (1) + c/(a− 1) and C2 = −c/(a− 1).When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k. Because f is increasing, it follows that
f (n) ≤ f (bk+1) = C1ak+1 + C2
≤ (C1a)alogb n + C2
≤ (C1a)nlogb a + C2
Because k ≤ logb n < k + 1. Hence, we have f (n) is O(nlogb a).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 15 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Proof III
Because
ak = alogb n = blogb alogb n
= b(logb n)·(logb a) = blogb nlogb a
= nlogb a,
where C1 = f (1) + c/(a− 1) and C2 = −c/(a− 1).When n is not a power of b, we have bk < n < bk+1, for a positiveinteger k. Because f is increasing, it follows that
f (n) ≤ f (bk+1) = C1ak+1 + C2
≤ (C1a)alogb n + C2
≤ (C1a)nlogb a + C2
Because k ≤ logb n < k + 1. Hence, we have f (n) is O(nlogb a).MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 15 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Example I
Question: Letf (n) = 5f (n/2) + 3 and f (1) = 7. Find f (2k), wherek is a positive integer. Also, estimate f (n) if f is an increasingfunction.Solution: From the proof of the theorem, with a = 5, b = 2, andc = 3, we see that if n = 2k , then
f (n) = ak [f (1) + c/(a− 1)] + [−c/(a− 1)]
= 5k [7 + (3/4)]− 3/4
= 5k(31/4)− 3/4.
Also, if f (n) is increasing, the above theorem shows that f (n) isO(nlogb a) = O(nlog 5).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 16 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Example II
Binary search
Question: Give a big-O estimate for the number of comparisonsused by a binary search.Solution: We have known that f (n) = f (n/2) + 2 when n is even,where f is the number of comparisons required to perform a binarysearch on a sequence of size n.
Note that a = 1. Hence, from the above theorem, it follows thatf (n) is O(log n).
Search maximum and minimum elements
Question: Give a big-O estimate for the number of comparisonsused to locate the maximum and minimum elements in a sequence.Solution: We have already known that f (n) = 2f (n/2) + 2 witha = 2, b = 2, and c = 2.Hence, it follows that f (n) is O(nlog 2) = O(n).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 17 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Example II
Binary search
Question: Give a big-O estimate for the number of comparisonsused by a binary search.Solution: We have known that f (n) = f (n/2) + 2 when n is even,where f is the number of comparisons required to perform a binarysearch on a sequence of size n.Note that a = 1. Hence, from the above theorem, it follows thatf (n) is O(log n).
Search maximum and minimum elements
Question: Give a big-O estimate for the number of comparisonsused to locate the maximum and minimum elements in a sequence.Solution: We have already known that f (n) = 2f (n/2) + 2 witha = 2, b = 2, and c = 2.Hence, it follows that f (n) is O(nlog 2) = O(n).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 17 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Example II
Binary search
Question: Give a big-O estimate for the number of comparisonsused by a binary search.Solution: We have known that f (n) = f (n/2) + 2 when n is even,where f is the number of comparisons required to perform a binarysearch on a sequence of size n.Note that a = 1. Hence, from the above theorem, it follows thatf (n) is O(log n).
Search maximum and minimum elements
Question: Give a big-O estimate for the number of comparisonsused to locate the maximum and minimum elements in a sequence.Solution: We have already known that f (n) = 2f (n/2) + 2 witha = 2, b = 2, and c = 2.
Hence, it follows that f (n) is O(nlog 2) = O(n).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 17 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Example II
Binary search
Question: Give a big-O estimate for the number of comparisonsused by a binary search.Solution: We have known that f (n) = f (n/2) + 2 when n is even,where f is the number of comparisons required to perform a binarysearch on a sequence of size n.Note that a = 1. Hence, from the above theorem, it follows thatf (n) is O(log n).
Search maximum and minimum elements
Question: Give a big-O estimate for the number of comparisonsused to locate the maximum and minimum elements in a sequence.Solution: We have already known that f (n) = 2f (n/2) + 2 witha = 2, b = 2, and c = 2.Hence, it follows that f (n) is O(nlog 2) = O(n).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 17 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Master theorem
Theorem
Let T be an increasing function that satisfies the recurrence relation
T (n) = aT (n/b) + cnd
whenever n is divisible by b, where a ≥ 1, b is an integer greater than1, and c and d are real numbers with c positive and d nonnegative.Then
T (n) is
O(nd), if a < bd ;O(nd log n), if a = bd ;O(nlogb a), if a > bd ;
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 18 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Master theorem Cont’d
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 19 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Proof of Master theorem
At internal level k of the tree, the work done is
akc(n/bk)d
= cnd(a/bd)k
Therefore
T (n) = calogb n +
logb n−1∑k=0
cnd(a
bd)k
= calogb n + cndlogb n−1∑k=0
(a
bd)k
= cnlogb a + cndlogb n−1∑k=0
(a
bd)k
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 20 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Proof of Master theorem
At internal level k of the tree, the work done is
akc(n/bk)d = cnd(a/bd)k
Therefore
T (n) = calogb n +
logb n−1∑k=0
cnd(a
bd)k
= calogb n + cndlogb n−1∑k=0
(a
bd)k
= cnlogb a + cndlogb n−1∑k=0
(a
bd)k
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 20 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Case I proof
Case I
If a/bd = 1, that is a = bd and d = logb a.
T (n) = cnlogb a + cndlogb n−1∑k=0
(a
bd)k
= cnd + cndlogb n−1∑k=0
1 = cnd + cnd logb n
= O(nd log n)
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 21 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Case II proof
Case II
If a/bd < 1, that is a < bd and d > logb a.
T (n) = cnlogb a + cndlogb n−1∑k=0
(a
bd)k
< cnd + cndlogb n−1∑k=0
(a
bd)k
< cnd + cnd∞∑k=0
(a
bd)k = cnd
(1 +
1
1− a/bd)
= O(nd)
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 22 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Case III proof
Case III
If a/bd > 1, that is a > bd and d < logb a.
T (n) = cnlogb a + cndlogb n−1∑k=0
(a
bd)k = cnd + cnd
(a/bd)logb n − 1
(a/bd)− 1
< cnlogb a + cnd(a/bd)logb n1
(a/bd)− 1
= cnlogb a + cnd(a/bd)logb nΘ(1)
= cnlogb a + cnd(alogb n/bd logb n)Θ(1)
= cnlogb a + cnd(nlogb a/nd)Θ(1) = cnlogb a + cnlogb aΘ(1)
= O(nlogb a)
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 23 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Example I of Master theorem
wherea = 2, b = 2and d = 1,that isa = bd .Therefore,we have
T (n) = O(n logn).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 24 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Example II of Master theorem
where a = 1, b = 2 and d = 1,that is a < bd . Therefore, wehave
T (n) = O(nd) = O(n).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 25 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Example III
Question: Let # comparisons used by the Mergesort to sort a listof n elements be less than M(n), where M(n) = 2M(n/2) + n.Solution: By the master theorem, we find that M(n) is O(n log n).
Question: Let f (n) be # bit operations for multiplying two n-bitintegers, where f (n) = 3f (n/2) + Cn.Solution: Hence, from the master theorem, it follows that f (n) isO(nlog 3). Note that log 3 ≈ 1.6.
Question: Let f (n) be # multiplications and additions required tomultiply two n × n matrices, where f (n) = 7f (n/2) + 15n2/4.Solution: Hence, from the master theorem, it follows that f (n) isO(nlog 7). Note that log 7 ≈ 2.8.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 26 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Example III
Question: Let # comparisons used by the Mergesort to sort a listof n elements be less than M(n), where M(n) = 2M(n/2) + n.Solution: By the master theorem, we find that M(n) is O(n log n).
Question: Let f (n) be # bit operations for multiplying two n-bitintegers, where f (n) = 3f (n/2) + Cn.Solution: Hence, from the master theorem, it follows that f (n) isO(nlog 3). Note that log 3 ≈ 1.6.
Question: Let f (n) be # multiplications and additions required tomultiply two n × n matrices, where f (n) = 7f (n/2) + 15n2/4.Solution: Hence, from the master theorem, it follows that f (n) isO(nlog 7). Note that log 7 ≈ 2.8.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 26 / 43
Divide-and-Conquer Recurrence Relations Master Theorem
Example III
Question: Let # comparisons used by the Mergesort to sort a listof n elements be less than M(n), where M(n) = 2M(n/2) + n.Solution: By the master theorem, we find that M(n) is O(n log n).
Question: Let f (n) be # bit operations for multiplying two n-bitintegers, where f (n) = 3f (n/2) + Cn.Solution: Hence, from the master theorem, it follows that f (n) isO(nlog 3). Note that log 3 ≈ 1.6.
Question: Let f (n) be # multiplications and additions required tomultiply two n × n matrices, where f (n) = 7f (n/2) + 15n2/4.Solution: Hence, from the master theorem, it follows that f (n) isO(nlog 7). Note that log 7 ≈ 2.8.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 26 / 43
Generating Functions
Generating functions
Definition
The generating function for sequence a0, a1, · · · , ak , · · · of real num-bers is the infinite series
G (x) = a0 + a1x + · · ·+ akxk + · · · =
∞∑k=0
akxk .
Examples
{ak} g(x)
ak = 3∑∞
k=0 3xk
ak = k + 1∑∞
k=0(k + 1)xk
ak = 2k∑∞
k=0 2kxk
ak = 1(k = 0, 1, · · · , 5)∑5
k=0 xk = x6−1
x−1
ak = C (m, k)(k = 0, 1, · · · ,m)∑m
k=0 C (m, k)xk = (1 + x)m
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 27 / 43
Generating Functions
Generating functions
Definition
The generating function for sequence a0, a1, · · · , ak , · · · of real num-bers is the infinite series
G (x) = a0 + a1x + · · ·+ akxk + · · · =
∞∑k=0
akxk .
Examples
{ak} g(x)
ak = 3∑∞
k=0 3xk
ak = k + 1∑∞
k=0(k + 1)xk
ak = 2k∑∞
k=0 2kxk
ak = 1(k = 0, 1, · · · , 5)∑5
k=0 xk = x6−1
x−1
ak = C (m, k)(k = 0, 1, · · · ,m)∑m
k=0 C (m, k)xk = (1 + x)m
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 27 / 43
Generating Functions Useful Facts About Power Series
Operations of generating functions
Theorem
Let f (x) =∑∞
k=0 akxk and g(x) =
∑∞k=0 bkx
k . Then
f (x) + g(x) =∞∑k=0
(ak + bk)xk , f (x)g(x) =∞∑k=0
( k∑j=0
ajbk−j)xk
Examples of power series
{ak} g(x)
ak = 1∑∞
k=0 xk = 1
1−x for |x | < 1
ak = ak∑∞
k=0 akxk = 1
1−ax for |ax | < 1
ak = k + 1∑∞
k=0
(∑kj=0 1
)xk = 1
(1−x)2
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 28 / 43
Generating Functions Useful Facts About Power Series
Operations of generating functions
Theorem
Let f (x) =∑∞
k=0 akxk and g(x) =
∑∞k=0 bkx
k . Then
f (x) + g(x) =∞∑k=0
(ak + bk)xk , f (x)g(x) =∞∑k=0
( k∑j=0
ajbk−j)xk
Examples of power series
{ak} g(x)
ak = 1∑∞
k=0 xk = 1
1−x for |x | < 1
ak = ak∑∞
k=0 akxk = 1
1−ax for |ax | < 1
ak = k + 1∑∞
k=0
(∑kj=0 1
)xk = 1
(1−x)2
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 28 / 43
Generating Functions Extended Binomial Coefficient
Extended binomial coefficient
Definition
Let u be a real number and k a nonnegative integer. Then theextended binomial coefficient
(uk
)is defined by(
u
k
)=
{u(u−1)···(u−k+1)
k! , if k > 0;1, if k = 0.
Examples
Question: Find the values of(−2
3
)and
(1/23
).
Solution: (−2
3
)=
(−2)(−3)(−4)
3!= −4;(
1/2
3
)=
(1/2)(1/2− 1)(1/2− 2)
3!=
1
16.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 29 / 43
Generating Functions Extended Binomial Coefficient
Extended binomial coefficient
Definition
Let u be a real number and k a nonnegative integer. Then theextended binomial coefficient
(uk
)is defined by(
u
k
)=
{u(u−1)···(u−k+1)
k! , if k > 0;1, if k = 0.
Examples
Question: Find the values of(−2
3
)and
(1/23
).
Solution: (−2
3
)=
(−2)(−3)(−4)
3!= −4;(
1/2
3
)=
(1/2)(1/2− 1)(1/2− 2)
3!=
1
16.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 29 / 43
Generating Functions Extended Binomial Coefficient
Corollary
Let n and r are two positive integers, the extended binomial coeffi-cient can be expressed as(
−nr
)= (−1)r
(n + r − 1
r
).
Proof: (−nr
)=−n(−n − 1) · · · (−n − r + 1)
r !
=(−1)rn(n + 1) · · · (n + r − 1)
r !
=(−1)r (n + r − 1)!
r !(n − 1)!
= (−1)r(n + r − 1
r
).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 30 / 43
Generating Functions Extended Binomial Coefficient
The extended binomial theorem
Let x be a real number with |x | < 1 and let u be a real number.Then
(1 + x)u =∞∑k=0
(u
k
)xk .
Examples
Find the generating functions for (1+x)−n and (1−x)−n for n ∈ Z+.Solution:
(1 + x)−n =∞∑k=0
(−nk
)xk =
∞∑k=0
(−1)k(n + k − 1
k
)xk .
Replacing x by −x , we find that
(1− x)−n =∞∑k=0
(n + k − 1
k
)xk .
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 31 / 43
Generating Functions Extended Binomial Coefficient
The extended binomial theorem
Let x be a real number with |x | < 1 and let u be a real number.Then
(1 + x)u =∞∑k=0
(u
k
)xk .
Examples
Find the generating functions for (1+x)−n and (1−x)−n for n ∈ Z+.Solution:
(1 + x)−n =∞∑k=0
(−nk
)xk =
∞∑k=0
(−1)k(n + k − 1
k
)xk .
Replacing x by −x , we find that
(1− x)−n =∞∑k=0
(n + k − 1
k
)xk .
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 31 / 43
Generating Functions Counting Problems and Generating Functions
Running example I
Question: Find the number of solutions of e1 + e2 + e3 = 17, wheree1, e2, and e3 are nonnegative integers with 2 ≤ e1 ≤ 5, 3 ≤ e2 ≤ 6,and 4 ≤ e3 ≤ 7.Solution: The number of solutions with the indicated constraints isthe coefficient of x17 in the expansion of
(x2 + x3 + x4 + x5)(x3 + x4 + x5 + x6)(x4 + x5 + x6 + x7).
Note thatWay # cases1: x5 · x6 · x6 12: x5 · x5 · x7 13: x4 · x6 · x7 1
It is not hard to see that the coefficient of x17 in this product is 3.Hence, there are three solutions.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 32 / 43
Generating Functions Counting Problems and Generating Functions
Running example I
Question: Find the number of solutions of e1 + e2 + e3 = 17, wheree1, e2, and e3 are nonnegative integers with 2 ≤ e1 ≤ 5, 3 ≤ e2 ≤ 6,and 4 ≤ e3 ≤ 7.Solution: The number of solutions with the indicated constraints isthe coefficient of x17 in the expansion of
(x2 + x3 + x4 + x5)(x3 + x4 + x5 + x6)(x4 + x5 + x6 + x7).
Note thatWay # cases1: x5 · x6 · x6 12: x5 · x5 · x7 13: x4 · x6 · x7 1
It is not hard to see that the coefficient of x17 in this product is 3.Hence, there are three solutions.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 32 / 43
Generating Functions Counting Problems and Generating Functions
Running example II
Question: In how many different ways can eight identical cookiesbe distributed among three distinct children if each child receives atleast two cookies and no more than four cookies?
Solution: Because each child receives at least two but no more thanfour cookies, for each child there is a factor equal to (x2 + x3 + x4)in the generating function for the sequence {cn}, where cn is thenumber of ways to distribute n cookies. Because there are threechildren, this generating function is
(x2 + x3 + x4)3.
We need the coefficient of x8 in this product.Computation shows that this coefficient equals 6. Hence, there aresix ways to distribute the cookies so that each child receives at leasttwo, but no more than four, cookies.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 33 / 43
Generating Functions Counting Problems and Generating Functions
Running example II
Question: In how many different ways can eight identical cookiesbe distributed among three distinct children if each child receives atleast two cookies and no more than four cookies?Solution: Because each child receives at least two but no more thanfour cookies, for each child there is a factor equal to (x2 + x3 + x4)in the generating function for the sequence {cn}, where cn is thenumber of ways to distribute n cookies.
Because there are threechildren, this generating function is
(x2 + x3 + x4)3.
We need the coefficient of x8 in this product.Computation shows that this coefficient equals 6. Hence, there aresix ways to distribute the cookies so that each child receives at leasttwo, but no more than four, cookies.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 33 / 43
Generating Functions Counting Problems and Generating Functions
Running example II
Question: In how many different ways can eight identical cookiesbe distributed among three distinct children if each child receives atleast two cookies and no more than four cookies?Solution: Because each child receives at least two but no more thanfour cookies, for each child there is a factor equal to (x2 + x3 + x4)in the generating function for the sequence {cn}, where cn is thenumber of ways to distribute n cookies. Because there are threechildren, this generating function is
(x2 + x3 + x4)3.
We need the coefficient of x8 in this product.
Computation shows that this coefficient equals 6. Hence, there aresix ways to distribute the cookies so that each child receives at leasttwo, but no more than four, cookies.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 33 / 43
Generating Functions Counting Problems and Generating Functions
Running example II
Question: In how many different ways can eight identical cookiesbe distributed among three distinct children if each child receives atleast two cookies and no more than four cookies?Solution: Because each child receives at least two but no more thanfour cookies, for each child there is a factor equal to (x2 + x3 + x4)in the generating function for the sequence {cn}, where cn is thenumber of ways to distribute n cookies. Because there are threechildren, this generating function is
(x2 + x3 + x4)3.
We need the coefficient of x8 in this product.Computation shows that this coefficient equals 6. Hence, there aresix ways to distribute the cookies so that each child receives at leasttwo, but no more than four, cookies.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 33 / 43
Generating Functions Counting Problems and Generating Functions
Running example III
Question: Please determine # ways to insert tokens worth 1,2, and$5 into a vending machine that costs r dollars not matter the orders.
Solution: The number of solutions with the indicated constraints isthe coefficient of x r in the expansion of
(1+x+x2 +x3 +· · · )(1+x2 +x4 +x6 +· · · )(1+x5 +x10 +x15 +· · · ).
When the order in which the tokens are inserted matters, the numberof ways to insert exactly n tokens to produce a total of r dollars isthe coefficient of x r in (x + x2 + x5)n,Because any number of tokens may be inserted, the number of waysto produce r dollars using $1, $2, or $5 tokens, when the order inwhich the tokens are inserted matters, is the coefficient of x r in
1 + (x + x2 + x5) + (x + x2 + x5)2 + · · · =1
1− (x + x2 + x5).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 34 / 43
Generating Functions Counting Problems and Generating Functions
Running example III
Question: Please determine # ways to insert tokens worth 1,2, and$5 into a vending machine that costs r dollars not matter the orders.Solution: The number of solutions with the indicated constraints isthe coefficient of x r in the expansion of
(1+x+x2 +x3 +· · · )(1+x2 +x4 +x6 +· · · )(1+x5 +x10 +x15 +· · · ).
When the order in which the tokens are inserted matters, the numberof ways to insert exactly n tokens to produce a total of r dollars isthe coefficient of x r in (x + x2 + x5)n,Because any number of tokens may be inserted, the number of waysto produce r dollars using $1, $2, or $5 tokens, when the order inwhich the tokens are inserted matters, is the coefficient of x r in
1 + (x + x2 + x5) + (x + x2 + x5)2 + · · · =1
1− (x + x2 + x5).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 34 / 43
Generating Functions Counting Problems and Generating Functions
Running example III
Question: Please determine # ways to insert tokens worth 1,2, and$5 into a vending machine that costs r dollars not matter the orders.Solution: The number of solutions with the indicated constraints isthe coefficient of x r in the expansion of
(1+x+x2 +x3 +· · · )(1+x2 +x4 +x6 +· · · )(1+x5 +x10 +x15 +· · · ).
When the order in which the tokens are inserted matters, the numberof ways to insert exactly n tokens to produce a total of r dollars isthe coefficient of x r in (x + x2 + x5)n,
Because any number of tokens may be inserted, the number of waysto produce r dollars using $1, $2, or $5 tokens, when the order inwhich the tokens are inserted matters, is the coefficient of x r in
1 + (x + x2 + x5) + (x + x2 + x5)2 + · · · =1
1− (x + x2 + x5).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 34 / 43
Generating Functions Counting Problems and Generating Functions
Running example III
Question: Please determine # ways to insert tokens worth 1,2, and$5 into a vending machine that costs r dollars not matter the orders.Solution: The number of solutions with the indicated constraints isthe coefficient of x r in the expansion of
(1+x+x2 +x3 +· · · )(1+x2 +x4 +x6 +· · · )(1+x5 +x10 +x15 +· · · ).
When the order in which the tokens are inserted matters, the numberof ways to insert exactly n tokens to produce a total of r dollars isthe coefficient of x r in (x + x2 + x5)n,Because any number of tokens may be inserted, the number of waysto produce r dollars using $1, $2, or $5 tokens, when the order inwhich the tokens are inserted matters, is the coefficient of x r in
1 + (x + x2 + x5) + (x + x2 + x5)2 + · · · =1
1− (x + x2 + x5).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 34 / 43
Generating Functions Counting Problems and Generating Functions
Running example IV
Question: Find # k-combinations of a set with n elements. Assumethat the binomial theorem has already been established.
Solution: Each of the n elements in the set contributes term (1 +x)to the generating function f (x) =
∑nk=0 akx
k . Here f (x) is thegenerating function for {ak}, where ak represents the number of k-combinations of a set with n elements. Hence, f (x) = (1 + x)n
But by the binomial theorem, we have
f (x) =n∑
k=0
(n
k
)xk .
Hence, C (n, k) is the number of k-combinations of a set with nelements.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 35 / 43
Generating Functions Counting Problems and Generating Functions
Running example IV
Question: Find # k-combinations of a set with n elements. Assumethat the binomial theorem has already been established.Solution: Each of the n elements in the set contributes term (1 +x)to the generating function f (x) =
∑nk=0 akx
k . Here f (x) is thegenerating function for {ak}, where ak represents the number of k-combinations of a set with n elements. Hence, f (x) = (1 + x)n
But by the binomial theorem, we have
f (x) =n∑
k=0
(n
k
)xk .
Hence, C (n, k) is the number of k-combinations of a set with nelements.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 35 / 43
Generating Functions Counting Problems and Generating Functions
Running example IV
Question: Find # k-combinations of a set with n elements. Assumethat the binomial theorem has already been established.Solution: Each of the n elements in the set contributes term (1 +x)to the generating function f (x) =
∑nk=0 akx
k . Here f (x) is thegenerating function for {ak}, where ak represents the number of k-combinations of a set with n elements. Hence, f (x) = (1 + x)n
But by the binomial theorem, we have
f (x) =n∑
k=0
(n
k
)xk .
Hence, C (n, k) is the number of k-combinations of a set with nelements.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 35 / 43
Generating Functions Counting Problems and Generating Functions
Running example IV
Question: Find # k-combinations of a set with n elements. Assumethat the binomial theorem has already been established.Solution: Each of the n elements in the set contributes term (1 +x)to the generating function f (x) =
∑nk=0 akx
k . Here f (x) is thegenerating function for {ak}, where ak represents the number of k-combinations of a set with n elements. Hence, f (x) = (1 + x)n
But by the binomial theorem, we have
f (x) =n∑
k=0
(n
k
)xk .
Hence, C (n, k) is the number of k-combinations of a set with nelements.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 35 / 43
Generating Functions Counting Problems and Generating Functions
Running example V
Question: Find # r -combinations from a set with n elements whenrepetition of elements is allowed.
Solution: Each of the n elements in the set contributes term (1 +x + x2 + · · · ) to the generating function f (x) =
∑∞k=0 akx
k . Heref (x) is the generating function for {ak}, where ak represents # r -combinations from a set with n elements with repetition. Hence,f (x) = (1 + x + x2 + · · · )n.As long as |x | < 1, we have 1 + x + x2 + · · · = 1
1−x . Thus f (x) =(1− x)−n.Applying the extended binomial theorem, it follows that
(1− x)−n =∞∑r=0
(−nr
)(−x)r .
Thus, we have(−n
r
)(−1)r = (−1)rC (n + r − 1, r)(−1)r =
(n+r−1r
).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 36 / 43
Generating Functions Counting Problems and Generating Functions
Running example V
Question: Find # r -combinations from a set with n elements whenrepetition of elements is allowed.Solution: Each of the n elements in the set contributes term (1 +x + x2 + · · · ) to the generating function f (x) =
∑∞k=0 akx
k . Heref (x) is the generating function for {ak}, where ak represents # r -combinations from a set with n elements with repetition. Hence,f (x) = (1 + x + x2 + · · · )n.
As long as |x | < 1, we have 1 + x + x2 + · · · = 11−x . Thus f (x) =
(1− x)−n.Applying the extended binomial theorem, it follows that
(1− x)−n =∞∑r=0
(−nr
)(−x)r .
Thus, we have(−n
r
)(−1)r = (−1)rC (n + r − 1, r)(−1)r =
(n+r−1r
).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 36 / 43
Generating Functions Counting Problems and Generating Functions
Running example V
Question: Find # r -combinations from a set with n elements whenrepetition of elements is allowed.Solution: Each of the n elements in the set contributes term (1 +x + x2 + · · · ) to the generating function f (x) =
∑∞k=0 akx
k . Heref (x) is the generating function for {ak}, where ak represents # r -combinations from a set with n elements with repetition. Hence,f (x) = (1 + x + x2 + · · · )n.As long as |x | < 1, we have 1 + x + x2 + · · · = 1
1−x . Thus f (x) =(1− x)−n.
Applying the extended binomial theorem, it follows that
(1− x)−n =∞∑r=0
(−nr
)(−x)r .
Thus, we have(−n
r
)(−1)r = (−1)rC (n + r − 1, r)(−1)r =
(n+r−1r
).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 36 / 43
Generating Functions Counting Problems and Generating Functions
Running example V
Question: Find # r -combinations from a set with n elements whenrepetition of elements is allowed.Solution: Each of the n elements in the set contributes term (1 +x + x2 + · · · ) to the generating function f (x) =
∑∞k=0 akx
k . Heref (x) is the generating function for {ak}, where ak represents # r -combinations from a set with n elements with repetition. Hence,f (x) = (1 + x + x2 + · · · )n.As long as |x | < 1, we have 1 + x + x2 + · · · = 1
1−x . Thus f (x) =(1− x)−n.Applying the extended binomial theorem, it follows that
(1− x)−n =∞∑r=0
(−nr
)(−x)r .
Thus, we have(−n
r
)(−1)r = (−1)rC (n + r − 1, r)(−1)r =
(n+r−1r
).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 36 / 43
Generating Functions Counting Problems and Generating Functions
Running example VI
Question: Find # ways to select r objects of n different kinds if wemust select at least one object of each kind.
Solution: Each of the n elements in the set contributes term (x +x2 + · · · ) to the generating function f (x) =
∑∞k=0 akx
k . Here f (x)is the generating function for {ak}, where ak represents the numberof r objects of n different kinds with at least one object of each kind.Hence, f (x) = (x + x2 + · · · )n.As long as |x | < 1, we have f (x) = xn(1 + x + x2 + · · · )n = xn
(1−x)n .Applying the extended binomial theorem, it follows that
f (x) = xn∞∑r=0
(−nr
)(−x)r =
∞∑t=n
(t − 1
t − n
)x t .
Hence, there are C (r−1, r−n) ways to select r objects of n differentkinds if we must select at least one object of each kind.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 37 / 43
Generating Functions Counting Problems and Generating Functions
Running example VI
Question: Find # ways to select r objects of n different kinds if wemust select at least one object of each kind.Solution: Each of the n elements in the set contributes term (x +x2 + · · · ) to the generating function f (x) =
∑∞k=0 akx
k . Here f (x)is the generating function for {ak}, where ak represents the numberof r objects of n different kinds with at least one object of each kind.Hence, f (x) = (x + x2 + · · · )n.
As long as |x | < 1, we have f (x) = xn(1 + x + x2 + · · · )n = xn
(1−x)n .Applying the extended binomial theorem, it follows that
f (x) = xn∞∑r=0
(−nr
)(−x)r =
∞∑t=n
(t − 1
t − n
)x t .
Hence, there are C (r−1, r−n) ways to select r objects of n differentkinds if we must select at least one object of each kind.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 37 / 43
Generating Functions Counting Problems and Generating Functions
Running example VI
Question: Find # ways to select r objects of n different kinds if wemust select at least one object of each kind.Solution: Each of the n elements in the set contributes term (x +x2 + · · · ) to the generating function f (x) =
∑∞k=0 akx
k . Here f (x)is the generating function for {ak}, where ak represents the numberof r objects of n different kinds with at least one object of each kind.Hence, f (x) = (x + x2 + · · · )n.As long as |x | < 1, we have f (x) = xn(1 + x + x2 + · · · )n = xn
(1−x)n .
Applying the extended binomial theorem, it follows that
f (x) = xn∞∑r=0
(−nr
)(−x)r =
∞∑t=n
(t − 1
t − n
)x t .
Hence, there are C (r−1, r−n) ways to select r objects of n differentkinds if we must select at least one object of each kind.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 37 / 43
Generating Functions Counting Problems and Generating Functions
Running example VI
Question: Find # ways to select r objects of n different kinds if wemust select at least one object of each kind.Solution: Each of the n elements in the set contributes term (x +x2 + · · · ) to the generating function f (x) =
∑∞k=0 akx
k . Here f (x)is the generating function for {ak}, where ak represents the numberof r objects of n different kinds with at least one object of each kind.Hence, f (x) = (x + x2 + · · · )n.As long as |x | < 1, we have f (x) = xn(1 + x + x2 + · · · )n = xn
(1−x)n .Applying the extended binomial theorem, it follows that
f (x) = xn∞∑r=0
(−nr
)(−x)r =
∞∑t=n
(t − 1
t − n
)x t .
Hence, there are C (r−1, r−n) ways to select r objects of n differentkinds if we must select at least one object of each kind.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 37 / 43
Generating Functions Using Generating Functions to Solve Recurrence Relations
Running example VI
Question: Solve recurrence relation ak = 3ak−1 for k = 1, 2, 3, · · ·and initial condition a0 = 2.
Solution: Let f (x) be the generating function for the sequence {ak},that is, f (x) =
∑∞k=0 akx
k . First note that
xf (x) =∞∑k=0
akxk+1 =
∞∑k=1
ak−1xk .
Using the recurrence relation, we see that
f (x)− 3xf (x) =∞∑k=0
akxk − 3
∞∑k=1
ak−1xk
= a0 +∞∑k=1
(ak − 3ak−1)xk = 2.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 38 / 43
Generating Functions Using Generating Functions to Solve Recurrence Relations
Running example VI
Question: Solve recurrence relation ak = 3ak−1 for k = 1, 2, 3, · · ·and initial condition a0 = 2.Solution: Let f (x) be the generating function for the sequence {ak},that is, f (x) =
∑∞k=0 akx
k . First note that
xf (x) =∞∑k=0
akxk+1 =
∞∑k=1
ak−1xk .
Using the recurrence relation, we see that
f (x)− 3xf (x) =∞∑k=0
akxk − 3
∞∑k=1
ak−1xk
= a0 +∞∑k=1
(ak − 3ak−1)xk = 2.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 38 / 43
Generating Functions Using Generating Functions to Solve Recurrence Relations
Running example VI
Question: Solve recurrence relation ak = 3ak−1 for k = 1, 2, 3, · · ·and initial condition a0 = 2.Solution: Let f (x) be the generating function for the sequence {ak},that is, f (x) =
∑∞k=0 akx
k . First note that
xf (x) =∞∑k=0
akxk+1 =
∞∑k=1
ak−1xk .
Using the recurrence relation, we see that
f (x)− 3xf (x) =∞∑k=0
akxk − 3
∞∑k=1
ak−1xk
= a0 +∞∑k=1
(ak − 3ak−1)xk = 2.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 38 / 43
Generating Functions Using Generating Functions to Solve Recurrence Relations
Running example VI Cont’d
Thus,f (x)− 3xf (x) = (1− 3x)f (x) = 2,
i.e.,f (x) = 2/(1− 3x).
Using the identity
1/(1− ax) =∞∑k=0
akxk ,
we havef (x) = 2
∞∑k=0
3kxk .
Consequently,ak = 2 · 3k .
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 39 / 43
Generating Functions Using Generating Functions to Solve Recurrence Relations
Running example VI Cont’d
Thus,f (x)− 3xf (x) = (1− 3x)f (x) = 2,
i.e.,f (x) = 2/(1− 3x).
Using the identity
1/(1− ax) =∞∑k=0
akxk ,
we havef (x) = 2
∞∑k=0
3kxk .
Consequently,ak = 2 · 3k .
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 39 / 43
Generating Functions Using Generating Functions to Solve Recurrence Relations
Running example VII
Question: Suppose that a valid codeword is an n-digit number indecimal notation containing an even number of 0s. Let an denotethe number of valid codewords of length n. Note that a1 = 9, andthe recurrence relation is
an = 8an−1 + 10n−1.
Solution: We extend this sequence by setting a0 = 1, then
f (x)− 1 =∞∑n=1
anxn =
∞∑n=1
(8an−1xn + 10n−1xn)
= 8x∞∑n=1
an−1xn−1 + x
∞∑n=1
10n−1xn−1
= 8x∞∑n=0
anxn + x
∞∑n=0
10nxn = 8xf (x) +x
1− 10x.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 40 / 43
Generating Functions Using Generating Functions to Solve Recurrence Relations
Running example VII
Question: Suppose that a valid codeword is an n-digit number indecimal notation containing an even number of 0s. Let an denotethe number of valid codewords of length n. Note that a1 = 9, andthe recurrence relation is
an = 8an−1 + 10n−1.
Solution: We extend this sequence by setting a0 = 1, then
f (x)− 1 =∞∑n=1
anxn =
∞∑n=1
(8an−1xn + 10n−1xn)
= 8x∞∑n=1
an−1xn−1 + x
∞∑n=1
10n−1xn−1
= 8x∞∑n=0
anxn + x
∞∑n=0
10nxn = 8xf (x) +x
1− 10x.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 40 / 43
Generating Functions Using Generating Functions to Solve Recurrence Relations
Running example VII Cont’d
That is,f (x)− 1 = 8xf (x) +
x
1− 10x
i.e.,
f (x) =1− 9x
(1− 8x)(1− 10x)=
1
2
( 1
1− 8x+
1
1− 10x
).
Furthermore,
f (x) =∞∑k=0
1
2(8k + 10k)xk .
Consequently,an =
1
2(8n + 10n).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 41 / 43
Generating Functions Using Generating Functions to Solve Recurrence Relations
Running example VII Cont’d
That is,f (x)− 1 = 8xf (x) +
x
1− 10x
i.e.,
f (x) =1− 9x
(1− 8x)(1− 10x)=
1
2
( 1
1− 8x+
1
1− 10x
).
Furthermore,
f (x) =∞∑k=0
1
2(8k + 10k)xk .
Consequently,an =
1
2(8n + 10n).
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 41 / 43
Generating Functions Proving Identities via Generating Functions
Running example VIII
Statement: Let n is a positive integer, using generating functionsto show that
n∑k=0
C (n, k)2 = C (2n, n).
Proof: First note that by the binomial theorem C (2n, n) is the co-efficient of xn in (1 + x)2n. However, we also have
(1 + x)2n = [(1 + x)n]2
= [C (n, 0) + C (n, 1)x + C (n, 2)x2 + · · ·+ C (n, n)xn]2
The coefficient of xn in this expression is∑n
k=0 C (n, k)C (n, n−k) =∑nk=0 C (n, k)2.
Because both C (2n, n) and∑n
k=0 C (n, k)2 represent the coefficientof xn in (1 + x)2n, they must be equal.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 42 / 43
Generating Functions Proving Identities via Generating Functions
Running example VIII
Statement: Let n is a positive integer, using generating functionsto show that
n∑k=0
C (n, k)2 = C (2n, n).
Proof: First note that by the binomial theorem C (2n, n) is the co-efficient of xn in (1 + x)2n.
However, we also have
(1 + x)2n = [(1 + x)n]2
= [C (n, 0) + C (n, 1)x + C (n, 2)x2 + · · ·+ C (n, n)xn]2
The coefficient of xn in this expression is∑n
k=0 C (n, k)C (n, n−k) =∑nk=0 C (n, k)2.
Because both C (2n, n) and∑n
k=0 C (n, k)2 represent the coefficientof xn in (1 + x)2n, they must be equal.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 42 / 43
Generating Functions Proving Identities via Generating Functions
Running example VIII
Statement: Let n is a positive integer, using generating functionsto show that
n∑k=0
C (n, k)2 = C (2n, n).
Proof: First note that by the binomial theorem C (2n, n) is the co-efficient of xn in (1 + x)2n. However, we also have
(1 + x)2n = [(1 + x)n]2
= [C (n, 0) + C (n, 1)x + C (n, 2)x2 + · · ·+ C (n, n)xn]2
The coefficient of xn in this expression is∑n
k=0 C (n, k)C (n, n−k) =∑nk=0 C (n, k)2.
Because both C (2n, n) and∑n
k=0 C (n, k)2 represent the coefficientof xn in (1 + x)2n, they must be equal.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 42 / 43
Generating Functions Proving Identities via Generating Functions
Running example VIII
Statement: Let n is a positive integer, using generating functionsto show that
n∑k=0
C (n, k)2 = C (2n, n).
Proof: First note that by the binomial theorem C (2n, n) is the co-efficient of xn in (1 + x)2n. However, we also have
(1 + x)2n = [(1 + x)n]2
= [C (n, 0) + C (n, 1)x + C (n, 2)x2 + · · ·+ C (n, n)xn]2
The coefficient of xn in this expression is∑n
k=0 C (n, k)C (n, n−k) =∑nk=0 C (n, k)2.
Because both C (2n, n) and∑n
k=0 C (n, k)2 represent the coefficientof xn in (1 + x)2n, they must be equal.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 42 / 43
Generating Functions Proving Identities via Generating Functions
Running example VIII
Statement: Let n is a positive integer, using generating functionsto show that
n∑k=0
C (n, k)2 = C (2n, n).
Proof: First note that by the binomial theorem C (2n, n) is the co-efficient of xn in (1 + x)2n. However, we also have
(1 + x)2n = [(1 + x)n]2
= [C (n, 0) + C (n, 1)x + C (n, 2)x2 + · · ·+ C (n, n)xn]2
The coefficient of xn in this expression is∑n
k=0 C (n, k)C (n, n−k) =∑nk=0 C (n, k)2.
Because both C (2n, n) and∑n
k=0 C (n, k)2 represent the coefficientof xn in (1 + x)2n, they must be equal.
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 42 / 43
Take-aways
Take-aways
Divide-and-Conquer Recurrence Relations
Definition and Examples of DCR2
Master Theorem
Generating Functions
Useful Facts About Power SeriesExtended Binomial CoefficientCounting Problems and Generating FunctionsUsing Generating Functions to Solve Recurrence RelationsProving Identities via Generating Functions
MING GAO (DaSE@ECNU) Discrete Mathematics and Its Applications Dec. 5, 2017 43 / 43