mathematical preliminaries - home pages of all faculty at kfupm

Lecture 10: Mathematical Preliminaries

Objective:

Reviewing mathematical concepts and tools that are frequently used in the analysis of algorithms.

Lecture # 10 Slide # 1<BUTTONS AREA: Do not write anything here>

• In this and the next two lectures, our main objective is to give a review of a variety of mathematical concepts and tools that are frequently used in the analysis of algorithms.

• Most of these concepts and tools should be familiar to the student. They are

provided here mainly as a reference and for completeness.

ICS-353 Lecture Template

Topics reviewed in this lecture

• Sets • Tuples and Cartesian Products • Relations and Functions • Proof Methods

- Direct Proof - Indirect Proof - Proof by Contradiction - Proof by Counterexample - Mathematical Induction

• Logarithms • Floor and Ceiling Functions • Factorial and Binomial coefficients • The Pigeonhole Principle


• The topics briefly reviewed in this lecture include the following:

- Sets - Tuples and Cartesian Products - Relations and Functions - Five mathematical Proof methods which include:

1. The Direct Proof method 2. Indirect Proof 3. Proof by Contradiction 4. Proof by Counterexample 5. Mathematical Induction

- Then we will review logarithms - Floor and Ceiling Functions - Factorial and Binomial coefficients, and finally - The Pigeonhole Principle

First we will introduce Sets


Sets

Why sets are important to the algorithm course?

Definition: A set is a collection of distinct elements, which are called members or elements of the set. This collection of elements is treated as a single object.

Two commonly used methods to represent a set:

1. By listing its elements between a pair of curly braces, or 2. By describing its elements between curly braces

Lecture # 1 <BUTTONS AREA: Do not write anything here>

Why sets are important to the algorithm course? We study set concepts because input of an algorithm may be considered as a set drawn from some particular domain like the set of integers.

We will briefly present basic concepts and a few elementary properties of sets used in the design and analysis of algorithms. But first let us define sets.

Definition: A set is a collection of distinct elements, which are called members or elements of the set. This collection of elements is treated as a single object.

The two commonly used methods to represent a set are:

The first method is by listing list elements between a pair of curly braces. The second method is by describing list elements between curly braces


Sets (Cont.)

Examples. Example 1:

• The set of integers between 1 and 100 can be represented as {1, 2, 3, ...,

100}.

• This same set can be represented using the second method as S = {x | 1 ≤ x ≤ 100 and x is an integer}.

• The second method is the only way used to describe uncountable set. Example 2:

S = {x | x is a real number and 0 ≤ x ≤ 1}.


Examples on Sets: Example 1: Using the first method of representation, the set of integers between 1 and 100 can be represented as {1, 2, 3, ..., 100}. On the other hand, this same set can be represented using the second method by specifying some properties as follows: S = {x | 1 ≤ x ≤ 100 and x is an integer}. The expression for S is read “ the set of all elements x such that x is equal or greater than 1 and less than or equal to 100 and x is an integer.” The second method is the only way used to describe uncountable set. Example 2: The set notation S = {x | x is a real number and 0 ≤ x ≤ 1} is used to represent the set of real numbers between 0 and 1.


Sets (Cont.)

Set Properties • A set is finite if it contains n elements for some constant n > 0, and infinite otherwise.

- Example 1 above represents a finite set.

- Examples of infinite sets: • Set of natural numbers {1, 2, …} • Set of integers • Set of rationals and • Set of reals.

• Infinite set is countable if its elements can be listed as the first element, second element,

and so on; otherwise it is called uncountable. The set of real numbers is uncountable.

Lecture # 1 <BUTTONS AREA: Do not write anything here>

Elementary properties of sets • A set is said to be finite if it contains n elements for some constant n > 0, and infinite

otherwise. - The set in Example 1 above is a finite set. - Examples of infinite sets are:

• Set of natural numbers {1, 2, …} • Set of integers • Set of rationals and • Set of reals.

• An infinite set is called countable if its elements can be listed as the first element,

second element, and so on; otherwise it is called uncountable. For example, the set of real numbers is uncountable.


Sets (Cont.)

Set properties (Cont.)

• The empty set is denoted by {} or by φ, and it is a subset of every set. • The cardinality of a finite set A denoted by |A|, is the number of elements in A.

• x ∈ A indicates that x is a member of A. • A set B is a subset of a set A, denoted by B ⊆ A, if each element of B is an element

of A. A is said to be a superset of the set B.

• B ⊂ A denotes that B is a proper subset of A. That is, in addition to B ⊆ A, B ≠ A.

• For any set A, A ⊆ A, and φ ⊆ A.

• If A and B are two sets, then A ⊆ B and B ⊆ A, means A = B.

• The Union of two sets A and B, denoted by A U B, is the set {x | x ∈ A or x ∈ B}


• The empty set is denoted by {} or by φ, and it is a subset of every set. • The cardinality of a finite set A denoted by |A|, is the number of elements in A.

• x ∈ A indicates that x is a member of A. • A set B is a subset of a set A, denoted by B ⊆ A, if each element of B is an

element of A. A is said to be a superset of the set B.

• B ⊂ A denotes that B is a proper subset of A. That is, in addition to B ⊆ A, B ≠ A.

• For any set A, A ⊆ A, and φ ⊆ A.

• If A and B are two sets, then A ⊆ B and B ⊆ A, means A = B.

• The Union of two sets A and B, denoted by A U B, is the set {x | x ∈ A or x ∈ B}


Sets (Cont.)

Set properties (Cont.)

• The intersection of two sets A and B, denoted by A ∩ B, is the set {x | x ∈ A and x ∈ B}.

• The difference of a set A from a set B, denoted by A - B, is the set {x | x ∈ A and x ∉ B}.

• The complement of a set A, denoted by A , is defined as U - A, where U is the universal set containing A, which is usually understood from the context.

• If A, B and C are sets, then A U ( B U C) = (A U B) U C, and A ∩ (B ∩ C) = (A ∩ B) ∩ C.

• Two sets A and B are said to be disjoint if A ∩ B = φ. • The power set of a set A, denoted by P(A), or 2A is the set of all subsets of A. • If |A| = n, then |P (A) | = 2 n. Note that φ ∈P(A) and A ∈P(A). • To describe the number of distinct subsets of cardinality k that a finite set of n

elements have, we use the special notation , read as “number of combinations of

n items taken k at a time, or “n choose k.” Also, the notation C (n, k) is sometimes used. These quantities are called binomial coefficients, as will be discussed later.

kn


• The intersection of two sets A and B, denoted by A ∩ B, is the set {x | x ∈ A and

x ∈ B}. • The difference of a set A from a set B, denoted by A - B, is the set {x | x ∈ A and

x ∉ B}. • The complement of a set A, denoted by A , is defined as U - A, where U is the

universal set containing A, which is usually understood from the context. • If A, B and C are sets, then A U ( B U C) = (A U B) U C, and A ∩ (B ∩ C) = (A ∩

B) ∩ C. • Two sets A and B are said to be disjoint if A ∩ B = φ. • The power set of a set A, denoted by P(A), or 2A is the set of all subsets of A. • If |A| = n, then |P (A) | = 2 n. Note that φ ∈P(A) and A ∈P(A). • To describe the number of distinct subsets of cardinality k that a finite set of n

elements have, we use the special notation , read as “number of combinations

of n items taken k at a time, or “n choose k.” Also, the notation C (n, k) is sometimes used. These quantities are called binomial coefficients, as will be discussed later.

kn


Tuples and the Cartesian Products

• An ordered n-tuple is an ordered collection (a1, a2, …, an) that has a1 as its first element, a2 as its second element,…, and an as its nth element.

• A tuple often do not have the same type. • Short tuples have special names: ordered pairs refer for 2-tuples, triples for 3-tuples,

quadruple for 4-tuples, quintuple for 5-tuples, and so on.

• The Cartesian (or cross) product of two sets A and B, denoted by A * B, is the set of all ordered pairs (a, b) where a ∈ A and b ∈ B. In set notation: A * B = {(a, b) | a ∈A and b ∈B}.

• In general, the Cartesian product of n sets is: A1 * A2 * …* An = {(a1, a2,…, an) | ai ∈Ai, 1 ≤ i ≤ n }.


Tuples and the Cartesian Products

• An ordered n-tuple is an ordered collection (a1, a2, …, an) that has a1 as its first element, a2 as its second element,…, and an as its nth element.

• A tuple often do not have the same type. • Short tuples have special names: ordered pairs refer for 2-tuples, triples for 3-

tuples, quadruple for 4-tuples, quintuple for 5-tuples, and so on.

• The Cartesian (or cross) product of two sets A and B, denoted by A * B, is the set of all ordered pairs (a, b) where a ∈ A and b ∈ B. In set notation: A * B = {(a, b) | a ∈A and b ∈B}.

• In general, the Cartesian product of n sets is: A1 * A2 * …* An = {(a1, a2,…, an) | ai ∈Ai, 1 ≤ i ≤ n }.


Relations and Functions

• Why we study functions and relations? • Elementary properties of relations and functions.

- Let A and B be two nonempty sets, a binary relation, or simply a relation, R from A to B is a set of ordered pairs (a, b) where a ∈A and b∈B. That is R ⊆ A * B.

- If A = B, we say that R is a relation on the set A. - The domain of R written as Dom (R) = {a | for some b ∈B, (a, b) ∈ R}. - The range of R written as Ran (R) = {b | for some a ∈A, (a, b) ∈ R}.

Examples on domain and range S1 = {(2, 5), (3, 3)}, S2 = {(x, y) | x, y are positive integers and x ≤ y} and S3 = {(x, y) | x, y are real numbers and x2 + y2 ≤ 1} then, Dom (S1) = {2, 3}; Ran (S1) = {5, 3); Dom (S2) = Ran (S2) is the set of natural numbers, and Dom (S3) = Ran (S3) is the set of real numbers in the interval [-1 .. 1].


Relations and Functions • Why we study functions and relations? We study relations and functions because

an algorithm can be considered as a function (or constrained relation) that maps each possible input to a specific output.

• Elementary properties of relations and functions used in the design and analysis of algorithms: - Let A and B be two nonempty sets, a binary relation, or simply a relation,

R from A to B is a set of ordered pairs (a, b) where a ∈A and b ∈B. That is R ⊆ A * B.

- If A = B, we say that R is a relation on the set A. - The domain of R written as Dom (R) = {a | for some b ∈B, (a, b) ∈ R}. - The range of R written as Ran (R) = {b | for some a ∈A, (a, b) ∈ R}.

Example on domain and range Assume the three sets S1 = {(2, 5), (3, 3)}, S2 = {(x, y) | x, y are positive integers and x ≤ y} and S3 = {(x, y) | x, y are real numbers and x2 + y2 ≤ 1} then, Dom (S1) = {2, 3}; Ran (S1) = {5, 3); Dom (S2) = Ran (S2) is the set of natural numbers, and Dom (S3) = Ran (S3) is the set of real numbers in the interval [-1 .. 1].


Relations and Functions (Cont.)

• Elementary properties of relations and functions (Cont.) - Let R be a relation on a set A: then R is said to be: reflexive if (a, a) ∈ R ∀ a ∈ A. symmetric if (a, b) ∈ R implies (b, a) ∈R. transitive if (a, b) ∈ R and (b, c) ∈ R implies (a, c) ∈ R. irreflexive if (a, a) ∉ R ∀ a ∈ A asymmetric if (a, b) ∈ R implies (b, a) ∉R antisymmetric if (a, b) ∈ R and (b, a) ∈ R implies a = b.

- A relation that is reflexive, antisymmetric and transitive is called a partial

order. Example on a partial order relation:

Let S1 = {(x, y)| x, y are positive integers and x divides y} S2 = {(x, y) | x, y are integers and x ≤ y} then:

both S1 and S2 are reflexive, antisymmetric and transitive and hence both are partial order.


• Elementary properties of relations and functions (Cont.) - Let R be a relation on a set A: then R is said to be: reflexive if (a, a) ∈ R ∀ a ∈ A. symmetric if (a, b) ∈ R implies (b, a) ∈R. transitive if (a, b) ∈ R and (b, c) ∈ R implies (a, c) ∈ R. irreflexive if (a, a) ∉ R ∀ a ∈ A asymmetric if (a, b) ∈ R implies (b, a) ∉R antisymmetric if (a, b) ∈ R and (b, a) ∈ R implies a = b.

- A relation that is reflexive, antisymmetric and transitive is called a partial

order. Example on a partial order relation:

Let S1 = {(x, y)| x, y are positive integers and x divides y} S2 = {(x, y) | x, y are integers and x ≤ y} then:

both S1 and S2 are reflexive, antisymmetric and transitive and hence both are partial order.



Equivalence Relations • It is a relation that is, reflexive, transitive and symmetric. • If R is a relation on a set A, R partitions A into equivalence classes such that:

- For any Ci, 1 ≤ i ≤ k, if x ∈ Ci and y ∈ Ci, then (x, y) ∈ R. - If x ∈ Ci and y ∈ Ci and i ≠ j, then (x, y) ∉ R.

Example: x ≡ y (mod n) if x - y = k * n for some integer k. That is, x ≡ y (mod n) if both x and y leave the same remainder when divided by n. For instance: 13 ≡ 8 (mod 5) If we define the relation: R = {(x, y) | x, y are integers and x ≡ y (mod n)}, then R is an equivalence relation. R partitions the set of integers into n classes: C0, C1, …, Cn-1 such that x ∈ Ci and y ∈ Ci if and only if x ≡ y (mod n).


Equivalence Relations • An equivalence relation is a relation that is, reflexive, transitive and symmetric. • If R is a relation on a set A, R partitions A into equivalence classes C1, C2,…, Ck

such that any two elements in one equivalence class are related by R. That is: - For any Ci, 1 ≤ i ≤ k, if x ∈ Ci and y ∈ Ci, then (x, y) ∈ R. - If x ∈ Ci and y ∈ Ci and i ≠ j, then (x, y) ∉ R.

Example on Equivalence relation: Let x and y be two integers, and let n be a positive integer, then x and y are said to be congruent modulo n, denoted by x ≡ y (mod n) if x - y = k * n for some integer k. Thus we say that x and y are congruent modulo n if both x and y leave the same remainder when divided by n. For instance: 13 ≡ 8 (mod 5) as 13 – 8 = 1 * 5, or both 13 and 8 both have the same remainder (=3) when divided by 5. If we define the relation: R = {(x, y) | x, y are integers and x ≡ y (mod n)}, then R is an equivalence relation. R partitions the set of integers into n classes: C0, C1, …, Cn-1 such that x ∈ Ci and y ∈ Ci if and only if x ≡ y (mod n).



Functions:

• Definition • We write this as f (x) = y

Examples: 1. The relation {(1, 2), (3, 4), (2, 4) is a function, 2. The relation {(1, 2), (1, 4) is not since for the element 1 in the domain there is more

than on element in the range. 3. The relation {(x, y)} | x, y are positive integers and x = y3} is a function 4. The relation {(x, y) | x is a positive integer, y is integer and x = y2} is not a function. • Important properties of functions


Functions: • A function f is a (binary) relation such that for every element x ∈ Dom (f) there

exists one element y ∈ Ran (f) with (x, y) ∈ f. • Usually we write this as f (x) = y instead of (x, y) ∈ f and says that y is the value

or image of f at x. Examples:

1. The relation {(1, 2), (3, 4), (2, 4) is a function, 2. The relation {(1, 2), (1, 4) is not since for the element 1 in the domain there is

more than on element in the range. 3. The relation {(x, y)} | x, y are positive integers and x = y3} is a function 4. The relation {(x, y) | x is a positive integer, y is integer and x = y2} is not a

function. • Properties of functions

- Let f be a function with Dom (f)=A and Ran (f) ⊆ B for nonempty sets A and B: We say that f is one to one if f (x) = f (y) implies x = y, that is if for no different elements x and y in A, f (x)= f (y).

- f is said to be onto B if Ran (f) = B - f is said to be one to one correspondence between A and B (also called

bijection) if it is both one to one and onto B.


Proof Techniques

1. Direct Proof

2. Indirect Proof

3. Proof by Contradiction

4. Proof by Counterexample

5. Proof by Mathematical Induction


Proof Techniques Mathematical proofs is considered as an essential component and prerequisite to algorithm design and analysis. We will give here a brief review of five proof methods that are commonly used in algorithm design and analysis.

1. Direct Proof 2. Indirect Proof

3. Proof by Contradiction

4. Proof by Counterexample

5. Proof by Mathematical Induction


Notations used in mathematical proofs

• A proposition or assertion. • The negation symbol “¬”. • The implication symbol “ →”.

• The “if and only if” symbol “↔”.

• Thus:

- “P → Q” stands for “P implies Q” or “if P then Q”. - The statement “P ↔ Q”, stands for: “P if and only if Q”. It consists of two parts

(i.e., two implications) each of which should be proven separately: The if part (implication): “P → Q”.

The only if part (implication): “Q → P”.


Notations used in mathematical proofs

The following notation is used in mathematical proof. • A statement that can be either true or false but not both is called a proposition or

assertion. • The symbol “¬” is the negation symbol. • The symbol “ →” is the implication symbol, and “↔” is the “if and only if”

symbol. Thus: - “P → Q” stands for “P implies Q” or “if P then Q”. - The statement “P ↔ Q”, stands for: “P if and only if Q”. That is: “P is true if

and only if Q is true”. It consists of two parts (two implications) each of which should be proven separately: The if part (implication): “P → Q”.

The only if part (implication): “Q → P”.


Direct proof method

We want to prove that if P then Q, that is: “P → Q”. We start the proof by assuming that P is true, and then deduce the truth of Q from the truth of P. Example: If N is an even integer, then prove that N is also even integer. 2

Solution:

Since N is even, N = 2k for some integer k.

So, N2 = (2K)2 = 4K2 = 2 (2K 2).

It follows that N 2 is an even integer.


Direct Proof method We want to prove that if P then Q, that is: “P → Q” We start the proof by assuming that P is true, and then deduce the truth of Q from the truth of P. Example: Use the direct proof method to prove that if N is an even integer, then provthat N 2 is also even integer. Solution: Since N is even, N = 2k for some integer k. So, N2 = (2K)2 = 4K2 = 2 (2K 2).

It follows that N 2 is an even integer.


Indirect proof method

Indirect proof • “if not Q then not P” • “P → Q” is logically equivalent to “if not Q then not P “ • The statement “if it is raining then it is cloudy” is equivalent to the statement “if it is not

cloudy then it is not raining”. Example: Prove the assertion that “if n2 is an even integer, then n is an even integer” Solution: This assertion is logically equivalent to “if n is an odd integer, then n2 is an

odd integer”. Since n is odd then n = 2k+1 for some integer k. Hence, n2 = (2k + 1) 2 = 4k2 + 4k + 1 = 2(2k2 + 2k) + 1.

Which implies that n2 is an odd integer.


Indirect proof method It is some times much easier to prove “if not Q then not P” than using a direct proof for the statement “if P then Q”.

“P → Q” is logically equivalent to the contrapositive implication “if not Q then not P ”, which is also written as “¬Q → ¬P ”. For instance, the statement “if it is raining then it is cloudy” is equivalent to the statement “if it is not cloudy then it is not raining”.

Example: Prove the assertion that “if n 2 is an even integer, then n is an even integer” using indirect proof. Solution: This assertion is logically equivalent to “if n is an odd integer, then n2 is an

odd integer” which is proved using the direct method as follows: Since n is odd then n = 2k+1 for some integer k. Hence, n2 = (2k + 1) 2 = 4k2 + 4k + 1 = 2(2k2 + 2k) + 1.

Which implies that n2 is an odd integer.


Proof by contradiction

• Widely used and extremely powerful. • To prove the statement “if P then Q”:

- Start with the assumption that P is true but Q is false

• Logical reasoning for proof by contradiction.

• Other contradictions may be reached as well.


Proof by contradiction

• Widely used and extremely powerful. Its power comes from making proofs concise and short as will be seen in the next example.

• To prove the statement “if P then Q”: - start with the assumption that P is true but Q is false: If this leads to a contradiction, it implies that our assumption “Q is false”

must be wrong and hence Q must follow from P.

• Logical reasoning for proof by contradiction: - If we know that P → Q is true and Q is false, then P must be false. - So, if we assume at the beginning that P is true, Q is false, and reach the

conclusion that P is false, then we have that P is both true and false. - But P cannot be both true and false - a contradiction. - We conclude our assumption that Q is false is wrong ⇒ Q must be true.

• Other contradictions may be reached as well:

For instance, after assuming that P is true and Q is false, we may reach the conclusion that, say, 1 = -1.


Proof by contradiction (Cont.)

Example. Prove the theorem that: If n balls are distributed into m boxes, then one box must contain at least balls. mn /

Solution: If all boxes have less than balls, then the total number of balls is at most mn /

−

1mnm ≤

−

−

+ 11m

mmnm = n + m –1 – m < n

which is a contradiction. This is an excellent example that shows its power in producing very short proofs.


Example on proof by contradiction: Prove by contradiction the pigeonhole principal theorem that: If n balls are distributed into m boxes, then one box must contain at least balls. mn /

The definition of this principle will be given later in this lecture. Solution: If all boxes have less than balls, then the total number of balls is at most mn /

−

1mnm ≤

−

−

+ 11m

mmnm = n + m –1 – m < n

which is a contradiction. This is an excellent example that shows the power of this proof method in producing very short proofs.


Proof by counterexample method

General Introduction

Example: Consider the function f (n) = n2 + n + 41 defined on the set of nonnegative

integers:

- f (0) = 41, f (1) =43,…, f (39) = 1601 are all primes. - Assertion: f (n) is always a prime number.

- We use Proof by counterexample, to falsify this assertion:

consider f (40) = 1681 = (41)2 which is not a prime. We conclude

that the assertion is false.


Proof by counterexample method

General Introduction: - This method is used to show that a proposition that holds true quite often is

not always true. - Also, it is usually used when we are faced with a problem that requires

proving or disproving a given assertion. In this case, we may start by trying to disprove the assertion with a

counterexample. - In algorithm analysis, this technique is frequently used to show that an

algorithm does not always produce a result with certain properties.

Example: Let f (n) = n2 + n + 41 be a function defined on the set of nonnegative integers:

- f (0) = 41, f (1) =43,…, f (39) = 1601 are all primes. - Assertion: f (n) is always a prime number. - We use Proof by counterexample, to falsify this assertion:

consider f (40) = 1681 = (41)2 which is not a prime. We conclude that the assertion is false.


Proof by mathematical induction

Proof By Mathematical Induction:

• A powerful proof technique. • The truth of a number of statements (propositions) can be inferred from the truth of a

few specific instances.

• Suppose a sequence of statements S1, S2, ..., we want to prove to be true.

• Suppose also that:

1. For some i ≥ 1, we know that S1, S2,…, Si are true. 2. The problem is such that for any n ≥ i, the truths of S1, S2, ... Sn imply the truth

of Sn+1.

• We can then use induction to show that every statement in this sequence is true.


Proof By Mathematical Induction:

• It is a powerful proof technique for proving that a property holds for a sequence of natural numbers n0, n0 + 1, n0 + 2, …. Typically, n0 is taken to be 0 or 1, but it can be any natural number. Also, the sequence n0, n0 + 1, n0 + 2, …can be any linearly ordered set. However, in the majority of proofs by induction, the set of natural numbers is assumed.

•

• In this technique, the truth of a number of statements (propositions) can be inferred from the truth of a few specific instances.

• Suppose we have a sequence of statements S1, S2, ..., we want to prove to be true.

Suppose also that the following holds: 1. For some i ≥ 1, we know that S1, S2,…, Si are true. 2. The problem is such that for any n ≥ i, the truths of S1, S2, ... Sn imply the

truth of Sn+1.

• We can then use induction to show that every statement in this sequence is true.


Proof by mathematical induction (Cont.)

Example Prove that a binary tree of height n has at most 2n leaves: Proof: Let L (n) represent the number of leaves of a binary tree of height n. Show that L (n) ≤ 2n:

Induction Basis: Clearly L (0) = 1 ≤ 20 since a tree of height 0 has at most one leaf. Induction Hypothesis: Assume L (k) ≤ 2k for all k, 1 ≤ k ≤ n. Induction Step: Let T be a tree of height n+1. By induction hypothesis, the number of leaves in the left and right subtrees of the root is ≤ 2n. It follows that the number of leaves in T is ≤ 2n + 2n = 2{n+1}.


Example on Proof by mathematical induction Using the Principle of Mathematical Induction, Prove that a binary tree of height n has at most 2n leaves: Proof: Let L (n) represent the number of leaves of a binary tree of height n. We want to show that L (n) ≤ 2n using mathematical induction:

Induction Basis: Clearly L (0) = 1 ≤ 20 since a tree of height 0 can have no nodes other than the root. That is, it has at most one leaf. Induction Hypothesis: Assume L (k) ≤ 2k for all k, 1 ≤ k ≤ n. Induction Step: Let T be a tree of height n+1. By induction hypothesis, the number of leaves in the left and right subtrees of the root is ≤ 2n. It follows that the number of leaves in T is ≤ 2n + 2n = 2{n+1}. Thus, the property is true for (n +1). Since T is arbitrary, the proof is complete.


Logarithms

• Importance of the logarithm function.

• Definition of the logarithm function

• The base is omitted when it is = 2.

• The natural number e, defined by

... 7182818.2...!3

1!2

1!1

1111lim =++++=

+=

∞→

n

n ne

• loge x is written as ln x , and is called the natural logarithm of x. • ln x is also defined by:

. 11

dtt

xx

∫=ln


Logarithms • The logarithm function, and specifically to the base 2, is used very extensively in

the analysis of many algorithms.

• Definition. Let b > 1, be a positive real number, x a real number, and suppose that for some positive real number y we have: y = bx. Then, x is called the logarithm of y to the base b and is written as x = log . yb

• Since the log to the base 2 is used most often in computational complexity,

therefore, we omit the base when it is = 2. • Another useful base is the natural number e, which is defined by

... 7182818.2...!3

1!2

1!1

1111lim =++++=

+=

∞→

n

n ne

• Generally, loge x is written as ln x , and is called the natural logarithm of x. • ln x is also defined by:

. 1ln1

dtt

xx

∫=


Logarithms (Cont.)

Properties of logarithms • xy = log b x + log b y, and log b (cblog y) = y log b c, if c > 0 . • is a strictly increasing function. blog

• is a one-to-one function. blog

• . 01log =b

• log e ≈ 1.443 and log 10 ≈ 3.32.

• The derivative of ln (x) is 1/x.

• The derivative of log (x) = log (e) / x.

• . ab a

b =log


Properties of logarithms

For any real numbers x and y greater than 0, the following properties of logarithms follow from the definition:

• log b xy = log b x + log b y, and log b (cy) = y log b c, if c > 0 . • is a strictly increasing function, that is, if x > y, then log . blog yx bb log>• is a one-to-one function, that is, if log , then x = y. blog y log b=xb

• . 01log =b

• log e ≈ 1.443. • log 10 ≈ 3.32. • The derivative of ln (x) is 1/x. • The derivative of log (x) = log (e) / x. • . ab a

b =log


Logarithms (Cont.)

Properties of logarithms (Cont.)

• Converting between bases:

bx

xa

ab log

loglog = .

For example:

2lnlnlog xx = and

exx

loglogln = .

• Important identity:

xy bb yx loglog = , x, y > 0.


More properties of logarithms

• To convert from one base to another, we use the following chain rule:

bx

xa

ab log

loglog = .

For example:

2lnlnlog xx = and

exx

loglogln = .

• The following identity can be proved by taking the logarithms of both sides:

xy bb yx loglog = , x, y > 0.


Floor and Ceiling Functions

• Definitions:

- Monotonically increasing function - Monotonically decreasing function.

- Strictly increasing function.

- Strictly decreasing function.

• The floor and ceiling functions to be defined later are monotonically increasing.


Floor and Ceiling Functions

• Definitions:

- A function is monotonically increasing if m ≤ n implies f (m) ≤ f (n). - A function is monotonically decreasing if m ≤ n implies f (m) ≥ f (n).

- A function is strictly increasing if m < n implies f (m) < f (n).

- A function is strictly decreasing if m < n implies f (m) > f (n).

• The floor and ceiling functions to be defined later are monotonically increasing.


Floor and Ceiling Functions (Cont.)

• Let x be any real number, then: - is defined as the greatest integer ≤ x. x- is defined as the smallest integer ≥ x. xExamples:

27.2 =

37.2 =

35.2 −=−

25.2 −=−

• .2/2/ xxx =+• . xx −=−

• . xx −=−• x-1 < ≤ x ≤ < x + 1. x mn /


• Let x be any real number, then:

- The floor of x, denoted by is defined as the greatest integer ≤ x. x- The ceiling of x, denoted by is defined as the smallest integer ≥ x. xExamples:

27.2 = 37.2 =

35.2 −=− 25.2 −=−

Few more important properties are: • .2/2/ xxx =+• . xx −=−

• . xx −=−• x-1 < ≤ x ≤ < x + 1. x mn /


Floor and Ceiling Functions (Cont.)

• Theorem. Let f (x) be a monotonically increasing function such that if f (x) is

integer, then x is integer. Then, )()( xfxf =

and . )()( xfxf = Examples: for any positive integer n and integers a ≠ 0 and b ≠ 0:

- = . nx / nx /- = . nx / nx /- . abnban /// =- . abnban /// =- . 8/2/4/2/2/2/ nnn ==

• If n is not a power of 2, then there is an integer k such that 2k < n < 2 k + 1. In this case, and . nk log= 1log += kn

• The following two inequalities hold: - . nn n 22 log <≤

- nn n ≤< log22


• Theorem. Let f (x) be a monotonically increasing function such that if f (x) is integer, then x is integer. Then,

)()( xfxf =

and . )()( xfxf = Examples: for any positive integer n and integers a ≠ 0 and b ≠ 0:

- = . nx / nx /- = . nx / nx /- . abnban /// =- . abnban /// =- . 8/2/4/2/2/2/ nnn ==

• If n is not a power of 2, then there is an integer k such that 2k < n < 2 k + 1. In this case, and . nk log= 1log += kn

• The following two inequalities hold: - . nn n 22 log <≤

- nn n ≤< log22


Factorial and Binomial coefficients

• General

• Factorials - Permutation of a set of n distinct objects - k-permutation

- When k = n, the quantity becomes

n

nP = 1 * 2 *… * n,

and is called “n factorial” and denoted by n!.

- Useful approximation to n! is Stirling’s formula nenn )/(*2( π . - Using Stirling’s formula, we compute 30! with relative error of about 0.27%

only.


• Factorial and Binomial coefficients We briefly list some of the important combinatorial properties that are frequently

used in the analysis of algorithm. These properties are used with combinatorial problems.

n

kP

In particular, we will limit our review to permutations and combinations, which lead to the definitions of factorials and binomial coefficients.

• Factorials - A permutation of a set of n distinct objects is defined as an ordered

arrangement of these objects. For example, in the 3-elements set {a, b, c} there are 6 permutations: abc, acb, bac, bca, cab, cba.

- We may be also interested in ordered arrangements of some of the elements

of a set. In this case, we use the term k-permutation to refer to an ordered arrangement of k elements of a set. The number of ways to choose k ≤ n objects in an ordered arrangement (also, called the number of permutations of n objects taken k at a time) is:

n (n-1) (n-2) …(n – k +1) and denoted by or P (n, k)


This is not a new slide

Please note that the spoken part below belongs to the previous slide. There was no space over there.


- When k = n, the quantity becomes n

nP = 1 * 2 *… * n,

and is called the number of permutations of n objects. Because of its importance, this quantity is designated as “n factorial” and denoted by n!. By convention, 0! = 1, which gives the following simple recursive definition of n!:

- 0! = 1, n! = n (n – 1) ! if n ≥ 1. - A useful approximation to n! is Stirling’s formula nenn )/(*2( π . - For example, using Stirling’s formula, we compute 30! with relative error of

about 0.27% only.


Binomial coefficients

• Combinations (also, referred to as “ k-combinations “ or “ n choose k”) denoted by C . n

k

Example: the combinations of the four letters a, b, c and d taken three at a time

• This is in fact the unordered selection of 3 elements from the 4-elements set.

• 0, )!(!

!!

≥≥−

== knknk

nkP

Cn

knk

• Other common notations for C are C (n, k) and nk

kn

• Binomial coefficient. • This number is equivalent to the number of k-element subsets in a set of n elements.

Example, Subsets of the set {a, b, c, d} are : {a, b, c}, {a, b, d}, {a, c, d}, {b, c, d}.


Binomial coefficients • Another important concept is the combination of n objects taken k at a time (also,

referred to as “ k-combinations “ or “ n choose k”). It refers to the possible ways to choose k objects out of n objects, disregarding order, and denoted by C . n

k

Example: the combinations of the four letters a, b, c and d taken three at a time are: a b c, a b d, a c d, b c d.

• This is in fact the unordered selection of 3 elements from the 4-elements set. • Since order does not matter here, the combinations of n objects taken k at a time is equal

to the number of permutations of n objects taken k at a time divided by k!. That is

0, )!(!

!!

≥≥−

== knknk

nkP

Cn

knk

• Other common notations for are C (n, k) and nkC

kn

• This number is also called the Binomial coefficient. This name is used because these numbers occur as coefficients in the expansion of powers of binomial expressions such as (x+ y)n.

• This number is equivalent to the number of k-element subsets in a set of n elements. For example, the 3-element subsets of the set {a, b, c, d} are : {a, b, c}, {a, b, d}, {a, c, d}, {b, c, d}.


Binomial coefficients (Cont.)

• Important properties of Binomial coefficients:

- The number of ways to choose k elements out of n elements is equal to the number of ways not to choose n-k elements out of n elements. This implies that:

k

n

− kn

n

n

n

0

n

k

n

−

k

n 1

−

−

1

1

k

n

= , in particular

=

= 1

- The following identity is also important one:

= +


• Some of the important properties of binomial coefficients:

- The # of ways to choose k elements out of n elements is equal to the number of ways not to choose n-k elements out of n elements. This implies that:

k

n

− kn

n

n

n

0

n

k

n

−

k

n 1

−

−

1

1

k

n

= , in particular

=

= 1

This means that the n-combination (= the number of n objects taken n at a time) equals the 0-combination (= the number of combinations of n objects

taken 0 at a time) and equals to 1.

- The following identity is also important one:

= +

That is, the number of combinations of n objects taken k at a time equals to the number of combinations of n - 1 objects taken k at a time + the number of combinations of n - 1 objects taken k - 1 at a time.



• Properties of Binomial coefficients (Cont.)

- Pascal’s Identity. Used in the geometric arrangement of the binomial coefficients in Pascal triangle.

- It is stated as follows. Let n and k be positive integers with n ≥ k. Then

+

k

n 1 =

+

−1k

n

k

n

The nth row in Pascal’s triangle consists of the binomial coefficients , k = 0,

1, ..., n as follows:

kn


• Properties of binomial coefficients (Cont.)

- Pascal’s Identity. It is used in the geometric arrangement of the binomial coefficients in a triangle called Pascal triangle shown later.

- Pascal’s Identity is stated as follows. Let n and k be positive integers with n

≥ k. Then

+

k

n 1 =

+

−1k

n

k

n

The nth row in Pascal’s triangle consists of the binomial coefficients

, k = 0, 1, ..., n as shown in the next slide.

kn




1

00

1 1

11 0

1

1 2 1

22 1

2 02

1 3 3 1

33 2

3 13 0

3

1 4 6 4 1

44 3

4 24 1

4 04

The first 5 rows in Pascal’s Triangle.


1

00

1 1

11 0

1

1 2 1

22 1

2 02

1 3 3 1

33 2

3 13 0

3

1 4 6 4 1

44 3

4 24 1

4 04

The first 5 rows in Pascal’s Triangle.




- Binomial theorem. It gives coefficients of the expansion of powers of binomial expressions.

- Binomial theorem statement: Let n be a positive integer, and x, y be two variables then:

. jjnn

j

n yxjnyx −

=∑

=+ )(

0

= .nnnnn ynnxyn

nyxnyxnxn

+

−++

+

+

−−− 1221

1 ...210


• Properties of binomial coefficients (Cont.)

- A fundamental tool in the analysis of algorithms is the binomial theorem. It gives coefficients of the expansion of powers of binomial expressions, where a binomial expression is simply the sum of two terms, such as x + y.

- The binomial theorem is stated as follows: Let n be a positive integer, and x,y be two variables then:

. jjnn

j

n yxjnyx −

=∑

=+ )(

0

= .nnnnn ynnxyn

nyxnyxnxn

+

−++

+

+

−−− 1221

1 ...210



Examples on Binomial theorem: Example 1: Find the expansion of (x + y)3. Solution: Using the binomial theorem, we get:

(x + y)3 = . jj

jyxj

−

=∑

3

3

0 3

= C (3, 0) x3 + C(3, 1) x2y + C (3, 2) xy2 + C (3, 3) y3 = x3 + 3x2y + 3xy2 + y3

Example 2: What is the coefficient of xy3 in the expansion (x + y)4? Solution: Using the binomial theorem, we have the coefficient as follows:

34 = 4

!1 !3!4

)!34(!3!4

==−


Examples on Binomial theorem: Example 1: Find the expansion of (x + y)3. Solution: Using the binomial theorem, we get:

(x + y)3 = . jj

jyxj

−

=∑

3

3

0 3

= C (3, 0) x3 + C(3, 1) x2y + C (3, 2) xy2 + C (3, 3) y3 = x3 + 3x2y + 3xy2 + y3

Example 2: What is the coefficient of xy3 in the expansion (x + y)4? Solution: Using the binomial theorem, we have the coefficient as follows:

34 = 4

!1 !3!4

)!34(!3!4

==−



If x = 1, then we get:

. )1(0

jn

j

n yjny ∑

=

=+

Making x =1 and y = 1, then we get, j

n

j

njn )1( )11(

0∑

=

=+

that is nnnnn 2...10 =

++

+

This states that the number of all subsets of a set of size n is equal to 2n.

If we let x = -1 and y = 1, then we get:

0)1()1()1(- )1)1((00

=−

=

=+− ∑∑

=

−

=

jn

j

jnjn

j

njn

jn

That is or . 0...110 =

±−

+

−

nnnnn ∑∑

=

n

oddj

n

evenjjn

jn


If we let x = 1 in the Binomial theorem above, then we get the following important identity:

. )1(0

jn

j

n yjny ∑

=

=+

Making x =1 and y = 1, in the above theorem, then we get, j

n

j

njn )1( )11(

0∑

=

=+

that is nnnnn 2...10 =

++

+

In terms of combinations, this identity states that the number of all subsets of a set of size n is equal to 2n, as expected.

If we let x = -1 and y = 1 in the binomial theorem, then we get:

0)1()1()1(- )1)1((00

=−

=

=+− ∑∑

=

−

=

jn

j

jnjn

j

njn

jn

That is , or . 0...110 =

±−

+

−

nnnnn ∑∑

=

n

oddj

n

evenjjn

jn



• From the Binomial theorem above, we have:

(1 + x) 2 = 1 + 2x + x2; (1+x)3 = 1 + 3x + 3x2 + x3 and so on. • Continuing this way indefinitely, we get the elements of the Pascal triangle, where:

Each row after the first is computed from the previous row using the equation

= +

k

n

−

k

n 1

−

−

1

1

k

n

• The Binomial coefficients are the digits of the number (1 + b)n written in base b

notation, if b is sufficiently large (e.g., 10).

Example: Consider base 10, where 11 = (1 + 10), we have: (11)0 = 1, (11)1 = 11, (11)2 = 1+2*10+(10)2 = 121, (11)3 = 1331, (11)4 = 14641 and so on.


• From the Binomial theorem above, we have: (1 + x) 2 = 1 + 2x + x2; (1+x)3 = 1 + 3x + 3x2 + x3 and so on.

• Continuing this way indefinitely, we get the elements of the Pascal triangle

shown previously, where:

Each row after the first is computed from the previous row using the equation

= +

k

n

−

k

n 1

−

−

1

1

k

n

• It is observed that the binomial coefficients are the digits of the number (1 + b)n

written in base b notation, if b is sufficiently large (e.g., 10).

Example: Consider base 10, where 11 = (1 + 10), we have: (11)0 = 1, (11)1 = 11, (11)2 = 1+2*10+(10)2 = 121, (11)3 = 1331, (11)4 = 14641 and so on., where the digits 1,4,6,4,1 in the last number for example are the entries in the fifth row of Pascal’s triangle!!!


The Pigeonhole Principle

• An easy, intuitive and extremely powerful. • States that: If k + 1 or more objects are placed into k boxes, then there is at least one

box containing two or more of the objects.

• Theorem: If n objects are distributed into m boxes, then: - One box contain at least objects and one box must contain at most

objects. mn /

mn /Example: Suppose that G = (V, E) be a connected undirected graph with m vertices. Also, let p be a path in G that visits n > m vertices. Using the Pigeonhole Principle, we show that p must contain a cycle.

Solution: Since the ≥ 2, there is at least one vertex, say v that is visited by p mn / more than once. Thus, the portion of the path that starts and ends at v constitutes a cycle.


The Pigeonhole Principle (also called: the Dirichlet drawer principle) • It is an easy, intuitive and extremely powerful principle and is essential in the

analysis of algorithms. In the 19th century, the German mathematician Dirichlet used this principle in his work.

• The Pigeonhole Principle states that: If k + 1 or more objects are placed into k boxes, then there is at least one box containing two or more of the objects.

• Theorem: If n objects are distributed into m boxes, then:

- One box contain at least objects and one box must contain at most objects.

mn /

mn /Example on the Pigeonhole Principle: Suppose that G = (V, E) be a connected undirected graph with m vertices. Also, let p be a path in G that visits n > m vertices. Using the Pigeonhole Principle, we show that p must contain a cycle.

Solution: Since the ≥ 2, there is at least one vertex, say v that is visited mn /

by p more than once. Thus, the portion of the path that starts and ends at v constitutes a cycle.


mathematical preliminaries - home pages of all faculty at kfupm

Documents