discrete mathematics1 znj/dm2017lcs.ios.ac.cn/~znj/dm2017/lecture-notes-chapter4 (2).pdf · 1.2...

Discrete Mathematics1

http://lcs.ios.ac.cn/˜znj/DM2017

Naijun Zhan

April 5, 2017

1Special thanks to Profs Hanpin Wang (PKU) and Lijun Zhang (ISCAS) fortheir courtesy of the slides on this course.

Contents

1. The Foundations: Logic and Proofs

2. Basic Structures: Sets, Functions, Sequences, Sum-s, and Matrices

3. Algorithms

4. Number Theory and Cryptography

5. Induction and Recursion

6. Counting

7. Discrete Probability

8. Advanced Counting Techniques

9. Relations

10. Graphs

11. Trees

12. Boolean Algebra

13. Modeling Computation

1

Chapter 1

The Foundations: Logic andProofs

Logic in Computer Science

During the past fifty years there has been extensive, continuous,and growing interaction between logic and computer science. Inmany respects, logic provides computer science with both a u-nifying foundational framework and a tool for modeling compu-tational systems. In fact, logic has been called the calculus ofcomputer science. The argument is that logic plays a fundamen-tal role in computer science, similar to that played by calculusin the physical sciences and traditional engineering disciplines.Indeed, logic plays an important role in areas of computer sci-ence as disparate as machine architecture, computer-aided de-sign, programming languages, databases, artificial intelligence,algorithms, and computability and complexity. Moshe Vardi

2

• The origins of logic can be dated back to Aristotle’s time.

• The birth of mathematical logic:

– Leibnitz’s idea

– Russell paradox

– Hilbert’s plan

– Three schools of modern logic:

logicism (Frege, Russell, Whitehead)

formalism (Hilbert)

intuitionism (Brouwer)

• One of the central problem for logicians is that: “why isthis proof correct/incorrect?”

• Boolean algebra owes to George Boole.

• Now, we are interested in: “is the program correct?”

3

1.1 Propositional Logic – An Appetizer

Definition 1.1.1 (Proposition). A proposition is a declarativesentence that is either true or false, but not both.

Definition 1.1.2 (Propositional Logic). Fix a countable propo-sition set AP. Syntax of propositional formulas in BNF (Backus-Naur form) is given by:

ϕ ::= p ∈ AP | ¬ϕ | ϕ ∧ ϕ

Accordingly,

• Atomic proposition p ∈ AP is a formula.

• Compound formulas: ¬ϕ (negation) and ϕ ∧ ψ (conjunc-tion), provided that ϕ and ψ are formulas.

4

For p ∈ AP , negation and conjunction, we can construct thetruth tables. We define the following derived operators:

• Disjunction: ϕ ∨ ψ := ¬(¬ϕ ∧ ¬ψ)

• Implication: ϕ→ ψ := ¬ϕ ∨ ψ

• Bi-implication: ϕ↔ ψ := (ϕ→ ψ) ∧ (ψ → ϕ)

• Exclusive Or: ϕ⊕ ψ := (ϕ ∨ ψ) ∧ (¬(ϕ ∧ ψ))

Definition 1.1.3 (Precedence of Logical Operators). Operators¬,∧,∨,→,↔ have precedence 1, 2, 3, 4, 5, respectively.

5

Definition 1.1.4 (Logic and Bit Operators).

• A bit is a symbol with possible values 0 and 1. A Booleanvariable is a variable with value true or false.

• Computer bit operations correspond to logic connectives:OR, AND, XOR in various programming languages corre-spond to ∨,∧,⊕, respectively.

• A bit string is a sequence of zero or more bits. The lengthof this string is the number of bits in the string.

• Bitwise OR, bitwise AND and bitwise XOR of two stringsof the same length are the strings that have as their bits theOR, AND and XOR of the corresponding bits in the twostrings, respectively.

6

1.2 Applications of Propositional Logic

Definition 1.2.1 (Applications).

• System Specifications: The automated reply cannot be sentwhen the file system is full.

• Boolean Searches: (one | two) - (three)

• Logic Puzzles. Knights always tell the truth, and the op-posite knaves always lie. A says: ”B is a knight”. B says”The two of us are opposite types”

Definition 1.2.2 (Logic Circuit).

• Propositional logic can be applied to the design of computerhardware. Claude Shannon

• A logic circuit receives input signals p1, p2, . . . , pn and pro-duces an output s. Complicated digital circuits are con-structed from three basic circuits, called gates.

• Build a digital circuit producing (p∨¬r)∧ (¬p∨ (q ∨¬r)).

7

1.3 Propositional Equivalences

A formula ϕ is called a

• tautology if it is always true, no matter what the truthvalues of the propositional variables are;

• contradiction if it is always false;

• contingency if it is neither a tautology nor a contradiction.

Moreover, ϕ is satisfiable if it is either a tautology or a contin-gency, unsatisfiable if it is a contradiction.

Formulas ϕ and ψ are called logically equivalent if ϕ ↔ ψ isa tautology. This is denoted by ϕ ≡ ψ.

Definition 1.3.1 (Logical Equivalence). Show the following log-ical equivalences:

1. Identity laws:

ϕ ∧T ≡ ϕ, ϕ ∨ F ≡ ϕ

2. Dominations laws

ϕ ∨T ≡ T, ϕ ∧ F ≡ F

8

3. Idempotent laws

ϕ ∨ ϕ ≡ ϕ, ϕ ∧ ϕ ≡ ϕ

4. Double negation law

¬(¬ϕ) ≡ ϕ

5. Commutative laws

ϕ ∨ ψ ≡ ψ ∨ ϕ, ϕ ∧ ψ ≡ ψ ∧ ϕ

6. Associative laws

(ϕ1∨ϕ2)∨ϕ3 ≡ ϕ1∨(ϕ2∨ϕ3), (ϕ1∧ϕ2)∧ϕ3 ≡ ϕ1∧(ϕ2∧ϕ3)

7. Distributive laws

ϕ1 ∨ (ϕ2 ∧ ϕ3) ≡ (ϕ1 ∨ ϕ2) ∧ (ϕ1 ∨ ϕ3),

ϕ1 ∧ (ϕ2 ∨ ϕ3) ≡ (ϕ1 ∧ ϕ2) ∨ (ϕ1 ∧ ϕ3)

8. De Morgan’s laws

¬(ϕ ∧ ψ) ≡ ¬ϕ ∨ ¬ψ,¬(ϕ ∨ ψ) ≡ ¬ϕ ∧ ¬ψ

9. Absorption laws

ϕ ∨ (ϕ ∧ ψ) ≡ ϕ, ϕ ∧ (ϕ ∨ ψ) ≡ ϕ

10. Negation laws

ϕ ∨ ¬ϕ ≡ T, ϕ ∧ ¬ϕ ≡ F

9

Definition 1.3.2 (Logical Equivalence). Logical equivalencesinvolving conditional statements:

1. ϕ→ ψ ≡ ¬ϕ ∨ ψ

2. ϕ→ ψ ≡ ¬ψ → ¬ϕ

3. ϕ ∨ ψ ≡ ¬ϕ→ ψ

4. ϕ ∧ ψ ≡ ¬(ϕ→ ¬ψ)

5. ϕ ∧ ¬ψ ≡ ¬(ϕ→ ψ)

6. (ϕ1 → ϕ2) ∧ (ϕ1 → ϕ3) ≡ ϕ1 → (ϕ2 ∧ ϕ3)

7. (ϕ1 → ϕ3) ∧ (ϕ2 → ϕ3) ≡ (ϕ1 ∨ ϕ2)→ ϕ3

8. (ϕ1 → ϕ2) ∨ (ϕ1 → ϕ3) ≡ ϕ1 → (ϕ2 ∨ ϕ3)

9. (ϕ1 → ϕ3) ∨ (ϕ2 → ϕ3) ≡ (ϕ1 ∧ ϕ2)→ ϕ3

10. ϕ↔ ψ ≡ ¬ϕ↔ ¬ψ

11. ϕ↔ ψ ≡ (ϕ ∧ ψ) ∨ (¬ϕ ∧ ¬ψ)

12. ¬(ϕ↔ ψ) ≡ ϕ↔ ¬ψ

10

1.4 Induction and Recursion

PRINCIPLE OF MATHEMATICAL INDUCTION To prove thatP (n) is true for all positive integers n, where P (n) is a proposi-tional function, we complete two steps:

• BASIS STEP: We verify that P (1) is true.

• INDUCTIVE STEP: We show that the conditional state-ment P (k)→ P (k + 1) is true for all positive integers k.

Expressed as a rule of inference for first-order logic, this prooftechnique can be stated as:

Φ := (P (1) ∧ ∀k.(P (k)→ P (k + 1)))→ ∀n.P (n)

Exercise 1.4.1. Prove 1 + 2 + 22 + . . .+ 2n = 2n+1 − 1.

11

STRONG INDUCTION (Second principle of mathematical induc-

tion) To prove that P (n) is true for all positive integers n,where P (n) is a propositional function, we complete two steps:

• BASIS STEP: We verify that the proposition P (1) is true.

• INDUCTIVE STEP: We show that the conditional state-ment (P (1) ∧ P (2) ∧ . . . ∧ P (k))→ P (k + 1) is true for allpositive integers k.

Expressed as a rule of inference for first-order logic, this prooftechnique can be stated as:

Ψ := (P (1) ∧ ∀k.(∧ki=1P (i)→ P (k + 1)))→ ∀n.P (n)

Exercise 1.4.2. Prove that if n is a natural number greater than1, then n can be written as the product of primes.

12

Recursively Defined Sets and Structures and StructuralInduction

Definition 1.4.3 (Strings). The set Σ∗ of strings over the al-phabet Σ is defined recursively by

• BASIS STEP: λ ∈ Σ∗ (where λ is the empty string con-taining no symbols).

• RECURSIVE STEP: If w ∈ Σ∗ and x ∈ Σ, then wx ∈ Σ∗.

Definition 1.4.4. We define the set of well-formed formulasin propositional logic, denoted by L, from alphabet Σ := AP ∪¬,→, (, ).

• BASIS STEP: each p ∈ AP is a well-formed formula.

• RECURSIVE STEP: If ϕ and ψ are well-formed formu-las, i.e., ϕ, ψ ∈ L, then (¬ϕ), (ϕ → ψ) are well-formedformulas.

Thus, the set of well-formed formulas is a subset of L ⊆ Σ∗.

13

STRUCTURAL INDUCTION A proof by structural in-duction consists of two parts.

• BASIS STEP: Show that the result holds for all elementsspecified in the basis step.

• RECURSIVE STEP: Show that if the statement is true foreach of the elements used to construct new elements in therecursive step of the definition, the result holds for thesenew elements.

Remark: The validity of structural induction follows from theprinciple of mathematical induction for the nonnegative integers.

Exercise 1.4.5. Show that every well-formed formula for com-pound propositions contains an equal number of left and rightparentheses.

14

1.5 Normal Forms

Negation Normal Form

• Literal: An atomic proposition p or its negation ¬p;

• Negation Normal Form (NNF): A formula built upwith “∧”, “∨”, and literals.

• Using repeated DeMorgan and Double Negation, we cantransform any formula into a formula with Negation NormalForm.

• Example:

¬((A ∨B) ∧ ¬C) ↔ (DeMorgan)¬(A ∨B) ∨ ¬¬C ↔ (Double Neg, DeMorgan)(¬A ∧ ¬B) ∨ C

Disjunction Normal Form

• Disjunction Normal Form (DNF): A generalized dis-junction of generalized conjunctions of literals.

• Using repeated distribution of ∧ over ∨, any NNF formulacan be rewritten in DNF (exercise).

• Example:

(A ∨B) ∧ (C ∨D) ↔ (Distribution)

[(A ∨B) ∧ C] ∨ [(A ∨B) ∧D] ↔ (Distribution)(A ∧ C) ∨ (B ∧ C) ∨ (A ∧D) ∨ (B ∧D)

15

Conjunction Normal Form

• Conjunction Normal Form (CNF): A generalized con-junction of generalized disjunctions of literals.

• Using repeated distribution of ∨ over ∧, any NNF formulacan be rewritten in CNF (exercise).

• Example:

(A ∧B) ∨ (C ∧D) ↔ (Distribution)

[(A ∧B) ∨ C] ∧ [(A ∧B) ∨D] ↔ (Distribution)(A ∨ C) ∧ (B ∨ C) ∧ (A ∨D) ∧ (B ∨D)

Truth table

16

Expressive completeness

• What are the truth tables for (p ∧ q) ∨ r and p ∧ (q ∨ r)?

• Truth table for n-ary Boolean function.

• A set of logical connectives is called functionally com-plete if any n-ary Boolean function is definable with it,e.g. ¬,∧, ¬,∨.

• How about ¬,→?

Exercise. Exercise 49 of page 16, Exercise 15 of page 23,Exercise 39 of page 35, Exercise 45 of page 36.

17

1.6 Propositional Logic and Deduction Sys-

tems: a Sound and Complete Axiomati-

zation

This section considers a complete axiomatization system suchthat, a formula is a tautology if and only if it can be derived bymeans of the axioms and the deduction rules of the system.

1.6.1 Syntax and Semantics of Propositional Logic

Fix a countable proposition set AP , then formulas of proposi-tional logic are defined by:

Definition 1.6.1 (Syntax). Syntax of propositional formulas inBNF (Backus-Naur form) is given by:

ϕ ::= p ∈ AP | ¬ϕ | ϕ→ ϕ

It generates recursively the set of well-formed formulas, de-noted by L:

• Atomic formula: p ∈ AP implies p ∈ L.

• Compound formulas: (¬ϕ) and (ϕ → ψ), provided thatϕ, ψ ∈ L.

18

We omit parentheses if it is clear from the context.Semantics of a formula ϕ is given w.r.t. an assignment σ ∈

2AP , which is a subset of AP . Intuitively, it assigns true (or,T) to propositions belonging to it, and assigns false (or, F) toothers. Thus, it can also be viewed as a function from AP toT,F.

Definition 1.6.2 (Semantics). Inductively, we may define therelation ⊆ 2AP × L as follows:

• σ p iff p ∈ σ.

• σ ¬ϕ iff not σ ϕ (denoted by σ 6 ϕ).

• σ ϕ→ ψ iff either σ 6 ϕ or σ ψ.

where (σ, ϕ) ∈ is denoted as σ ϕ.

The formula ϕ is called a tautology if σ ϕ for all assignment,it is satisfiable if σ ϕ for some assignment.

1.6.2 The Axiom System: the Hilbert’s System

Definition 1.6.3 (Axioms). 1. ϕ→ (ψ → ϕ).

2. (ϕ→ (ψ → η))→ ((ϕ→ ψ)→ (ϕ→ η)).

19

3. (¬ϕ→ ¬ψ)→ (ψ → ϕ).

Definition 1.6.4 (MP Rule). 1.ϕ→ ψ ϕ

ψ

Given a formula set Γ, a deductive sequence of ϕ from Γ is asequence

ϕ0, ϕ1, . . . , ϕn = ϕ

where each ϕi should be one of the following cases:

1. ϕi ∈ Γ.

2. ϕi is an instance of some axiom.

3. There exists some j, k < i, such that ϕk = ϕj → ϕi.

And, we denote by Γ ` ϕ if there exists such deductive sequence.We write Γ, ψ ` ϕ for Γ ∪ ψ ` ϕ.

20

1.6.3 The Axiom System: Soundness

For a formula set Γ and an assignment σ, the satisfaction relation is defined by: σ Γ iff σ ϕ for every ϕ ∈ Γ.

Observe σ ∅ always holds. We say ϕ is a logical consequentof Γ, denoted as Γ |= ϕ, if σ Γ implies σ ϕ for eachassignment σ.

Thus, ϕ is a tautology if ϕ is the logical consequent of ∅,denoted as |= ϕ.

Theorem 1.6.5 (Soundness). With Hilbert’s axiom system, wehave that Γ ` ϕ implies Γ |= ϕ.

Proof. By induction of the length of deductive sequence of Γ `ϕ.

Corollary 1.6.6. If ` ϕ, then |= ϕ.

1.6.4 The Axiom System: Completeness

With Hilbert’s axiom system, we have the following elementaryproperties:

(Fin) If Γ ` ϕ, then there exists some finite subset Γ′ of Γ, suchthat Γ′ ` ϕ.

(∈) If ϕ ∈ Γ, then Γ ` ϕ.

(∈+) If Γ ` ϕ and Γ ⊆ Γ′ then Γ′ ` ϕ.

(MP) If Γ1 ` ϕ and Γ2 ` ϕ→ ψ, and Γ1,Γ2 ⊆ Γ, then Γ ` ψ.

21

1.6.5 The Axiom System: Examples of Theorems

Example 1.6.7. (Ide): ` ϕ→ ϕ

Solution. 1. ϕ→ (ϕ→ ϕ)

2. ϕ→ ((ϕ→ ϕ)→ ϕ)

3. (ϕ→ ((ϕ→ ϕ)→ ϕ))→ ((ϕ→ (ϕ→ ϕ))→ (ϕ→ ϕ))

4. (ϕ→ (ϕ→ ϕ))→ (ϕ→ ϕ)

5. ϕ→ ϕ

Example 1.6.8. (→−): If Γ ` ϕ→ ψ then Γ, ϕ ` ψ.

Solution. A simple application of MP and (∈).

Example 1.6.9. (→+): If Γ, ϕ ` ψ then Γ ` ϕ→ ψ.

Solution. By induction of the deductive sequence of Γ, ϕ ` ψ.

Example 1.6.10. (τ): If Γ ` ϕ → ψ and Γ ` ψ → η, thenΓ ` ϕ→ η.

Solution. By (→−), (→+) and (∈+).

Example 1.6.11. (Abs): ` ¬ϕ→ (ϕ→ ψ).

Solution. 1. ` ¬ϕ→ (¬ψ → ¬ϕ)

2. ` (¬ψ → ¬ϕ)→ (ϕ→ ψ)

3. ` ¬ϕ→ (ϕ→ ψ)

Example 1.6.12. (Abs’): ` ϕ→ (¬ϕ→ ψ)

Example 1.6.13. (¬w): ¬ϕ→ ϕ ` ϕ

Solution. 1. ¬ϕ→ ϕ ` ¬ϕ→ ϕ

22

2. ` ¬ϕ→ (ϕ→ ¬(¬ϕ→ ϕ))

3. ` (¬ϕ → (ϕ → ¬(¬ϕ → ϕ))) → ((¬ϕ → ϕ) → (¬ϕ →¬(¬ϕ→ ϕ)))

4. ¬ϕ→ ϕ ` ¬ϕ→ ¬(¬ϕ→ ϕ)

5. ` (¬ϕ→ ¬(¬ϕ→ ϕ))→ ((¬ϕ→ ϕ)→ ϕ)

6. ¬ϕ→ ϕ ` (¬ϕ→ ϕ)→ ϕ

7. ¬ϕ→ ϕ ` ϕ

Example 1.6.14. (¬¬−): ¬¬ϕ ` ϕ

Solution. 1. ` ¬¬ϕ→ (¬ϕ→ ϕ)

2. ` (¬ϕ→ ϕ)→ ϕ

3. ` ¬¬ϕ→ ϕ

4. ¬¬ϕ ` ϕ

Example 1.6.15. (¬s): ϕ→ ¬ϕ ` ¬ϕ

Solution. 1. ¬¬ϕ ` ϕ

2. ϕ→ ¬ϕ ` ϕ→ ¬ϕ

3. ϕ→ ¬ϕ,¬¬ϕ ` ϕ

4. ϕ→ ¬ϕ ` ¬¬ϕ→ ¬ϕ

5. ` (¬¬ϕ→ ¬ϕ)→ ¬ϕ

6. ϕ→ ¬ϕ ` ¬ϕ

Example 1.6.16. (¬¬+): ϕ ` ¬¬ϕ

23

Solution. 1. ` ϕ→ (¬ϕ→ ¬¬ϕ)

2. ` (¬ϕ→ ¬¬ϕ)→ ¬¬ϕ

3. ` ϕ→ ¬¬ϕ

4. ϕ ` ¬¬ϕExample 1.6.17. (R0) ϕ→ ψ ` ¬ψ → ¬ϕ

(R1) ϕ→ ¬ψ ` ψ → ¬ϕ

(R2) ¬ϕ→ ψ ` ¬ψ → ϕ

(R3) ¬ϕ→ ¬ψ ` ψ → ϕ

Solution. 1. ϕ→ ψ,¬¬ϕ ` ϕ

2. ϕ→ ψ,¬¬ϕ ` ψ

3. ` ψ → ¬¬ψ

4. ` (¬¬ϕ→ ¬¬ψ)→ (¬ψ → ¬ϕ)

5. ϕ→ ψ ` ¬ψ → ¬ϕDefinition 1.6.18 (Consistency). We say a formula set Γ isconsistent, iff there is some ϕ such that Γ 6` ϕ. Moreover, wesay ϕ is consistent w.r.t. Γ iff Γ ∪ ϕ is consistent.

Note that we have the theorem ¬ϕ, ϕ ` ψ and hence, Γ is

24

consistent iff for each ϕ, either Γ 6` ϕ or Γ 6` ¬ϕ.Further, ϕ is consistent w.r.t. Γ iff Γ 6` ¬ϕ. Suppose that

Γ 6` ¬ϕ and Γ ∪ ϕ is inconsistent, then we have Γ, ϕ ` ¬ϕhence Γ ` ϕ → ¬ϕ. Recall that we have ϕ → ¬ϕ ` ¬ϕ, andthis implies Γ ` ¬ϕ, contradiction!

Lemma 1.6.19. If the formula set Γ is inconsistent, then it hassome finite inconsistent subset ∆.

Theorem 1.6.20. Γ is consistent iff Γ is satisfiable.

Proof sketch. The “if” direction is easy: suppose that σ Γ butΓ ` ϕ and Γ ` ¬ϕ, then σ ϕ and σ ¬ϕ, contradiction.

For the “only if” direction, let us enumerate all propositionalformulas as following (note the cardinality of all such formulasis ℵ0):

ϕ0, ϕ1, . . . , ϕn, . . .

Let Γ0 = Γ and

Γi+1 =

Γi ∪ ϕi if Γi 6` ¬ϕiΓi ∪ ¬ϕi otherwise

and finally let Γ∗ = limi→∞

Γi.

The formula set Γ∗ has the following properties:

1. Each Γi is consistent, and Γ∗ is also consistent.

2. Γ∗ is a maximal set, i.e., for each formula ϕ, either ϕ ∈ Γ∗

or ¬ϕ ∈ Γ∗.

3. For each formula ϕ, we have Γ∗ |= ϕ iff ϕ ∈ Γ∗.

Then we have σ Γ∗, where σ = Γ∗ ∩ AP .

25

Theorem 1.6.21 (Completeness). If Γ |= ϕ, then Γ ` ϕ.

Proof. Assume by contradiction that Γ 6` ϕ, then there is anassignment σ such that σ Γ ∪ ¬ϕ. However, this impliesthat σ Γ and σ 6 ϕ, which violates the assumption Γ |= ϕ.

Corollary 1.6.22. |= ϕ implies that ` ϕ.

The Axiom System: Compactness

Theorem 1.6.23. Given a formula set Γ, we have

1. Γ is consistent iff each of its finite subsets is consistent;

2. Γ is satisfiable iff each of its finite subsets is satisfiable.

Proof. 1. The first property has been proven (see the previouslemma).

2. With the aforementioned theorem: for propositional logic,a set is satisfiable iff it is consistent.

Rules of Inference for Propositional Logic (cf. page 72):

26

Exercise 1.6.24. Show, by applying the rules of the deductionsystem presented in Section 1.6, the following statements:

1. ` (ϕ→ ψ)→ ((¬ϕ→ ¬ψ)→ (ψ → ϕ))

2. ` ((ϕ → (ψ → η)) → (ϕ → ψ)) → ((ϕ → (ψ → η)) →(ϕ→ η))

3. ` (ϕ→ (ϕ→ ψ))→ (ϕ→ ψ)

4. ` ϕ→ (ψ → (ϕ→ ψ))

5. ϕ→ ψ,¬(ψ → η)→ ¬ϕ ` ϕ→ η

6. ϕ→ (ψ → η) ` ψ → (ϕ→ η)

7. ` ((ϕ→ ψ)→ ϕ)→ ϕ

8. ` ¬(ϕ→ ψ)→ (ψ → ϕ)

Exercise 1.6.25. Find a deduction showing the correctness ofsome of the following equivalences, that is, if ϕ ≡ ψ, then pro-vide a deduction for ` ϕ→ ψ and for ` ψ → ϕ.

1. ϕ ∨ (ϕ ∧ ψ) ≡ ϕ,

2. (ϕ1 → ϕ2) ∨ (ϕ1 → ϕ3) ≡ ϕ1 → (ϕ2 ∨ ϕ3).

28

Exercise 1.6.26 (* not required). Fill the missing parts ofthe proofs of the soundness and completeness theorems in Sec-tion 1.6.

29

Chapter 2

Basic Structures: Sets,Functions, Sequences, Sums,and Matrices

2.1 Sets and Functions

Definition 2.1.1.

• Fix an universal set U . Set operations: union ∪, intersec-tion ∩, complement A.

• Set inclusion: A ⊆ B iff for all a ∈ A it holds a ∈ B.A = B iff A ⊆ B and B ⊆ A.

• Given a set S, the power set of S is the set of all subsetsof the set S. The power set is denoted by P(S), or 2S.

• The Cartesian product of sets A1, A2, . . . , An is defined by:A1 × · · · × An := (a1, . . . , an) | ai ∈ Ai for i = 1, . . . , n.

• The cardinality of finite set A, denoted by |A|, is the num-ber of its elements. The principle of inclusion-exclusion:

|A ∪B| = |A|+ |B| − |A ∩B|

30

Definition 2.1.2. Let A and B be nonempty sets. A functionf : A→ B from A to B is an assignment of exactly one elementof B to each element of A. We write f(a) = b if b ∈ B isassigned by f to the element a ∈ A. We say that

• A is the domain of f ,

• B is the codomain of f .

• If f(a) = b, we say that b is the image of a and a is apreimage of b.

• The range, or image, of f is the set of all images of ele-ments of A.

Given two sets A and B,

• a relation R between A and B is a subset of A×B.

• a function from A to B must be a relation, but the inversedoes not hold in general.

Definition 2.1.3. Let A and B be two sets. The functionf : A→ B is called

• one-to-one, or an injunction, if and only if f(a) = f(b)implies that a = b for all a and b in the domain of f .

32

• onto, or a surjection, if and only if for every element b ∈ Bthere is an element a ∈ A with f(a) = b.

• one-to-one correspondence, or a bijection, if it is both one-to-one and onto.

Definition 2.1.4. Let A, B, and C be three sets.

• Let f : A → B be bijective. The inverse function of f ,denoted by f−1, is the function that assigns to an elementb ∈ B the unique element a ∈ A such that f(a) = b.

• Let g : A → B and let f : B → C. The composition of thefunctions f and g, denoted f g, is defined by

(f g)(a) = f(g(a))

Let A, B, and C be three sets.

33

• Let R ⊆ A × B, then the inverse relation of R, denotedby R−1, is a subset of B × A such that (b, a) ∈ R−1 iff(a, b) ∈ R, which is a relation between B and A.

• Let R1 ⊆ : A×B and R2 ⊆ B×C. The composition of therelations R1 and R2, denoted R1 R2, is defined by

(R1 R2) = (a, c) | ∃b.(a, b) ∈ R1 ∧ (b, c) ∈ R2

Definition 2.1.5 (Some Notations). Let A and B be two sets.

• For a function f : A → B, and a set D ⊆ A, we usef |D : D → B to denote the function f with domain restrict-ed to the set D.

• A partial function f from a set A to a set B is an assign-ment to each element a ∈ D ⊆ A, called the domain ofdefinition of f , of a unique element b ∈ B. We say that fis undefined for elements in A \D. When D = A, we saythat f is a total function.

Definition 2.1.6. Consider the set U = 2AP of all assignments.The semantic bracket is a function J · K : PL→ 2U defined by:

• JpK = σ ∈ U | p ∈ σ,

34

• J¬ϕK = JϕK,

• Jϕ→ ψK = JϕK ∪ JψK.

Is J · K injective, surjective, or bijective?

Definition 2.1.7 (Currying). Let A1, . . . , An and B be sets

• for any f ∈ A1 × A2 × · · ·An → B, there exists f ′ ∈ A1 →(A2 → · · · (An → B) · · · ), such that for any (a1, · · · , an) ∈A1× · · ·×An, it follows f(a1, · · · , an) = f ′(a1)(a2) · · · (an).

• Conversely, for any f ′ ∈ A1 → (A2 → · · · (An → B) · · · ),there exists f ∈ A1 × A2 × · · ·An → B, such that for any(a1, · · · , an) ∈ A1 × · · ·An, it follows f ′(a1)(a2) · · · (an) =f(a1, · · · , an).

35

2.2 Cardinality, Diagonalization Argument

Definition 2.2.1. Let A and B be two sets.

• The sets A and B have the same cardinality if and only ifthere is a one-to-one correspondence from A to B. WhenA and B have the same cardinality, we write |A| = |B|.

• If there is a one-to-one function from A to B, the cardinal-ity of A is less than or the same as the cardinality of B andwe write |A| ≤ |B|. Moreover, when |A| ≤ |B| and A andB have different cardinality, we say that the cardinality ofA is less than the cardinality of B and we write |A| < |B|.

Definition 2.2.2. A set that is either finite or has the samecardinality as the set of positive integers is called countable. Aset that is not countable is called uncountable. When an infiniteset S is countable, we denote the cardinality of S by ℵ0. We write|S| = ℵ0 and say that S has cardinality aleph null.

36

Theorem 2.2.3 (SCHRODER-BERNSTEIN THEOREM). IfA and B are sets with |A| ≤ |B| and |B| ≤ |A|, then |A| = |B|.

37

Lemma 2.2.4. Prove that

1. The union, intersection of countable sets is countable.

2. The set N2 is countable.

3. The set Z of integer numbers is countable.

4. The set Q of rational numbers is countable.

5. The set Nc with c ∈ N is countable.

6. The countable union of countable sets is countable.

7. The set N∗ is countable.

38

Lemma 2.2.5. Prove that

1. |[0, 1]| = |(0, 1]| = |[0, 1)| = |(0, 1)|.

2. |(0, 1]| = |[1,∞)|.

3. |[0, 1]| = |[0, k]| = |[0,∞)| = |R|.

4. |0, 1ω| = |[0, 1]|.

5. |2N| = |0, 1ω|.

6. |2N| = | f | f : N→ 0, 1 |.

39

Lemma 2.2.6 (Cantor diagonalization argument).

• The set R of real numbers is uncountable.

• For a set A, it holds: |A| < |2A|.

Exercise

• Prove R is uncountable.

• Prove (ϕ1 → ψ1) ∨ (ϕ2 → ψ2) ` (ϕ1 → ψ2) ∨ (ϕ2 → ψ1).

• Prove (ϕ ∧ ψ)→ θ, θ → χ, ψ ∧ ¬χ ` ¬ϕ.

40

Chapter 3

First Order Logic (FOL)

3.1 Syntax of FOL

Propositional logic is a coarse language, which only concernsabout propositions and boolean connectives. Practically, thislogic is not powerful enough to describe important propertieswe are interested in.

Example 3.1.1 (Syllogism of Aristotle). Consider the followingassertions:

1. All men are mortal.

2. Socrates is a man.

3. So Socrates would die.

∀x(Man(x)→Mortal(x))

Definition 3.1.2. First order logic is an extension of proposi-tion logic:

1. To accept parameters, it generalized propositions to predi-cates.

41

2. To designate elements in the domain, it is equipped withfunctions and constants.

3. It also involves quantifiers to capture infinite conjunctionand disjunction.

Definition 3.1.3. We are given:

• an arbitrary set of variable symbols VS = x, y, x1, . . . ;

• an arbitrary set (maybe empty) of function symbols FS =f, g, f1, . . . , where each symbol has an arity;

• an arbitrary set (maybe empty) of predicate symbols PS =P,Q, P1, . . . , where each symbol has an arity;

• an equality symbol set ES which is either empty or oneelement set containing ≈.

Let L = VS ∪ (, ),→,¬,∀ ∪ FS ∪ PS ∪ ES. Here VS ∪(, ),→,¬,∀ are referred to as logical symbols, and FS ∪PS ∪ES are referred to as non-logical symbols.

We often make use of the

• set of constant symbols, denoted by CS = a, b, a1, . . . ⊆FS, which consist of function symbols with arity 0;

42

• set of propositional symbols, denoted by PS = p, q, p1, . . . ⊆FS, which consist of predicate symbols with arity 0.

Definition 3.1.4 (FOL terms). The terms of the first orderlogic are constructed according to the following grammar:

t ::= x | ft1 . . . tn

where x ∈ VS, and f ∈ FS has arity n.Accordingly, the set T of terms is the smallest set satisfying

the following conditions:

• each variable x ∈ VS is a term.

• Compound terms: ft1 . . . tn is a term (thus in T ), providedthat f is a n-arity function symbol, and t1, . . . , tn ∈ T .Particularly, a ∈ CS is a term.

We often write f(t1, . . . , tn) for the compound terms.

43

Definition 3.1.5 (FOL formulas). The well-formed formulas ofthe first order logic are constructed according to the followinggrammar:

ϕ ::= Pt1 . . . tn | ¬ϕ | ϕ→ ϕ | ∀xϕ

where t1, . . . , tn are terms, P ∈ PS has arity n, and x ∈ VS.We often write P (t1, . . . , tn) for clarity. Accordingly, the set

FOF of first order formulas is the smallest set satisfying:

• P (t1, . . . , tn) ∈ FOF is a formula, referred to as the atomicformula.

• Compound formulas: (¬ϕ) (negation), (ϕ → ψ) (impli-cation), and (∀xϕ) (universal quantification) are formulas(thus in FOF ), provided that ϕ, ψ ∈ FOF .

We omit parentheses if it is clear from the context.As syntactic sugar, we can define ∃xϕ as ∃xϕ := ¬∀x¬ϕ.

We assume that ∀ and ∃ have higher precedence than all logicaloperators.

44

Examples of first-order logics

1. Mathematical theories:

• Presburger Arithmetic 〈N, 0, 1,+,=, <〉.• Peano Arithmetic 〈N, 0, S,+, ·,=, <〉• Tarski Algebra 〈R, 0,+, ·,=, <〉• Group 〈e,+,=〉.• Equivalence 〈R〉.

2. Example

• Write “every son of my father is my brother” in predi-cate logic.

• Let me denote “me”, S(x, y) (x is a son of y), F (x; y)(x is the father of y), and B(x; y) (x is a brother of y)be predicate symbols of arity 2. Consider

∀x∀y(F (x; me) ∧ S(y;x)→ B(y; me)).

• Alternatively, let f (f(x) is the father of x) be a unaryfunction symbol. Consider

∀x(S(x; f(me))→ B(x; me)).

45

• Translating an English sentence into predicatelogic can be tricky.

Definition 3.1.6 (Sub-formulas). For a formula ϕ, we definethe sub-formula function Sf : FOF → 2FOF as follows:

Sf (P (t1, . . . , tn)) = P (t1, . . . , tn)Sf (¬ϕ) = ¬ϕ ∪ Sf (ϕ)

Sf (ϕ→ ψ) = ϕ→ ψ ∪ Sf (ϕ) ∪ Sf (ψ)

Sf (∀xϕ) = ∀xϕ ∪ Sf (ϕ)

Sf (∃xϕ) = ∃xϕ ∪ Sf (ϕ)

Definition 3.1.7 (Scope). The part of a logical expression towhich a quantifier is applied is called the scope of this quantifier.Formally, each sub-formula of the form Qxψ ∈ Sf (ϕ), the scopeof the corresponding quantifier Qx is ψ. Here Q ∈ ∀,∃.

46

Substitution for Terms

Definition 3.1.8 (Sentence). We say an occurrence of x in ϕ isfree if it is not in scope of any quantifiers ∀x (or ∃x). Otherwise,we say that this occurrence is a bound occurrence. If a variable ϕhas no free variables, it is called a closed formula, or a sentence.

Definition 3.1.9 (Substitution). The substitution of x with twithin ϕ, denoted as Sxt ϕ, is obtained from ϕ by replacing eachfree occurrence of x with t.

47

We would extend this notation to Sx1,...,xnt1,...,tn ϕ.

Remark 3.1.10. It is important to remark that Sx1,...,xnt1,...,tn ϕ is notthe same as Sx1t1 . . . S

xntn ϕ: the former performs a simultaneous

substitution.For example, consider the formula P (x, y): the subsitution

Sx,yy,xP (x, y) gives Sx,yy,xP (x, y) = P (y, x) while the substitutionsSxyS

yxP (x, y) give SxyS

yxP (x, y) = SxyP (x, x) = P (y, y).

Remark 3.1.11. Consider ϕ = ∃y(x < y) in the number theory.What is Sxt ϕ for the special case of t = y?

Definition 3.1.12 (Substitutable on Terms). We say that t issubstitutable for x within ϕ iff for each variable y occurring int, there is no free occurrence of x in scope of ∀y/∃y in ϕ.

48

Definition 3.1.13 (α-β condition). If the formula ϕ and thevariables x and y fulfill:

1. y has no free occurrence in ϕ, and

2. y is substitutable for x within ϕ,

then we say that ϕ, x and y meet the α-β condition, denoted asC(ϕ, x, y).

Lemma 3.1.14. If C(ϕ, x, y), then SyxSxyϕ = ϕ.

49

3.2 The Axiom System: the Hilbert’s System

As for propositional logic, also FOL can be axiomatized.

Definition 3.2.1 (Axioms). 1. ϕ→ (ψ → ϕ)

2. (ϕ→ (ψ → η))→ ((ϕ→ ψ)→ (ϕ→ η))

3. (¬ϕ→ ¬ψ)→ (ψ → ϕ)

4. ∀xϕ→ Sxt ϕ

if t is substitutable for x within ϕ

5. ∀x(ϕ→ ψ)→ (∀xϕ→ ∀xψ)

6. ϕ→ ∀xϕif x is not free in ϕ

7. ∀x1 . . . ∀xnϕif ϕ is an instance of (one of) the above axioms

MP Rule:ϕ→ ψ ϕ

ψ

50

Definition 3.2.2 (Syntactical Equivalence). We say ϕ and ψare syntactically equivalent iff ϕ ` ψ and ψ ` ϕ.

Theorem 3.2.3. (Gen): If x has no free occurrence in Γ, thenΓ ` ϕ implies Γ ` ∀xϕ.

Solution. Suppose that ϕ0, ϕ1, . . . , ϕn = ϕ is the deductive se-quence of ϕ from Γ.

• If ϕi is an instance of some axiom, then according to (A7),∀xϕi is also an axiom.

• If ϕi ∈ Γ, since x is not free in Γ, we have ` ϕi → ∀xϕiaccording to (A6). Therefore, we have Γ ` ∀xϕi in thiscase.

• If ϕi is obtained by applying (MP) to some ϕj and ϕk =ϕj → ϕi. By induction, we have Γ ` ∀xϕj and Γ `∀x(ϕj → ϕi). With (A5) and (MP), we also have Γ ` ∀xϕiin this case.

Thus, we have Γ ` ∀xϕn, i.e., Γ ` ∀xϕ.

51

Exercise 3.2.4. Prove that

1. ∀x∀yϕ ` ∀y∀xϕ,

2. ∃x∀yϕ ` ∀y∃xϕ.


1. ∀x(ϕ→ ψ) ` ∀x(¬ψ → ¬ϕ),

2. ∀x(ϕ→ ψ) ` ∃xϕ→ ∃xψ.

52


1. If Γ ` ϕ and Γ ` ¬ψ, then Γ ` ¬(ϕ→ ψ),

2. ∀x¬(ϕ→ ψ) ` ¬(ϕ→ ∃xψ).

Lemma 3.2.7. (Ren): If C(ϕ, x, y), then ∀xϕ and ∀ySxyϕ aresyntactical equivalent. That is,

1. ∀xϕ ` ∀ySxyϕ.

2. ∀ySxyϕ ` ∀xϕ.

53

Lemma 3.2.8. (RS): Let ηϕψ denote the formula obtained byreplacing (some or all) ϕ inside η by ψ.

If ϕ ` ψ and ψ ` ϕ then η ` ηϕψ and ηϕψ ` η.

Solution. By induction on the structure of η.

Lemma 3.2.9. If C(ϕ, x, y) and Γ ` ψ, then Γ ` ψ∀xϕ∀ySxyϕ

.

Solution. An immediate result of (Ren) and (RS).

54

Theorem 3.2.10. (GenC) If Γ ` Sxaϕ where a does not occurin Γ ∪ ϕ, then Γ ` ∀xϕ.

55

3.3 Semantics of FOL

To give semantics of terms/formulas of first order logic, we needan appropriate structure in which interpret the functions andpredicates of FOL.

Definition 3.3.1. A Tarski structure is a pair I = 〈D, I〉,where:

• D is a non-empty set, called the domain.

• For each n-ary function f , we have I(f) ∈ Dn → D.

• For each n-ary predicate P , we have I(P ) ∈ Dn → 0, 1.

Thus, for each constant a, we have I(a) ∈ D.

Definition 3.3.2. Given a Tarski structure I = 〈D, I〉, anassignment σ under I is a mapping σ : VS → D.

We use ΣI to denote the set consisting of assignments underI .

56

Definition 3.3.3. Let I = 〈D, I〉 and σ ∈ ΣI .Each term t is interpreted to an element I (t)(σ) belonging

to D:

• If t = x is a variable, then I (t)(σ) = σ(x).

• If t = f(t1, . . . , tn) where f is an n-ary function, thenI (t)(σ) = I(f)(I (t1)(σ), . . . ,I (tn)(σ)).

Thus, if t = a is a constant, then I (t)(σ) = I(a).

Definition 3.3.4. Each formula ϕ has a truth value I (ϕ)(σ) ∈0, 1:

• If ϕ = P (t1, . . . , tn), where P is an n-ary predicate, thenI (ϕ)(σ) = I(P )(I (t1)(σ), . . . ,I (tn)(σ)).

57

• If ϕ = ¬ψ, then I (ϕ)(σ) = 1−I (ψ)(σ).

• If ϕ = ψ → η, then

I (ϕ)(σ) =

1 if I (ψ)(σ) = 0 or I (η)(σ) = 1,

0 if I (ψ)(σ) = 1 and I (η)(σ) = 0.

• If ϕ = ∀xψ, then

I (ϕ)(σ) =

1 if I (ψ)(σ[x/d]) = 1 for each d ∈ D,

0 if I (ψ)(σ[x/d]) = 0 for some d ∈ D

where σ[x/d] is a new assignment defined as

σ[x/d](y) =

σ(y) if y 6= x,

d if y = x.

We write (I , σ) ϕ if I (ϕ)(σ) = 1.

58

Theorem 3.3.5 (Theorem of Substitution). Suppose that t issubstitutable for x within ϕ, then

(I , σ) Sxt ϕ if and only if (I , σ[x/I (t)(σ)]) ϕ.

59

We say that I is a model of ϕ, denoted as I ϕ, if (I , σ) ϕ for each σ ∈ ΣI .

In particular, we say that I = 〈D, I〉 is a frugal model of ϕif |D| is not more than the cardinality of the language.

Recall that ϕ is a sentence, if there is no free variable occur-ring in ϕ.

Theorem 3.3.6. If ϕ is a sentence, then

• I ϕ iff (I , σ) ϕ for some σ ∈ ΣI .

Definition 3.3.7. Let ϕ, ψ be FOL formulas and Γ be a set ofFOL formulas. Then we define:

• (I , σ) Γ if for each η ∈ Γ, (I , σ) η;

• Γ |= ϕ if for each I and σ ∈ ΣI , (I , σ) Γ implies(I , σ) ϕ;

• ϕ and ψ are equivalent if ϕ |= ψ and ψ |= ϕ;

• ϕ is valid if ∅ |= ϕ.

60

Definition 3.3.8 (Tautology for FOL). For a formula ϕ ∈ FOL,we construct ϕ′ as follows:

• for each sub-formula ψ of ϕ which is either an atomic for-mula, or a formula of the form ∀xη, we replace it with acorresponding propositional variable pψ.

If ϕ′ is a tautology in propositional logic, then we say ϕ is atautology for FOL.

Definition 3.3.9 (Prenex Normal Form (PNF)). A formula isin prenex normal form if and only if it is of the form Q1x1Q2x2 . . . QkxkP (x1, x2, . . . , xk),where each Qi, i = 1, 2, ... , k is either the existential quantifi-er or the universal quantifier, and P (x1, . . . , xk) is a predicateinvolving no quantifiers.

61

Question: can we transform a formula into an equivalent P-NF form?

3.4 A Sound and Complete Axiomatization

for FOL without Equality ≈

3.4.1 The Axiom System: Soundness

Similarly to propositional logic, for FOL we have the soundnessproperty:

Theorem 3.4.1. If Γ ` ϕ, then Γ |= ϕ.

Hint. For proving the theorem, show and make use of the fol-lowing results:

• ∀x(ϕ→ ψ),∀xϕ |= ∀xψ;

• if x is not free in ϕ, then ` ϕ→ ∀xϕ.

Corollary 3.4.2. If ` ϕ, then |= ϕ.

62

3.4.2 The Axiom System: Completeness

A Hintikka set Γ is a set of FOL formulas fulfilling the followingproperties:

1. For each atomic formula ϕ (i.e, ϕ = P (t1, . . . , tn), wheren ≥ 0), either ϕ /∈ Γ or ¬ϕ /∈ Γ.

2. ϕ→ ψ ∈ Γ implies that either ¬ϕ ∈ Γ or ψ ∈ Γ.

3. ¬¬ϕ ∈ Γ implies that ϕ ∈ Γ.

4. ¬(ϕ→ ψ) ∈ Γ implies that ϕ ∈ Γ and ¬ψ ∈ Γ.

5. ∀xϕ ∈ Γ implies that Sxt ϕ ∈ Γ for each t which is substi-tutable for x within ϕ.

6. ¬∀xϕ ∈ Γ implies that there is some t with C(ϕ, x, t) suchthat ¬Sxt ϕ ∈ Γ.

Note: C(ϕ, x, t) iff C(ϕ, x, y) for all y occurring in t.

Lemma 3.4.3. A Hintikka set Γ is consistent, and moreover,for each formula ϕ, either ϕ 6∈ Γ, or ¬ϕ 6∈ Γ.

Theorem 3.4.4. A Hintikka set Γ is satisfiable, i.e, there issome interpretation I and some σ ∈ ΣI such that (I , σ) ϕ

for each ϕ ∈ Γ.

63

Theorem 3.4.5. If Γ is a set of FOL formulas, then “Γ isconsistent” implies that “Γ is satisfiable”.

Particularly, if Γ consists only of sentences, then Γ has afrugal model.

Proof. Let us enumerate1 the formulas as ϕ0, ϕ1, . . . , ϕn, . . ., andsubsequently define a series of formula sets as follows. Let Γ0 =Γ, and

Γi+1 =

Γi ∪ ¬ϕi if Γi ` ¬ϕiΓi ∪ ϕi if Γi 6` ¬ϕi and ϕi 6= ¬∀xψΓi ∪ ϕi,¬Sxaψ if Γi 6` ¬ϕi, and ϕi = ¬∀xψ

Above, for each formula ∀xψ, we pick and fix the constant awhich does not occur in Γi ∪ ϕi.

1We assume the language to be countable, yet the result can be extended to languageswith arbitrary cardinality.

64

Finally let Γ∗ = limi→∞

Γi.

If Γ is consistent, the set Γ∗ is maximal and consistent, andis referred to as the Henkin set. Thus, a Henkin set is also aHintikka set.

65

Theorem 3.4.6. If Γ |= ϕ, then Γ ` ϕ.

Corollary 3.4.7. If |= ϕ, then ` ϕ.

Theorem 3.4.8. Γ is consistent iff each of its finite subset isconsistent. Moreover, Γ is satisfiable iff each of its finite subsetsis satisfiable.

66

3.5 A Sound and Complete Axiomatization

for FOL with Equality ≈

The axiomatization based on the Hilbert’s systems seen in theprevious section can be extended to the case of first order logicwith the equality ≈. To do this, two additional axioms have tobe included in the Hilbert’s system:

A≈: x ≈ x;

A′≈: (x ≈ y)→ (α→ αxy), where α is an atomic formula.

The soundness and completeness results can be proved sim-ilarly in the extended Hilbert’s system; note that for the com-pleteness one, a variation of the Tarski structure is required,namely, the domain considered in the construction modulo therelation ≈. This allows us so manage correctly the formulas thatare equivalent under ≈.

The actual details about the above construction are omitted;the interested reader is invited to formalize them.

67

Exercise 3.6.1. Show, by providing the appropriate functions,that the set of identifiers has the same cardinality as N.

Exercise 3.6.2. Show that the set of all well-formed WHILEprograms is countable.

Exercise 3.6.3. Show the following statements:

1. If A is an uncountable set and B is a countable set, then|A \B| = |A|.

2. Let I be the set of irrational numbers, i.e., I = r ∈ R |r /∈ Q . Then |0, 1ω \ 0, 1∗0ω| = |I ∩ [0, 1]|.

3. |0, 1ω| = |[0, 1]|.

4. |[0, 1]| = |[0, 1]× [0, 1]| = |[0, 1]n| for each n ≥ 1.

70

Hint. For point 3, make use of points 1 and 2; for point 4, makeuse of point 3.

Exercise 3.6.4. Consider the semantic bracket operator pre-sented in Definition 2.1.6 of the lecture notes. Show that |=ϕ→ ψ iff JϕK ⊆ JψK.

Exercise 3.6.5. Let Q1 ⊆ FOL be the set of FOL formulasϕ such that each quantifier operator Qx (with Q ∈ ∀,∃ andx ∈ VS) appears at most once in ϕ.

Provide a function Scope, including its type, such that, givena quantifier operator Qx (with Q ∈ ∀,∃ and x ∈ VS) anda formula ϕ ∈ Q1, it returns the formula corresponding to thescope of Qx.

71

Exercise 3.6.6. Given ϕ, ψ, η ∈ FOL,

1. provide a function #, including its type, that returns howmany times ψ occurs in ϕ as sub-formula;

2. provide a function R∞, including its type, that replaces eachoccurrence of ϕ in η with ψ;

3. provide a function R1, including its type, that replaces ex-actly one occurrence of ϕ in η with ψ if ϕ occurs in η, andthat returns η if ϕ does not occur in η. If ϕ occurs in η mul-tiple times, then there is no requirement on the particularinstance to be replaced.

Exercise 3.6.7. Prove the following:

72

1. For any predicate P with arity 2, ∀x∀yP (x, y) ` ∀y∀zP (y, z).

2. Assume x is not free in ϕ, then ϕ→ ∀xψ and ∀x(ϕ→ ψ)are syntactically equivalent.

3. We say a formula ϕ has repeated occurrences of a boundvariable x, if Qx appears more than once in the sub-formulasof ϕ (recall Q ∈ ∀,∃). Prove that there exists a formulaϕ′ which has no repeated occurrences of any bound variablesuch that ϕ and ϕ′ are syntactically equivalent.

Exercise 3.6.8. Prove the following:

Q1 ¬∀xϕ à ∃x¬ϕ; and ¬∃xϕ à ∀x¬ϕ.

Q2 ∀xϕ ∧ ψ à ∀x(ϕ ∧ ψ, if x does not occur in ψ.

Q3 ∃xϕ ∨ ∃xψ à ∃x(ϕ ∨ ψ).

73

Exercise 3.6.9. Let S be a binary predicate symbol, P and Qunary predicate symbols. Prove the following:

Q1 ∃x∃y(S(x, y) ∨ S(y, x)) ` ∃x∃yS(x, y).

Q2 ∀x∀y∀z(S(x, y)∧S(y, z)→ S(x, z)), ∀x¬S(x, x) ` ∀x∀y(S(x, y)→¬S(y, x)).

Q3 ∃x∃y(S(x, y) ∨ S(y, x)), ¬∃xS(x, x) ` ∃x∃y(x 6= y).

Q4 ∀x(P (x) ∨Q(x)) ` ∀xP (x) ∨ ∀Q(x) is not provable.

Exercise 3.6.10. Prove Lemma 3.4.3: a Hintikka set Γ is con-sistent, and moreover, for each formula ϕ, either ϕ 6∈ Γ, or¬ϕ 6∈ Γ.

74

Exercise 3.6.11. Given a formula ϕ, let H(ϕ) be the set ofHintikka sets containing ϕ, that is, H(ϕ) = Γ ⊆ FOF |ϕ ∈ Γ and Γ is a Hintikka set . We say that Γ ∈ H(ϕ) is min-imal if, for each Γ′ ∈ H(ϕ), it holds that Γ′ ⊆ Γ implies Γ′ = Γ;we denote by m(ϕ) the set of minimal Hintikka sets in H(ϕ),that is, m(ϕ) = Γ ∈ H(ϕ) | Γ is minimal .

1. Provide a minimal Hintikka set Γϕ ∈ m(ϕ) for the formula

ϕ = ∀x∀y(¬(x ≈ y)→ (R(x, y)→ ¬R(y, x)))

under the assumption that VS = x, y, FS = CS = a, b,PS = R, and ES = ≈.

2. Prove that H(ϕ) ∩ H(¬ϕ) = ∅ for each ϕ ∈ FOF .

3. Let PL ⊆ FOF be the set of FOL formulas in which eachpredicate appears at most once and in which no quantifierQ ∈ ∀,∃ occurs. Define a function c : PL→ N such that,for each ϕ ∈ PL, returns the number of different minimalHintikka sets containing ϕ.

75

Chapter 4

Algorithms

Definition 4.0.12 (Algorithm (from wikipedia)). • An algo-rithm is a set of rules that precisely defines a sequence ofoperations, which can perform calculation, data processingand automated reasoning tasks.

• An algorithm is an effective method that can be expressedwithin a finite amount of space and time and in a well-defined formal language for calculating a function. Startingfrom an initial state and initial input (perhaps empty), theinstructions describe a computation that, when executed,proceeds through a finite number of well-defined successivestates, eventually producing “output” and terminating at afinal ending state. The transition from one state to the nextis not necessarily deterministic; some algorithms, known asrandomized algorithms, incorporate random input.

• History

– The concept of algorithm has existed for centuries;

– what would become the modern algorithm began withattempts to solve the Entscheidungsproblem (the ”deci-

76

sion problem”) posed by David Hilbert in 1928.

– Subsequent formalizations were framed as attempts todefine “effective calculability” or “effective method”;those formalizations included the Godel-Herbrand-Kleenerecursive functions, Alonzo Church’s lambda calculus,Emil Post’s “Formulation 1”, and Alan Turing’s Tur-ing machines.

– Giving a formal definition of algorithms, cor-responding to the intuitive notion, remains achallenging problem.

• Church-Turing Thesis: any real-world computation canbe translated into an equivalent computation in-volving a Turing machine.

• An algorithm can be considered to be any sequenceof operations that can be simulated by a Turing-complete system.

77

Definition 4.0.13 (Algorithm (Cont’d)). • Expressing algo-rithms: high-level description, implementation-level descrip-tion and formal description.

• Complexity analysis: Formal versus empirical, execu-tion efficiency.

• Classification

– By implementation: recursion, logical, serial, parallel,distributed, deterministic vs nondeterministic, exact vsapproximation, quantum

– By design paradigm: brute-force or exhaustive search,divide and conquer, search and enumeration, random-ized algorithms.

– By optimization problems: linear programming, dynam-ic programming, integer programming, semi-definite pro-gramming, the greedy method, the heuristic method

– By field of study

– By complexity: space complexity and time complexity.

78

Procedure declaration

Pr ::= procedure Id(Id , . . . , Id) P

| procedure Id(Id , . . . , Id) P ; return Exp

Programs:

P ::= skip | Id := Exp | Id [Exp] := Exp | P ;P

| if BExp then P else P fi

| while BExp do P done

| for Id := Exp to Exp do P done

| Id(Exp, . . . ,Exp)

4.2 Examples of algorithms

Algorithm 4.2.1 (Collatz Conjecture). The following pseudo-code in PROC terminates when the value of n becomes to 1.The input n is a natural number, greater than or equal to 1.

procedure Collatz(n)while n 6= 1 do

if even(n) thenn := n/2

elsen := n× 3 + 1

fidone

Discussion: Does this algorithm terminate?

80

Algorithm 4.2.2 (Maximum in a generic array). The followingpseudo-code in PROC returns the maximum value occurring inthe provided array.

procedure max(array)maxvalue := array[0];for i := 1 to length(array)− 1 do

if array[i] > maxvalue thenmaxvalue := array[i]

elseskip

fidone;return maxvalue

Algorithm 4.2.3 (Index of a value in an array). The followingpseudo-code in PROC returns the index of the specified value

if it occurs in the provided array, otherwise length(array) isreturned.

procedure indexOf(value, array)index := length(array);for i := 0 to length(array)− 1 do

if array[i] = value thenindex := i

elseskip

fidone;return index

Algorithm 4.2.4 (Index of a value in a sorted array). The

81

following pseudo-code in PROC returns the index of the spec-ified value if it occurs in the provided sorted array, otherwiselength(array) is returned.

procedure indexOfSorted(value, array)index := length(array);low := 0;high := length(array)− 1;while low < high domiddle := (low + high)/2;if array[middle] < value thenlow := middle + 1

elsehigh := middle

fidone;if array[low] = value thenindex := low

elseskip

fi;return index

Algorithm 4.2.5 (Swap elements in an array). The followingpseudo-code in PROC swaps the elements at index i and j inthe provided array.

procedure swap(array, i, j)temp := array[i];array[i] := array[j];array[j] := temp

82

Algorithm 4.2.6 (Bubble sort). The following pseudo-code inPROC sorts the provided array.

procedure bubbleSort(array)for i := 0 to length(array)− 1 do

for j := 0 to (length(array)− 1)− i doif array[j] > array[j + 1] thenswap(array, j, j + 1)

elseskip

fidone

done

Algorithm 4.2.7 (Gnome sort). The following pseudo-code inPROC sorts the provided array.

procedure gnomeSort(array)i := 0;while i < length(array) do

if i = 0 ∨ array[i− 1] ≤ array[i] theni := i + 1

elseswap(array, i, i− 1);i := i− 1

fidone

Algorithm 4.2.8 (Insertion sort). The following pseudo-codein PROC sorts the provided array.

83

procedure insertionSort(array)for i := 1 to length(array)− 1 doj := i;while j > 0 ∧ array[j− 1] > array[j] doswap(array, j, j− 1);j := j− 1

donedone

Greedy Algorithms

Algorithm 4.2.9 (Change Making). The algorithm makes changesc1 > c2 . . . > cr for n cents.

procedure procedureChange(c1, c2, . . . , cr)for i := 1 to r dodi := 0;while n ≥ ci dodi := di + 1;n := n− ci

donedone

Algorithm 4.2.10 (Earliest Ending Time Job Scheduling). T-wo jobs are compatible if they do not overlap. Assumption: sortstalks with ending times e1 < e2 < . . . en.

procedure schedule(e1, e2, . . . , en)S := ∅;for i := 1 to n do

if talk i is compatible with S then S := S ∪ talk idone

84

Remark: set not defined in PROC

Definition 4.2.11. The Tableau approach for the satisfiabilityproblem of propositional logic: a propositional formula is satis-fiable iff it has a consistent tableau.

85

Definition 4.2.12 (Decidability). • A decision problem con-sists of a set of instances and a subset of yes-instances.

• The Primality problem: is the instance x a prime number?

• The answer (solution) to any decision problem is just onebit (true or false).

• A problem Q is decidable iff there is an algorithm A, suchthat for each instance q of Q, the computation A(q) stopswith an answer.

• A problem Q is semi-decidable iff there is an algorithm A,such that for each instance q of Q, if q holds, then A(q)stops with the positive answer; otherwise, A(q) either stopswith the negative answer, or does not stop.

86

Definition 4.2.13 (The Halting Problem). It takes as inputa computer program and input to the program and determineswhether the program will eventually stop when run with this in-put.

• If the program halts, we have our answer.

• If it is still running after any fixed length of time has e-lapsed, we do not know whether it will never halt or we justdid not wait long enough for it to terminate.

Definition 4.2.14 (Undecidability of the Halting Problem).

87

4.3 Satisfiability Problem and DPLL

The slides are downloadable from http://www.computational-logic.org/iccl/master/lectures/summer07/sat/slides/dpll.pdf

4.4 The Growth of Functions

Definition 4.4.1. Let f and g be functions from the set of in-tegers or the set of real numbers to the set of real numbers. Wesay that f(x) is O(g(x)) if there are constants C and k such that|f(x)| ≤ C|g(x)| whenever x > k.

88

Definition 4.4.2. Let f and g be functions from the set of in-tegers or the set of real numbers to the set of real numbers. Wesay that f(x) is Ω(g(x)) if there are positive constants C and ksuch that |f(x)| ≥ C|g(x)| whenever x > k.

Moreover, We say that f(x) is Θ(g(x)) if f(x) is O(g(x)) andΩ(g(x)).

Exercise 4.4.3. Show that:

1. Show that log n! is O(nlogn).

2. Show that n2 is not O(n).

3. Suppose that f1(x) is O(g1(x)) and that f2(x) is O(g2(x)).Then (f1 + f2)(x) is O(max(|g1(x)|, |g2(x)|)).

4. Suppose that f1(x) is O(g1(x)) and f2(x) is O(g2(x)). Then(f1f2)(x) is O(g1(x)g2(x)).

5. Show that 3x2 + 8xlogx is Θ(x2).

6. Assume an 6= 0. Show that∑n

i=0 aixi is Θ(xn).

89

4.5 Complexity of Algorithms

We are interested in the time complexity of the algorithm.

Definition 4.5.1. • A problem that is solvable using an algo-rithm with polynomial worst-case complexity is called tractable.Tractable problems are said to belong to class P .

90

• Problems for which a solution can be checked in polynomialtime are said to belong to the class NP .

• The P = NP problem asks whether NP , the class of prob-lems for which it is possible to check solutions in polynomialtime, equals P , the class of tractable problems.

• NP -complete problems: It is an NP problem and if a poly-nomial time algorithm for solving it were known, then P =NP .

• The satisfiability problem is NP -complete. Cook-LevinTheorem

A prize of 1, 000, 000 dollars is offered by the Clay Mathe-matics Institute for its solution.

4.6 Logic and Computer Science – Logical

Revolution

NOTE: The following material is from Moshe Vardi

Definition 4.6.1 (Hilberts Program). Hilberts Program (1922-1930): Formalize mathematics and establish that:

91

• Mathematics is consistent:

a mathematical statement and its negation cannot ever bothbe proved.

• Mathematics is complete:

all true mathematical statements can be proved.

• Mathematics is decidable: there is a mechanical way to de-termine whether a given mathematical statement is true orfalse.

Definition 4.6.2 (The Demise of Hilberts Program). Godel:

• Incompleteness of ordinary arithmetic - There is no sys-tematic way of resolving all mathematical questions.

• Impossibility of proving consistency of mathematics

Godel (1930): ”This sentence is not provable.”

• Church and Turing (1936): Unsolvability of first-order log-ic:

The set of valid first-order sentences is not computable.

92

Definition 4.6.3 (Entscheidungsproblem). Entscheidungsprob-lem (The Decision Problem) [Hilbert-Ackermann, 1928]: Decideif a given first-order sentence is valid (dually, satisfiable).

Church-Turing Theorem, 1936: The Decision Problem is un-solvable.

Turing, 1936:

• Defined computability in terms of Turing machines (TMs)

• Proved that the termination problem for TMs is unsolvable(”this machine terminates iff it does not terminate”)

• Reduced termination to Entscheidungsproblem.

Definition 4.6.4 (Mathematical Logic - 1936). Logic as Foun-dations of Mathematics:

93

• Incomplete (example: Continuum Hypothesis)

• Cannot prove its own consistency

• Unsolvable decision problem

• Unsolvable termination problem

Can we get some positive results?

• Focus on special cases!

Definition 4.6.5 (The Fragment-Classification Project). Idea:Identify decidable fragments of first-order logic - (1915-1983)

• Monadic Class (monadic predicates)

• Bernays-Schonfinkel Class (∃∗∀∗)

• Ackermann Class (∃∗∀∃∗)

• Godel Class (∃∗∀∀∃∗)

Outcome: Very weak classes! What good is first-order logic?

94

Definition 4.6.6 (Monadic Logic). Monadic Class: First-orderlogic with monadic predicates - captures syllogisms.∀x(Man(x)→Mortal(x))Lowenheim, 1915: The Monadic Class is decidable.

• Proof: Bounded-model property - if a sentence is satisfiable,it is satisfiable in a structure of bounded size.

• Proof technique: quantifier elimination.

Definition 4.6.7 (Logic of Integer Addition). Integer Addition:

• Domain: N (natural numbers)

• Predicate: =

95

• Addition function: +

• y = 2x : y = x+ x

• x ≤ y : (∃z)(y = x+ z)

• x = 0 : (∀y)(x ≤ y)

• x = 1 : x 6= 0 ∧ (∀y)(y = 0 ∨ x ≤ y)

• y = x+ 1 : (∃z)(z = 1 ∧ y = x+ z)

Bottom Line: Theory of Integer Addition can express IntegerProgramming (integer inequalities) and much more.

Definition 4.6.8 (Presburger Arithmetics). Mojzesz Presburg-er, 1929:

• Sound and complete axiomatization of integer addition

• Decidability: There exists an algorithm that decides whethera given first-order sentence in integer-addition theory istrue or false.

– Decidability is shown using quantifier elimination, sup-plemented by reasoning about arithmetical congruences.

96

– Decidability can also be shown using automata-theoretictechniques.

Definition 4.6.9 (Complexity of Presburger Arithmetics). Com-plexity Bounds:

• Oppen, 1978: TIME(222poly

) upper bound

• Fischer & Rabin, 1974: TIME(22lin

) lower bound

Rabin, 1974: ”Theoretical Impediments to Artificial Intelli-gence”: ”the complexity results point to a need for a carefulexamination of the goals and methods in AI”.

Definition 4.6.10 (Finite Words - Nondeterministic Finite Au-tomata). A = (Σ, S, S0, ρ, F )

97

• Alphabet: Σ

• States: S

• Initial states: S0 ⊆ S

• Nondeterministic transition function: ρ : S × Σ→ 2S

• Accepting states: F ⊆ S

Definition 4.6.11. Finite Words - Nondeterministic Finite Au-tomata

Input word: a0, a1, ..., an−1Run: s0, s1, ..., sn

• s0 ∈ S0

• si+1 ∈ ρ(si, ai) for i ≥ 0

Acceptance: sn ∈ FRecognition: L(A) - words accepted by A.Fact: NFAs define the class Reg of regular languages.

98

Definition 4.6.12 (Logic of Finite Words). View finite wordw = a0, ..., an−1 over alphabet Σ as a mathematical structure:

• Domain: 0, ..., n− 1

• Dyadic predicate: ≤

• Monadic predicates: Pa : a ∈ ΣMonadic Second-Order Logic (MSO):

• Monadic atomic formulas: Pa(x) (a ∈ Σ)

• Dyadic atomic formulas: x < y

• Set quantifiers: ∃P, ∀PExample: (∃x)((∀y)(¬(x < y)) ∧ Pa(x)) - last letter is a.

99

Definition 4.6.13 (Automata and Logic). [Buchi, Elgot, Trakht-enbrot, 1957-8 (independently)]: MSO ≡ NFA

Both MSO and NFA define the class Reg.Proof: Effective

• From NFA to MSO (A→ ϕA)

• From MSO to NFA (ϕ→ Aϕ)

Definition 4.6.14 (NFA Nonemptiness). Nonemptiness: L(A) =∅

Nonemptiness Problem: Decide if given A is nonempty.Directed Graph GA = (S,E) of NFA A = (Σ, S, S0, ρ, F ):

• Nodes: S

• Edges: E = (s, t) : t ∈ ρ(s, a) for some a ∈ Σ

It holds: A is nonempty iff there is a path in GA from S0 toF .

Decidable in time linear in size of A, using breadth-first searchor depth-first search.

100

Definition 4.6.15 (MSO Satisfiability - Finite Words). Satis-fiability: models(ψ) = ∅

Satisfiability Problem: Decide if given ψ is satisfiable.It holds: ψ is satisfiable iff Aψ is nonnempty.It holds: MSO satisfiability is decidable.

• Translate ψ to Aψ.

• Check nonemptiness of Aψ .

Definition 4.6.16 (Complexity Barrier). Computational Com-plexity:

• Naive Upper Bound: Nonelementary Growth 2 to the powerof the tower of height O(n)

101

• Lower Bound [Stockmeyer, 1974]: Satisfiability of FO overfinite words is nonelementary (no bounded-height tower).

Definition 4.6.17 (Program Verification). • The Dream - Hoare,1969: ”When the correctness of a program, its compiler,and the hardware of the computer have all been establishedwith mathematical certainty, it will be possible to place greatreliance on the results of the program.”

• The Nightmare - De Millo, Lipton, and Perlis, 1979: ”Webelieve that . . . program verification is bound to fail.We cannot see how it is going to be able to affect anyone’sconfidence about programs.”

• “. . . software verification . . . has been the Holy Grail of com-puter science for many decades but not in some very keyareas, for example, driver verification we are building toolsthat can do actual proof about the software and how it worksin order to guarantee the reliability.” (Bill Gates, keynoteaddress at Winhec 2002)

• Hoare has pose a grand challenge: “The verification chal-lenge is to achieve a significant body of verified programsthat have precise external specifications, complete internal

102

specifications, machine-checked proofs of correctness withrespect to a sound theory of programming.”

• NICTA seL4 using Isabelle/HOL, INRIA CompCert usingCoq

103

The Hoare Triple ϕPψ

104

Definition 4.6.18 (Logic in Computer Science: c. 1980). Sta-tus: Logic in CS is not too useful!

• First-order logic is undecidable.

• The decidable fragments are either too weak or too intractable.

• Even Boolean logic is intractable.

• Program verification is hopeless.

Definition 4.6.19. Post 1980: From Irrelevance to RelevanceA Logical Revolution:

• Relational databases

• Boolean reasoning

• Model checking

• Termination checking

• ...

105

Definition 4.6.20 (The Temporal Logic of Programs). Crux:Need to specify ongoing behavior rather than input/output rela-tion! ”Temporal logic to the rescue” [Pnueli, 1977]:

• Linear temporal logic (LTL) as a logic for the specificationof non-terminating programs

• Model checking via reduction to MSO

But: nonelementary complexity!

In 1996, Pnueli received the Turing Award for seminal workintroducing temporal logic into computing science and for out-standing contributions to program and systems verification.

106

Definition 4.6.21 (Examples). • always not (CS1 and C-S2): safety

• always (Request implies eventually Grant): liveness

• always (Request implies (Request until Grant)): liveness

Definition 4.6.22 (Model Checking). ”Algorithmic verification”[Clarke & Emerson, 1981, Queille & Sifakis, 1982]: Model check-ing programs of size m wrt CTL formulas of size n can be donein time mn.

Linear-Time Response [Lichtenstein & Pnueli, 1985]: Modelchecking programs of size m wrt LTL formulas of size n can bedone in time m2O(n) (tableau heuristics).

Seemingly:

• Automata: non-elementary

• Tableaux: exponential

107

Definition 4.6.23 (Back to Automata). Exponential-CompilationTheorem [Vardi & Wolper, 1983-1986]: Given an LTL formulaϕ of size n, one can construct an automaton Aϕ of size 2O(n)

such that a trace σ satisfies ϕ if and only if σ is accepted by Aϕ.Automata-Theoretic Algorithms:

• LTL Satisfiability: ϕ is satisfiable iff L(Aϕ) = ∅ (PSPACE)

• LTL Model Checking: M |= ϕ iff L(M×A¬ϕ) = ∅ (m2O(n))

Today: Widespread industrial usageIndustrial Languages: PSL, SV A (IEEE standards)

Definition 4.6.24 (Solving the Unsolvable). B. Cook, A. Podel-ski, and A. Rybalchenko, 2011:”in contrast to popular belief,proving termination is not always impossible”

108

• The Terminator tool can prove termination or divergenceof many Microsoft programs.

• Tool is not guaranteed to terminate! Explanation:

• Most real-life programs, if they terminate, do so for rathersimple reasons.

• Programmers almost never conceive of very deep and so-phisticated reasons for termination.

Definition 4.6.25 (Logic: from failure to success). Key Lesson-s:

• Algorithms

• Heuristics

• Experimentation

• Tools and systems

Key Insight: Do not be scared of worst-case complexity.

• It barks, but it does not necessarily bite!

109

discrete mathematics1 znj/dm2017lcs.ios.ac.cn/~znj/dm2017/lecture-notes-chapter4 (2).pdf · 1.2...

Documents