cisc 4090: theory of computation - chapter 0 …a ﬁrst course in theory of computation generally...

CISC 4090: Theory of ComputationChapter 0Introduction

Xiaolan Zhang, adapted from slides by Prof. Prof. Werschulz

Fordham UniversityDepartment of Computer and Information Sciences

Spring, 2014

Xiaolan Zhang, adapted from slides by Prof. Prof. Werschulz CISC 4090/Spring, 2014/Chapter 0

What is this course about?

This course is about the fundamental capabilities andlimitations of computers/computation.

This course covers three areas, which make up the the theoryof computation:

Automata and languagesComputability theoryComplexity theory


Automata and languages

Introduces models of computation

We will study finite automata and context-free grammars

Each model determines what can be expressed, as we will seein Part I of this course

Will allow us to become familiar with simple models before wemove on to more complex models such as Turing machines

Given a model, we can examine computability and complexity


Computability theory

A major mathematical discovery in the 1930s

Certain problems cannot be solved by computers:




Certain problems cannot be solved by computers:They have no algorithmic solution




Certain problems cannot be solved by computers:They have no algorithmic solutionWe can ask what a model can and cannot do





As it turns out, we have a simple model of a computer:





As it turns out, we have a simple model of a computer:a Turing machine





As it turns out, we have a simple model of a computer:a Turing machineSo we can ask what a computer can and cannot do (e.g.,compute)


Complexity theory

How hard is a problem?

You may already know a lot about this:

Big-O notationMaybe big-Θ notationIf Problem A requires time Θ(n2) and Problem B requirestime Θ(n), then Problem A is harder than Problem B.

We take one step forward

We will study NP-completenessA first course in theory of computation generally stops thereand hence we will not cover Chapters 8, 9, or 10 of the text


Mathematical preliminaries

We will now quickly review discrete math

Mathematical Notation

SetsSequences and tuplesFunctions and relationsGraphsStrings and languagesBoolean logic

Proofs and types of proofs


Sets

A set is a collection of objects; order doesn’t matter.

The objects are called elements or members.

Examples:

A finite set: {1, 3, 5}An infinite set:

{1, 3, 5, . . . }{ x | x ∈ Z and x mod 2 6= 0 }{ x : x ∈ Z and x mod 2 6= 0 }{x ∈ Z : x mod 2 6= 0 }


Sets (continued)

You should know these operators/concepts:

Subset (A ⊂ B or A ⊆ B)Cardinality: number of elements in set (|A| or n(A)).Intersection (∩) and union (∪)Complement ( or ′)


Sets (continued)


Subset (A ⊂ B or A ⊆ B)Cardinality: number of elements in set (|A| or n(A)).Intersection (∩) and union (∪)Complement ( or ′)What do we need to know to determine complement of A?


Sets (continued)


Subset (A ⊂ B or A ⊆ B)Cardinality: number of elements in set (|A| or n(A)).Intersection (∩) and union (∪)Complement ( or ′)What do we need to know to determine complement of A?Venn diagrams: can be used to visualize sets and theiroperationsPower set P(X ): All possible subsets of a given set X

If A = {0, 1}, then what is P(A)?In general, what is the cardinality of P(B)?


Sequences and tuples

A sequence is a list of objects; order matters.Examples: (1, 3, 5), (5, 3, 1).


Sequences and tuples

A sequence is a list of objects; order matters.Examples: (1, 3, 5), (5, 3, 1).

In this course we will use the term tuple instead.

(1, 3, 5) is a 3-tuple.A k-tuple has k elements.


Sequences and tuples (continued)

Cartesian product × is an operation on sets that yields a setof tuples.

Example: If A = {1, 2} and B = {x , y , z}, then

A× B = {(1, x), (1, y), (1, z), (2, x), (2, y), (2, z)}

If we have k sets A1,A2, . . . ,Ak , we can make the cartesianproduct A1 × A2 × · · · × Ak , which is the set of all k-tuples(a1, a2, . . . , ak), where ai ∈ Ai for i ∈ {1, 2, . . . }.


Sequences and tuples (continued)

We can take the Cartesian product of a set with itself:

Ak =

k times︷︸︸︷

A× A× . . .A

The set Z2 represents Z× Z, the set of all pairs of integers,which may be written

Z2 = { (a, b) : a, b ∈ Z }


Functions and relations

A function maps an input to a (single) output:f (a) = b, f maps a to b, f : a 7→ b.

The set of possible inputs is the domain

The set containing all possible outputs is the codomain

The set of all actual outputs is the range

If f has domain X and codomain Y , write f : X → Y

Example 1: For abs : Z → Z, what is the range?








Example 1: For abs : Z → Z, what is the range?Ran(abs) = N = {0, 1, 2, . . . }








Example 1: For abs : Z → Z, what is the range?Ran(abs) = N = {0, 1, 2, . . . }Example 2: For the sum: Z× Z → Z, what is the range?








Example 1: For abs : Z → Z, what is the range?Ran(abs) = N = {0, 1, 2, . . . }Example 2: For the sum: Z× Z → Z, what is the range?Ran(sum) = Cod(sum) = Z.








Example 1: For abs : Z → Z, what is the range?Ran(abs) = N = {0, 1, 2, . . . }Example 2: For the sum: Z× Z → Z, what is the range?Ran(sum) = Cod(sum) = Z.

Functions can be described using tables

Example 1: Describe f (x) = 2x , where x ∈ D = {1, 2, 3, 4}.Example 2: Describe add(x , y), where (x , y) ∈ D × D, withD = {1, 2, 3}.


Relations

A predicate is a function with range {true, false},Example: even(4) = true.

A k-ary relation is a predicate whose domain is Ak .

If k = 2, then binary relation (e.g., =, <, . . . )Can just list what is true (e.g., even(4))A k-ary relation can now considered a subset of Ak .Text represents the beats relation (page 9) using a table and aset

Key properties of relations:

ReflexiveSymmetricTransitive

A reflexive, symmetric, transitive relation is called anequivalence relation.Try the following: =, <, friend.


Graphs

An undirected graph (more simply, a graph) G = (V ,E ) is givenby a set V of vertices and a set E of edges.Each edge is of the form {v ,w}, for some vertices v ,w ∈ V .


Graphs

An undirected graph (more simply, a graph) G = (V ,E ) is givenby a set V of vertices and a set E of edges.Each edge is of the form {v ,w}, for some vertices v ,w ∈ V .

Example: G = (V ,E ) with

V = {a, b, c , d}

E = {(a, b), (a, c), (c , d), (a, d), (b, c)}

Picture of G :

a b

c d


Graphs (continued)

Definitions:

The degree of a vertex is the number of edges touching it

A path is a sequence of nodes connected by edges

A simple path does not repeat nodes

A cycle is a path that starts and ends at the same node

A simple cycle repeats only first and last node

A tree is a connected graph with no simple cycles


Strings and Languages

An alphabet is any nonempty finite set

Members of the alphabet are alphabet symbols

Σ1 = {0, 1}Σ2 ={a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z}Σ3 = {0, 1, a, b, c}

A string over an alphabet Σ is a finite sequence of symbolsfrom Σ.

0100 is a string from Σ1.cat is a string from Σ2.

Denote the set of all strings over Σ by Σ∗.


Strings and Languages (continued)

The length |w | of a string w is its number of symbols

The empty string ε has length 0.

If w ∈ Σ∗ has length n, then can write w = w1w2 . . .wn,where w1,w2, . . . ,wn ∈ Σ.

Strings can be concatenated

ab is string a concatenated with b.If x ∈ Σ∗, then

xk =

k times︷︸︸︷

x x . . . x

A language is a set of strings.


Boolean logic

Boolean logic is a mathematical system built around true

and false (alternatively, 0 and 1).

Here are the main boolean operators:

¬ (not/negation): ¬0 = 1 and ¬1 = 0.∧ (and/conjunction): 1 ∧ 1 = 1; else 0∨ (or/disjunction): 0 ∧ 0 = 0; else 1↔ (equality): 1 ↔ 1 = 1, 0 ↔ 0 = 1; else 0.

They may be defined via a truth table

Can prove equality using truth tables

Distributive lawsDeMorgan’s laws


Proofs

Proofs are a big part of this course

A proof is a convincing logical argumentProofs in this class need to be clear, but not very formalThe book’s proofs are often informal, using English

Types of proofs:

A ⇐⇒ B means “A if and only if B”:Prove A =⇒ B and prove B =⇒ A.“Proof” by counterexample (prove false)Proof by constructionProof by contradictionProof by induction


Proofs: Example 1

Prove that for every graph G , the sum of the degrees of all nodesis even (Ex. 0.19, pg. 18).

The book does not really say it, but this is a proof by

induction (on the number of edges).


Proofs: Example 1




A proof by induction means:

Showing it is true for the base case.


Proofs: Example 1




A proof by induction means:

Showing it is true for the base case.Showing thattruth for the case n =⇒ truth for the case n + 1.


Proofs: Example 1 (continued)

Base case: 0 edges in G means that the sum of the degrees iszero, which is even.



Base case: 0 edges in G means that the sum of the degrees iszero, which is even.

Induction step: Suppose that the sum of degrees is even forn edges; show that the sum of degrees is even for n+1 edges.

When you add an edge, it connects two vertices (may besame).Each vertex’s degree is increased by 1.Sum of degrees is increased by 2 overall.even number + 2 = even number (we will accept that for now).


Proofs: Example 2

Show that A ∪ B = A ∩ B for any two sets A and B

(Theorem 0.20, pg. 20).

We prove sets are equal by showing they have the sameelements

What proof technique do we use? Any ideas?


Proofs: Example 2

Show that A ∪ B = A ∩ B for any two sets A and B

(Theorem 0.20, pg. 20).

We prove sets are equal by showing they have the sameelements

What proof technique do we use? Any ideas?

Prove in each direction:

First prove forward direction.Then prove backwards direction.We will do in words, bu not as informal as it sounds



Forward direction (LHS =⇒ RHS)

Assume x ∈ A ∪ B

Then x 6∈ A ∪ B [defn of complement]Then x 6∈ A and x 6∈ B [defn of union]Then x ∈ A and x ∈ B [defn of complement]And so x ∈ A ∩ B [defn of intersection]






Backward direction (RHS =⇒ LHS)

Assume x ∈ A ∩ B

Then x ∈ A and x ∈ B [defn of intersection]Then x 6∈ A and x 6∈ B [defn of complement]Then x 6∈ A ∪ B [defn of union]And so x ∈ A ∪ B [defn of complement]






Backward direction (RHS =⇒ LHS)

Assume x ∈ A ∩ B

Then x ∈ A and x ∈ B [defn of intersection]Then x 6∈ A and x 6∈ B [defn of complement]Then x 6∈ A ∪ B [defn of union]And so x ∈ A ∪ B [defn of complement]

So we are done!


Proofs: Example 3

Prove that for every even number n > 2, there is a 3-regular graphwith n nodes (Theorem 0.22, pg. 21)(A graph is k-regular if every node has degree k .)We use a proof by construction

Many theorems say that a specific type of object exists. Oneway to do this is by constructing it.

May sound weird, but this is probably most common prooftechnique we use in this courseWe may be asked to show some property is true. We mayneed to construct a model making it clear that said propertyis true.



Can you construct such a graph for n = 4? n = 6? n = 8?Try now!



Can you construct such a graph for n = 4? n = 6? n = 8?Try now!

If you see a pattern, then generalize it; this is the proof.

Solution:

Place the nodes in a circle.Connect each node to its nearest neighbor; this gives us a2-regular graph.Now connect each node to the one opposite it.

Text’s description: More formal.

Why doesn’t this work if n is odd?


Proofs: Example 4

Jack sees Jill, who has come in from outside.Since Jill is not wet, he concludes it is not raining (Exercise 0.23,pg. 22).

This is a proof by contradiction.

Assume the opposite: Assume it is raining.

If it were raining, then Jill would be wet.

This contradicts Jill’s dryness.

This is the process; not a good example.


Proofs: Example 4

Jack sees Jill, who has come in from outside.Since Jill is not wet, he concludes it is not raining (Exercise 0.23,pg. 22).

This is a proof by contradiction.

Assume the opposite: Assume it is raining.

If it were raining, then Jill would be wet.

This contradicts Jill’s dryness.

This is the process; not a good example.(Maybe Jill left her umbrella at the door!)


Proofs: Example 5

Prove that n2 ≥ 2n for all integers n ≥ 2.Use proof by induction.

Base case (n = 2): 22 ≥ 2× 2?


Proofs: Example 5


Base case (n = 2): 22 ≥ 2× 2? Yes!


Proofs: Example 5


Base case (n = 2): 22 ≥ 2× 2? Yes!

Induction case: Assume true for n = m; must prove true forn = m + 1.


Proofs: Example 5


Base case (n = 2): 22 ≥ 2× 2? Yes!


So we start with the assumption that m2 ≥ 2m.


Proofs: Example 5


Base case (n = 2): 22 ≥ 2× 2? Yes!


So we start with the assumption that m2 ≥ 2m.We must then show that (m + 1)2 ≥ 2(m + 1).


Proofs: Example 5


Base case (n = 2): 22 ≥ 2× 2? Yes!



Hence we must show that m2 + 2m + 1 ≥ 2m + 2, i.e., thatm

2 ≥ 1.


Proofs: Example 5


Base case (n = 2): 22 ≥ 2× 2? Yes!




2 ≥ 1.But m2 ≥ 2m (ind. hyp.), and so need only show that 2m ≥ 1.


Proofs: Example 5


Base case (n = 2): 22 ≥ 2× 2? Yes!




2 ≥ 1.But m2 ≥ 2m (ind. hyp.), and so need only show that 2m ≥ 1.By hypothesis, m ≥ 2, and so 2m ≥ 1.


Proofs: Example 5


Base case (n = 2): 22 ≥ 2× 2? Yes!




2 ≥ 1.But m2 ≥ 2m (ind. hyp.), and so need only show that 2m ≥ 1.By hypothesis, m ≥ 2, and so 2m ≥ 1.Hence the induction step is okay.

We’re done!


cisc 4090: theory of computation - chapter 0 …a ﬁrst course in theory of computation generally...

Documents