csc312 automata theory lecture # 1 introduction

CSC312

Automata Theory

Lecture # 1

Introduction

Administrative Stuff

• Instructor: Dr. Mudasser Naseer

mnaseer@ciitahore.edu.pkCabin # 1, Faculty Room C7

• Lectures: Sec-A: Tue 16:30 (C-6), Wed 16:30 (C-11) Sec-B: Mon 11:30 (C-13), Wed 15:00 (C-11)

•Office Hrs: Tue & Thu 1400 – 1600 hrs

(or by appointment)

•Prerequisite: CSC102 - Discrete Structures

Course Objectives:

To study and mathematically model various abstract computing machines that serve as models for computations and examine the relationship between these automata and formal languages.

Course OutlineRegular expressions, NFAs. Core concepts of Regular Languages and Finite Automata; Decidability for Regular Languages; Non-regular Languages; Context-free Languages and Pushdown Automata; Decidability for Context-free Languages; Non-context-free Languages; Turing Machines and Their Languages are important part of the course. Transducers (automata with output).

Course OrganizationText Book:i) Denial I. A. Cohen Introduction to Computer Theory, Second Edition, John Wiley & Sons, 1997.

Reference Books:

i) J. E. Hopcroft, R. Motwani, & J. D. UllmanIntroduction to Automata Theory,Languages, and Computation, Third Edition,Pearson, 2008.

Instruments:There will be 2~3 assignments, 4~5 quizzes, Weights: Assignments 10%

Quizzes 15%S-I 10%

S-II 15%Final Exam 50%

Schedule of LecturesLect.# Topics/Contents

1 Introduction to Automata theory, Its background, Mathematical Preliminaries, Sets, Functions, Graphs, Proof Techniques

2 Formal Languages, Introduction to defining languages, alphabet, language, word, null string, length of a string, reverse of a string, Palindrome, Kleene closure.

3 Formal definition of Regular Expressions, Defining languages with regular expressions, Languages associated with regular expressions.

4 Equality of Regular Expressions, Introducing the language EVEN-EVEN.

5 More examples related to regular expressions.

6 Introducing Finite Automata., Defining languages using Finite Automata. Constructing Finite Automata for different languages.

7 Recognizing the language defined by the given Finite Automata.

8 More examples related to Finite Automata.

9 Transition Graphs with examples, Generalized Transition Graphs, Non-determinism in case of Transition Graphs.

10 Non-deterministic FA’s. Differences between FA, TG and NFA.

11 Sessional I

Schedule of LecturesLect.# Topics/Contents

12 Kleene’s Theorem, Algorithm for turning TGs into REs

13 Kleene’s Theorem, Algorithm for turning REs into FAs

14 Nondeterminism, NFA, converting NFA into FA.

15 Finite Automata with output, Moore’s machines and Mealy machines with examples. 1’s Complement machine, Increment machine.

16 Theorems for Converting Moore machines into Mealy machines and vice versa. Transducers as models of sequential circuits.

17 Regular Languages, Closure properties (i.e. , Concatenation and Kleene closure) of Regular Languages with examples.

18 Complements and Intersections of Regular Languages, Theorems relating to regular languages and the related examples.

19 Non-Regular Languages, The pumping Lemma, Examples relating to Pumping Lemma.

20 Decidability, decision procedure, Blue-paint method, Effective decision procedure to prove whether two given RE’s or FA’s are equivalent. Myhill-Nerode theorem, Related Examples.

21 Sessional II

Schedule of LecturesLect.# Topics/Contents

22 Context-Free Grammars, CFG’s for Regular Languages with examples. CFG’s for non-regular languages.

23 CFG’s of PALINDROME, EQUAL and EVEN-EVEN languages, Backus-Naur Form.

24 Parse Trees, Examples relating to Parse Trees, Lukasiewicz notation, Prefix and Postfix notations and their evaluation.

25 Ambiguous and Unambiguous CFG’s, Syntax tree, Total language tree.

26 Regular Grammars, Semi-word, Word, Working String, Converting FA’s into CFG’s. Constructing Transition Graphs from Regular Grammars.

27 Killing null productions. Killing unit productions, Chomsky Normal form with examples, Left most derivations.

28 Pushdown Automata, Constructing PDA’s for FA’s, Pushdown stack.

29 Examples related with PDA, PDA for Odd Palindrome, Even Palindrome, PalindromeX.

30 Nondeterministic PDA. Proving CFG = PDA with examples.

31 Constructing PDA from CFG in CNF with examples

32 Turing machines, Examples of Turing Machines with trace tables, Converting FA’s into Turing machines.

Some basics• Automaton = A self-operating machine or

mechanism (Dictionary definition), plural is Automata.

• Automata = abstract computing devices• Automata theory = the study of abstract

machines (or more appropriately, abstract 'mathematical' machines or systems, and the computational problems that can be solved using these machines.• Mathematical models of computation• Finite automata• Push-down automata• Turing machines

History• 1930s : Alan Turing defined machines more

powerful than any in existence, or even any that we could imagine – Goal was to establish the boundary between what was and was not computable.

• 1940s/150s : In an attempt to model “Brain function” researchers defined finite state machines.

• Late 1950s : Linguist Noam Chomsky began the study of Formal Grammars.

• 1960s : A convergence of all this into a formal theory of computer science, with very deep philosophical implications as well as practical applications (compilers, web searching, hardware, A.I., algorithm design, software engineering,…)

Courtesy Costas Busch - RPI 11

Computation

CPU memory

Courtesy Costas Busch - RPI 12

CPU

input memory

output memory

Program memory

temporary memory

Courtesy Costas Busch - RPI 13

CPU

input memory

output memoryProgram memory

temporary memory

3)( xxf

compute xx

compute xx 2

Example:

Courtesy Costas Busch - RPI 14

CPU

input memory

output memoryProgram memory

temporary memory

3)( xxf

compute xx

compute xx 2

2x

Courtesy Costas Busch - RPI 15

CPU

input memory

output memoryProgram memory

temporary memory3)( xxf

compute xx

compute xx 2

2x

42*2 z82*)( zxf

Courtesy Costas Busch - RPI 16

CPU

input memory

output memoryProgram memory

temporary memory3)( xxf

compute xx

compute xx 2

2x

42*2 z82*)( zxf

8)( xf

Courtesy Costas Busch - RPI 17

Automaton

CPU

input memory

output memory

Program memory

temporary memory

Automaton

Courtesy Costas Busch - RPI 18

Different Kinds of Automata

Automata are distinguished by the temporary memory

• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory

Courtesy Costas Busch - RPI 19

input memory

output memory

temporary memory

Finite

Automaton

Finite Automaton

Example: Vending Machines

(small computing power)

Courtesy Costas Busch - RPI 20

input memory

output memory

Stack

Pushdown

Automaton

Pushdown Automaton

Example: Compilers for Programming Languages

(medium computing power)

Push, Pop

Courtesy Costas Busch - RPI 21

input memory

output memory

Random Access Memory

Turing

Machine

Turing Machine

Examples: Any Algorithm

(highest computing power)

Courtesy Costas Busch - RPI 22

Finite

Automata

Pushdown

Automata

Turing

Machine

Power of Automata

Less power More power

Solve more

computational problems

Courtesy Costas Busch - RPI 23

Mathematical Preliminaries

• Sets

• Functions

• Relations

• Graphs

• Proof Techniques

Courtesy Costas Busch - RPI 24

}3,2,1{AA set is a collection of elements

SETS

},,,{ airplanebicyclebustrainB

We write

A1

Bship

Courtesy Costas Busch - RPI 25

Set Representations

C = { a, b, c, d, e, f, g, h, i, j, k }

C = { a, b, …, k }

S = { 2, 4, 6, … }

S = { j : j > 0, and j = 2k for some k>0 }

S = { j : j is nonnegative and even }

finite set

infinite set

Courtesy Costas Busch - RPI 26

A = { 1, 2, 3, 4, 5 }

Universal Set: all possible elements U = { 1 , … , 10 }

1 2 3

4 5

A

U

6

7

8

910

Courtesy Costas Busch - RPI 27

Set Operations

A = { 1, 2, 3 } B = { 2, 3, 4, 5}

• Union

A U B = { 1, 2, 3, 4, 5 }

• Intersection

A B = { 2, 3 }

• Difference

A - B = { 1 }

B - A = { 4, 5 }

U

A B2

31

4

5

2

3

1

Venn diagrams

Courtesy Costas Busch - RPI 28

A

• Complement

Universal set = {1, …, 7}

A = { 1, 2, 3 } A = { 4, 5, 6, 7}

12

3

4

5

6

7

A

A = A

Courtesy Costas Busch - RPI 29

02

4

6

1

3

5

7

even

{ even integers } = { odd integers }

odd

Integers

Courtesy Costas Busch - RPI 30

DeMorgan’s Laws

A U B = A B

U

A B = A U BU

Courtesy Costas Busch - RPI 31

Empty, Null Set:= { }

S U = S

S =

S - = S

- S =

U= Universal Set

Courtesy Costas Busch - RPI 32

Subset

A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }

A B

U

Proper Subset: A B

UA

B

Courtesy Costas Busch - RPI 33

Disjoint Sets

A = { 1, 2, 3 } B = { 5, 6}

A B =

UA B

Courtesy Costas Busch - RPI 34

Set Cardinality

• For finite sets

A = { 2, 5, 7 }

|A| = 3

(set size)

Courtesy Costas Busch - RPI 35

Powersets

A powerset is a set of sets

Powerset of S = the set of all the subsets of S

S = { a, b, c }

2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }

Observation: | 2S | = 2|S| ( 8 = 23 )

Courtesy Costas Busch - RPI 36

Cartesian Product

A = { 2, 4 } B = { 2, 3, 5 }

A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }

|A X B| = |A| |B|

Generalizes to more than two sets

A X B X … X Z

Courtesy Costas Busch - RPI 37

FUNCTIONSdomain

12

3

a

bc

range

f : A -> B

A B

If A = domain

then f is a total function

otherwise f is a partial function

f(1) = a4

5

Courtesy Costas Busch - RPI 38

Let A & B be sets. A binary relation “R” from A to B

R = {(x1, y1), (x2, y2), (x3, y3), …}

Where and

R ⊆ A x B

xi R yi to denote

e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1

RELATIONS

Axi Byi

Rba ),(

Courtesy Costas Busch - RPI 39

Equivalence Relations

• Reflexive: x R x

• Symmetric: x R y y R x

• Transitive: x R y and y R z x R z

Example: R = ‘=‘

• x = x

• x = y y = x

• x = y and y = z x = z

Courtesy Costas Busch - RPI 40

GRAPHSA directed graph

• Nodes (Vertices)

V = { a, b, c, d, e }

• Edges

E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }

node

edge

a

b

c

d

e

Courtesy Costas Busch - RPI 41

Labeled Graph

a

b

c

d

e

1 3

56

26

2

Courtesy Costas Busch - RPI 42

Walk

a

b

c

d

e

Walk is a sequence of adjacent edges

(e, d), (d, c), (c, a)

Courtesy Costas Busch - RPI 43

Path

a

b

c

d

e

Path is a walk where no edge is repeated

Simple path: no node is repeated

Courtesy Costas Busch - RPI 44

Cycle

a

b

c

d

e

12

3

Cycle: a walk from a node (base) to itself

Simple cycle: only the base node is repeated

base

Courtesy Costas Busch - RPI 45

Euler Tour

a

b

c

d

e1

23

45

6

7

8 base

A cycle that contains each edge once

Courtesy Costas Busch - RPI 46

Hamiltonian Cycle

a

b

c

d

e1

23

4

5 base

A simple cycle that contains all nodes

Courtesy Costas Busch - RPI 47

Treesroot

leaf

parent

child

Trees have no cycles

Courtesy Costas Busch - RPI 48

root

leaf

Level 0

Level 1

Level 2

Level 3

Height 3

Courtesy Costas Busch - RPI 49

Binary Trees

Courtesy Costas Busch - RPI 50

PROOF TECHNIQUES

• Proof by induction

• Proof by contradiction

Courtesy Costas Busch - RPI 51

Induction

We have statements P1, P2, P3, …

If we know

• for some b that P1, P2, …, Pb are true

• for any k >= b that

P1, P2, …, Pk imply Pk+1

Then

Every Pi is true

Courtesy Costas Busch - RPI 52

Proof by Induction• Inductive basis

Find P1, P2, …, Pb which are true

• Inductive hypothesis

Let’s assume P1, P2, …, Pk are true,

for any k >= b

• Inductive step

Show that Pk+1 is true

Courtesy Costas Busch - RPI 53

Example

Theorem: A binary tree of height n

has at most 2n leaves.

Proof by induction:

let L(i) be the maximum number of

leaves of any subtree at height i

Courtesy Costas Busch - RPI 54

We want to show: L(i) <= 2i

• Inductive basis

L(0) = 1 (the root node)

• Inductive hypothesis

Let’s assume L(i) <= 2i for all i = 0, 1, …, k

• Induction step

we need to show that L(k + 1) <= 2k+1

Courtesy Costas Busch - RPI 55

Induction Step

From Inductive hypothesis: L(k) <= 2k

height

k

k+1

Courtesy Costas Busch - RPI 56

L(k) <= 2k

L(k+1) <= 2 * L(k) <= 2 * 2k = 2k+1

Induction Step

height

k

k+1

(we add at most two nodes for every leaf of level k)

Courtesy Costas Busch - RPI 57

Remark

Recursion is another thing

Example of recursive function:

f(n) = f(n-1) + f(n-2)

f(0) = 1, f(1) = 1

Courtesy Costas Busch - RPI 58

Proof by Contradiction

We want to prove that a statement P is true

• we assume that P is false

• then we arrive at an incorrect conclusion

• therefore, statement P must be true

Courtesy Costas Busch - RPI 59

Example

Theorem: is not rational

Proof:

Assume by contradiction that it is rational

= n/m

n and m have no common factors

We will show that this is impossible

2

2

60

= n/m 2 m2 = n2

Therefore, n2 is evenn is even

n = 2 k

2 m2 = 4k2 m2 = 2k2m is even

m = 2 p

Thus, m and n have common factor 2

Contradiction!

2