csc312 automata theory lecture # 1 introduction. administrative stuff instructor: dr. mudasser...

CSC312

Automata Theory

Lecture # 1

Introduction

Administrative Stuff

• Instructor: Dr. Mudasser Naseer

[email protected] # 1, Faculty Room C7

• Lectures: Sec-A: Tue 16:30 (C-6), Wed 16:30 (C-11) Sec-B: Mon 11:30 (C-13), Wed 15:00 (C-11)

•Office Hrs: Tue & Thu 1400 – 1600 hrs

(or by appointment)

•Prerequisite: CSC102 - Discrete Structures

Course Objectives:

To study and mathematically model various abstract computing machines that serve as models for computations and examine the relationship between these automata and formal languages.

Course OutlineRegular expressions, NFAs. Core concepts of Regular Languages and Finite Automata; Decidability for Regular Languages; Non-regular Languages; Context-free Languages and Pushdown Automata; Decidability for Context-free Languages; Non-context-free Languages; Turing Machines and Their Languages are important part of the course. Transducers (automata with output).

Course OrganizationText Book:i) Denial I. A. Cohen Introduction to Computer Theory, Second Edition, John Wiley & Sons, 1997.

Reference Books:

i) J. E. Hopcroft, R. Motwani, & J. D. UllmanIntroduction to Automata Theory,Languages, and Computation, Third Edition,Pearson, 2008.

Instruments:There will be 2~3 assignments, 4~5 quizzes, Weights: Assignments 10%

Quizzes 15%S-I 10%

S-II 15%Final Exam 50%

Schedule of LecturesLect.# Topics/Contents

1 Introduction to Automata theory, Its background, Mathematical Preliminaries, Sets, Functions, Graphs, Proof Techniques

2 Formal Languages, Introduction to defining languages, alphabet, language, word, null string, length of a string, reverse of a string, Palindrome, Kleene closure.

3 Formal definition of Regular Expressions, Defining languages with regular expressions, Languages associated with regular expressions.

4 Equality of Regular Expressions, Introducing the language EVEN-EVEN.

5 More examples related to regular expressions.

6 Introducing Finite Automata., Defining languages using Finite Automata. Constructing Finite Automata for different languages.

7 Recognizing the language defined by the given Finite Automata.

8 More examples related to Finite Automata.

9 Transition Graphs with examples, Generalized Transition Graphs, Non-determinism in case of Transition Graphs.

10 Non-deterministic FA’s. Differences between FA, TG and NFA.

11 Sessional I


12 Kleene’s Theorem, Algorithm for turning TGs into REs

13 Kleene’s Theorem, Algorithm for turning REs into FAs

14 Nondeterminism, NFA, converting NFA into FA.

15 Finite Automata with output, Moore’s machines and Mealy machines with examples. 1’s Complement machine, Increment machine.

16 Theorems for Converting Moore machines into Mealy machines and vice versa. Transducers as models of sequential circuits.

17 Regular Languages, Closure properties (i.e. , Concatenation and Kleene closure) of Regular Languages with examples.

18 Complements and Intersections of Regular Languages, Theorems relating to regular languages and the related examples.

19 Non-Regular Languages, The pumping Lemma, Examples relating to Pumping Lemma.

20 Decidability, decision procedure, Blue-paint method, Effective decision procedure to prove whether two given RE’s or FA’s are equivalent. Myhill-Nerode theorem, Related Examples.

21 Sessional II


22 Context-Free Grammars, CFG’s for Regular Languages with examples. CFG’s for non-regular languages.

23 CFG’s of PALINDROME, EQUAL and EVEN-EVEN languages, Backus-Naur Form.

24 Parse Trees, Examples relating to Parse Trees, Lukasiewicz notation, Prefix and Postfix notations and their evaluation.

25 Ambiguous and Unambiguous CFG’s, Syntax tree, Total language tree.

26 Regular Grammars, Semi-word, Word, Working String, Converting FA’s into CFG’s. Constructing Transition Graphs from Regular Grammars.

27 Killing null productions. Killing unit productions, Chomsky Normal form with examples, Left most derivations.

28 Pushdown Automata, Constructing PDA’s for FA’s, Pushdown stack.

29 Examples related with PDA, PDA for Odd Palindrome, Even Palindrome, PalindromeX.

30 Nondeterministic PDA. Proving CFG = PDA with examples.

31 Constructing PDA from CFG in CNF with examples

32 Turing machines, Examples of Turing Machines with trace tables, Converting FA’s into Turing machines.

Some basics• Automaton = A self-operating machine or

mechanism (Dictionary definition), plural is Automata.

• Automata = abstract computing devices• Automata theory = the study of abstract

machines (or more appropriately, abstract 'mathematical' machines or systems, and the computational problems that can be solved using these machines.• Mathematical models of computation• Finite automata• Push-down automata• Turing machines

History• 1930s : Alan Turing defined machines more

powerful than any in existence, or even any that we could imagine – Goal was to establish the boundary between what was and was not computable.

• 1940s/150s : In an attempt to model “Brain function” researchers defined finite state machines.

• Late 1950s : Linguist Noam Chomsky began the study of Formal Grammars.

• 1960s : A convergence of all this into a formal theory of computer science, with very deep philosophical implications as well as practical applications (compilers, web searching, hardware, A.I., algorithm design, software engineering,…)

Courtesy Costas Busch - RPI 11

Computation

CPU memory


CPU

input memory

output memory

Program memory

temporary memory


CPU

input memory

output memoryProgram memory

temporary memory

3)( xxf

compute xx

compute xx 2

Example:


CPU

input memory


temporary memory

3)( xxf

compute xx

compute xx 2

2x


CPU

input memory


temporary memory3)( xxf

compute xx

compute xx 2

2x

42*2 z82*)( zxf


CPU

input memory


temporary memory3)( xxf

compute xx

compute xx 2

2x

42*2 z82*)( zxf

8)( xf


Automaton

CPU

input memory

output memory

Program memory

temporary memory

Automaton


Different Kinds of Automata

Automata are distinguished by the temporary memory

• Finite Automata: no temporary memory

• Pushdown Automata: stack

• Turing Machines: random access memory


input memory

output memory

temporary memory

Finite

Automaton

Finite Automaton

Example: Vending Machines

(small computing power)


input memory

output memory

Stack

Pushdown

Automaton

Pushdown Automaton

Example: Compilers for Programming Languages

(medium computing power)

Push, Pop


input memory

output memory

Random Access Memory

Turing

Machine

Turing Machine

Examples: Any Algorithm

(highest computing power)


Finite

Automata

Pushdown

Automata

Turing

Machine

Power of Automata

Less power More power

Solve more

computational problems


Mathematical Preliminaries

• Sets

• Functions

• Relations

• Graphs

• Proof Techniques


}3,2,1{AA set is a collection of elements

SETS

},,,{ airplanebicyclebustrainB

We write

A1

Bship


Set Representations

C = { a, b, c, d, e, f, g, h, i, j, k }

C = { a, b, …, k }

S = { 2, 4, 6, … }

S = { j : j > 0, and j = 2k for some k>0 }

S = { j : j is nonnegative and even }

finite set

infinite set


A = { 1, 2, 3, 4, 5 }

Universal Set: all possible elements U = { 1 , … , 10 }

1 2 3

4 5

A

U

6

7

8

910


Set Operations

A = { 1, 2, 3 } B = { 2, 3, 4, 5}

• Union

A U B = { 1, 2, 3, 4, 5 }

• Intersection

A B = { 2, 3 }

• Difference

A - B = { 1 }

B - A = { 4, 5 }

U

A B2

31

4

5

2

3

1

Venn diagrams


A

• Complement

Universal set = {1, …, 7}

A = { 1, 2, 3 } A = { 4, 5, 6, 7}

12

3

4

5

6

7

A

A = A


02

4

6

1

3

5

7

even

{ even integers } = { odd integers }

odd

Integers


DeMorgan’s Laws

A U B = A B

U

A B = A U BU


Empty, Null Set:= { }

S U = S

S =

S - = S

- S =

U= Universal Set


Subset

A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }

A B

U

Proper Subset: A B

UA

B


Disjoint Sets

A = { 1, 2, 3 } B = { 5, 6}

A B =

UA B


Set Cardinality

• For finite sets

A = { 2, 5, 7 }

|A| = 3

(set size)


Powersets

A powerset is a set of sets

Powerset of S = the set of all the subsets of S

S = { a, b, c }

2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }

Observation: | 2S | = 2|S| ( 8 = 23 )


Cartesian Product

A = { 2, 4 } B = { 2, 3, 5 }

A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }

|A X B| = |A| |B|

Generalizes to more than two sets

A X B X … X Z


FUNCTIONSdomain

12

3

a

bc

range

f : A -> B

A B

If A = domain

then f is a total function

otherwise f is a partial function

f(1) = a4

5


Let A & B be sets. A binary relation “R” from A to B

R = {(x1, y1), (x2, y2), (x3, y3), …}

Where and

R ⊆ A x B

xi R yi to denote

e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1

RELATIONS

Axi Byi

Rba ),(


Equivalence Relations

• Reflexive: x R x

• Symmetric: x R y y R x

• Transitive: x R y and y R z x R z

Example: R = ‘=‘

• x = x

• x = y y = x

• x = y and y = z x = z


GRAPHSA directed graph

• Nodes (Vertices)

V = { a, b, c, d, e }

• Edges

E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }

node

edge

a

b

c

d

e


Labeled Graph

a

b

c

d

e

1 3

56

26

2


Walk

a

b

c

d

e

Walk is a sequence of adjacent edges

(e, d), (d, c), (c, a)


Path

a

b

c

d

e

Path is a walk where no edge is repeated

Simple path: no node is repeated


Cycle

a

b

c

d

e

12

3

Cycle: a walk from a node (base) to itself

Simple cycle: only the base node is repeated

base


Euler Tour

a

b

c

d

e1

23

45

6

7

8 base

A cycle that contains each edge once


Hamiltonian Cycle

a

b

c

d

e1

23

4

5 base

A simple cycle that contains all nodes


Treesroot

leaf

parent

child

Trees have no cycles


root

leaf

Level 0

Level 1

Level 2

Level 3

Height 3


Binary Trees


PROOF TECHNIQUES

• Proof by induction

• Proof by contradiction


Induction

We have statements P1, P2, P3, …

If we know

• for some b that P1, P2, …, Pb are true

• for any k >= b that

P1, P2, …, Pk imply Pk+1

Then

Every Pi is true


Proof by Induction• Inductive basis

Find P1, P2, …, Pb which are true

• Inductive hypothesis

Let’s assume P1, P2, …, Pk are true,

for any k >= b

• Inductive step

Show that Pk+1 is true


Example

Theorem: A binary tree of height n

has at most 2n leaves.

Proof by induction:

let L(i) be the maximum number of

leaves of any subtree at height i


We want to show: L(i) <= 2i

• Inductive basis

L(0) = 1 (the root node)

• Inductive hypothesis

Let’s assume L(i) <= 2i for all i = 0, 1, …, k

• Induction step

we need to show that L(k + 1) <= 2k+1


Induction Step

From Inductive hypothesis: L(k) <= 2k

height

k

k+1


L(k) <= 2k

L(k+1) <= 2 * L(k) <= 2 * 2k = 2k+1

Induction Step

height

k

k+1

(we add at most two nodes for every leaf of level k)


Remark

Recursion is another thing

Example of recursive function:

f(n) = f(n-1) + f(n-2)

f(0) = 1, f(1) = 1


Proof by Contradiction

We want to prove that a statement P is true

• we assume that P is false

• then we arrive at an incorrect conclusion

• therefore, statement P must be true


Example

Theorem: is not rational

Proof:

Assume by contradiction that it is rational

= n/m

n and m have no common factors

We will show that this is impossible

2

2

60

= n/m 2 m2 = n2

Therefore, n2 is evenn is even

n = 2 k

2 m2 = 4k2 m2 = 2k2m is even

m = 2 p

Thus, m and n have common factor 2

Contradiction!

2

csc312 automata theory lecture # 1 introduction. administrative stuff instructor: dr. mudasser...

Documents

different languages

finite automata decidability

transducers automata

given finite automata

related examples

pushdown automata decidability

mealy machines

moores machines