1

CD5560

FABERFormal Languages, Automata

and Models of ComputationLecture 12

Mälardalen University

2007

2

Content

Chomsky Language Hierarchy Turing Machines DeterminismHaltingTM ExamplesStandard TMComputing Functions with TMCombining TMsTuring Thesis

3

The Language Hierarchy

*aRegular Languages

Context-Free Languagesnnba Rww

nnn cba ww?

**ba

?

4

Regular Languages**ba

Context-free languages

Unrestricted grammar languages}0:{ ncba nnn

}0:{ nba nn

}},{:{ bawww

}*},{:{ bawwwR

}0:{ ! nan }0:{2

nba nn

5

*aFA

Push-down Automatannba Rww

nnn cba ww

**ba

Languages accepted byTuring Machines

6

Turing Machines

7

............Tape

Read-Write headControl Unit

A Turing Machine

8

............

Read-Write head

No boundaries -- infinite length

The head moves Left or Right

The Tape

9

............

Read-Write head

1. Reads a symbol2. Writes a symbol3. Moves Left or Right

The head at each time step:

10

ExampleTime 0

............ a a cb

Time 1

............ a b k c

1. Reads a2. Writes k

3. Moves left

11

Time 1

............ a b k c

Time 2

............ a k cf

1. Reads b2. Writes f3. Moves Right

12

Head starts at the leftmost positionof the input string

............

Blank symbol

head

a b ca

Input string

The Input String

#####

13

1q 2qLba ,

Read WriteMove Left

1q 2qRba ,

Move Right

States & Transitions

14

Example

1q 2qRba ,

............ a b ca

Time 1

1qcurrent state

#####

15

............ # a b caTime 1

1q 2qRba ,

............ a b cbTime 2

1q

2q

# # # #

# # # # #

16

............ a b caTime 1

1q 2qLba ,

............ a b cbTime 2

1q

2q

Example

# # # # #

# # # # #

17

............ a b caTime 1

1q 2qRg,#

............ ga b cbTime 2

1q

2q

Example

# # # # #

# # # #

18

Determinism

1q

2qRba ,

Allowed Not Allowed

3qLdb ,

1q

2qRba ,

3qLda ,

No lambda transitions allowed in TM!

Turing Machines are deterministic

19

Partial Transition Function

1q

2qRba ,

3qLdb ,

............ a b ca

1q

Example

No transitionfor input symbol c

Allowed

# # # # #

20

Halting

The machine halts if there are no possible transitions to follow

21

Example

............ a b ca

1q

1q

2qRba ,

3qLdb ,

No possible transition

HALT!

# # # # #

22

Final States

1q 2q Allowed

1q 2q Not Allowed

• Final states have no outgoing transitions• In a final state the machine halts

23

Acceptance

Accept input If machine halts in a final state

Reject input

If machine halts in a non-final state or If machine enters an infinite loop

24

Turing Machine Example

A TM that accepts the language *aa

0q

Raa ,

L,##1q

25

Time 0 aa

0q

a

0q

Raa ,

1qL,##

# # # #

26

Time 1 aa

0q

a

0q

Raa ,

1q

# # # #

L,##

27

Time 2 aa

0q

a

0q

Raa ,

1q

# # # #

L,##

28

Time 3aa

0q

a

0q

Raa ,

1qL,##

# # # #

29

Time 4 aa

1q

a

0q

Raa ,

1q

Halt & Accept

L,##

# # # #

30

Rejection Example

Time 0 ba

0q

a

0q

Raa ,

1qL,##

# # # #

310q

Raa ,

1q

Time 1 ba

0q

a

No possible TransitionHalt & Reject

L,##

# # # #

32

Infinite Loop Example

0q

Raa ,

1q

Lbb ,

Another TM for language *aa

L,##

33

Time 0 ba

0q

a

0q

Raa ,

1q

Lbb ,

L,##

# # # #

34

Time 1 ba

0q

a

0q

Raa ,

1q

Lbb ,

L,##

# # # #

35

Time 2 ba

0q

a

0q

Raa ,

1q

Lbb ,

L,##

# # # #

36

Time 2 ba

0q

a

baTime 3

0q

a

baTime 4

0q

a

baTime 5

0q

a

... infinite loop…

# # # #

# # # #

# # # #

# # # #

37

Because of the infinite loop:

•The final state cannot be reached

•The machine never halts

•The input is not accepted

38

Another Turing Machine Example

Turing machine for the language }{ nnba

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

39

Time 0 ba

0q

a b

Ryy ,

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

4q

L,##

# # #}{ nnba

40

bx

1q

a bTime 1

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

41

bx

1q

a bTime 2

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

42

yx

2q

a bTime 3

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

43

yx

2q

a bTime 4

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

44

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

yx

0q

a bTime 5

L,##

# # #}{ nnba

45

yx

1q

x bTime 6

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

46

yx

1q

x bTime 7

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

47

yx x y

2q

Time 8

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

48

yx x y

2q

Time 9

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

49

Ryy ,

yx

0q

x yTime 10

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

4q

L,##

# # #}{ nnba

50

yx

3q

x yTime 11

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

51

yx

3q

x yTime 12

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,##

# # #}{ nnba

52

yx

4q

x y

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

Halt & Accept

Time 13

L,##

# # #}{ nnba

53

If we modify the machine for the language }{ nnba

we can easily construct a machine for the language }{ nnn cba

Observation

54

Formal Definitions for

Turing Machines

55

Transition Function

1q 2qRba ,

),,(),( 21 Rbqaq

56

1q 2qLdc ,

),,(),( 21 Ldqcq

Transition Function

57

Turing Machine

),#,,,,,( 0 FqQM

Transitionfunction

Initialstate

blank

Finalstates

States

Inputalphabet

Tapealphabet

58

Instantaneous description:

ba

1q

ac

baqca 1

Configuration

# # # #

59

Time 4 Time 5

yx

2q

a b yx

0q

a b

A move:aybqxxaybq 02

# # # # # #

60

Time 4 Time 5yx

2q

a b yx

0q

a b

bqxxyybqxxaybqxxaybq 1102

Time 6 Time 7yx

1q

x b yx

1q

x b

# # # # # #

# # # # # #

61

bqxxyxaybq 12

Equivalent notation:

bxxqybxxqaybxqxaybq 1102

62

Initial configuration: wq0

ba

0q

a b

wInput string

# # #

63

For any Turing Machine M

}:{)( 210 xqxwqwML f

Initial state Final state

The Accepted Language

64

Standard Turing Machine

• Deterministic

• Infinite tape in both directions

• Tape is the input/output file

The machine we described is the standard:

65

Computing Functionswith

Turing Machines

66

A function )(wf has:

Domain(domän):

DDw

Range(värdemängd):

S Swf )(

67

A function may have many parameters:

yxyxf ),(

Example: Addition function

68

Integer Domain

Binary: 101

Decimal: 5

We prefer unary representation:

easier to manipulate

Unary: 11111

69

Definition

A function is computable ifthere is a Turing Machine such that:

fM

Initial configuration Final configuration

Dw Domain

fq

)(wf

final state0q

w

initial state

For all

# # # #

70

)(0 wfqwq f

Initial Configuration

FinalConfiguration

In other words

Dw DomainFor all

A function is computable ifthere is a Turing Machine such that

fM

71

Example (Addition)

The function yxyxf ),( is computable

Turing Machine:

Input string: yx0 unary

Output string: 0xy unary

yx, are integers

72

Start

Finish 0

fq

11

yx

11

final state

0

0q

1 11 1

x y

1

initial state

# #

# #

73

0q 1q 2q 3qL,## L,01

L,11

R,##

R,10

R,11

4q

R,11

Turing machine for function yxyxf ),(

74

Execution Example:

11x

11y

Time 0

0

0q

1 11 1x y

Final Result

0

4q

1 11 1yx

(2)

(2)

# #

# #

75

Time 0 0

0q

1 11 1

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

76

0q

01 11 1Time 1

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

77

0

0q

1 11 1Time 2

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

78

1q

1 11 11Time 3

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

79

1q

1 11 11Time 4

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

80

1q

1 11 11Time 5

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

81

2q

1 11 11Time 6

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

82

3q

1 11 01Time 7

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

83

3q

1 11 01Time 8

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

84

3q

1 11 01Time 9

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

85

3q

1 11 01Time 10

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

86

3q

1 11 01Time 11

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

# #

L,##

R,##

yxyxf ),(

87

4q

1 11 01

0q 1q 2q 3qL,01

L,11

R,10

R,11

4q

R,11

HALT & accept

Time 12 # #

L,##

R,##

yxyxf ),(

88

Another Example (Multiplication)

The function xxf 2)( is computable

Turing Machine

Input string: x unaryOutput string: xx unary

x is integer

89

Start

Finish 1

fq

11

x2

11

final state

0q

11

x

1

initial state

# #

# #

90

Turing Machine Pseudocode for xxf 2)(

• Replace every 1 with $

• Repeat:

• Find rightmost $, replace it with 1• Go to right end, insert 1

Until no more $ remain

91

0q 1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

Turing Machine for xxf 2)(

92

0q

11

3q

1 11 1

Start Finish

# # # #

0q 1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

xxf 2)(

93

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

0q

1Time 0 # # # #

0q

1 xxf 2)(

94

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

0q

1$Time 1 # # # #

0q

xxf 2)(

95

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

0q

Time 2 # # # #

0q

$ $ xxf 2)(

96

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

1q

Time 3 # # # #

0q

$ $ xxf 2)(

97

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

2q

1Time 4 # # # #

0q

$ xxf 2)(

98

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

1q

1 1Time 5 # # #

0q

$ xxf 2)(

99

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

1q

1 1Time 6 # # #

0q

$ xxf 2)(

100

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

2q

1 1Time 7 # # #

0q

xxf 2)( 1

101

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

2q

1 1Time 8 # # #

0q

xxf 2)( 1

102

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

2q

1 1Time 9 # # #

0q

xxf 2)( 1

103

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 10 # #

0q

1q

1 1 xxf 2)(

104

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 11 # #

0q

1q

1 1 xxf 2)(

105

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 12 # #

0q

1q

1 1 xxf 2)(

106

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 13 # #

0q

1q

1 1 xxf 2)(

107

1q 2q

3q

R,1$

L,1#

L,##

R$,1 L,11 R,11

R,##

11Time 14 # #

0q

3q

1 1 xxf 2)(

108

Another Example

The function is computable

),( yxf0

1 yx

yx

if

if

Turing Machine for

Output: 1 0or

Input: yx0

109

Turing Machine Pseudocode

Match a 1 from with a 1 from x y• Repeat

Until all of or are matchedx y

• If a 1 from is not matched erase tape, write 1 else erase tape, write 0

x

110

Combining Turing Machines

111

Block Diagram

TuringMachine

input output

112

Example

),( yxf0

yx yx

yx

if

if

Comparer

Adder

Eraser

yx, yx

yx

yx

0

113

Turing’s Thesis

114

Do Turing machines have the same power with a digital computer?

Intuitive answer: Yes

There is no formal answer!

Question:

115

Turing’s thesis

Any computation carried outby mechanical meanscan be performed by a Turing Machine

(1930)

116

Computer Science Law

A computation is mechanical if and only ifit can be performed by a Turing Machine

There is no known model of computationmore powerful than Turing Machines

117

Definition of Algorithm

An algorithm for function f(w)is a Turing Machine which computes f(w)

118

When we say There exists an algorithm

Algorithms are Turing Machines

It meansThere exists a Turing Machine

119

Language Grammar MachineChomsky Hierarchy

Turing non-computable

Natural computing??

Recursively enumerable (RE) Unrestricted Turing machines 0

RecursiveTuring machines that always halt 0

Context-sensitiveContext-sensitive grammar

Linear-bounded automata 1

Context-freeContext-free grammar

Non-deterministic pushdown automata 2

RegularRegular expressions Finite state

automata 3

properly inclusive

Languages, Grammars, Machines

120

Regular LanguagesFINITE AUTOMATA

**ba

Context-free languagesPUSH-DOWN AUTOMATA

Unrestricted grammar languagesTURING MACHINES

}0:{ ncba nnn

}0:{ nba nn

}},{:{ bawww

}*},{:{ bawwwR

}0:{ ! nan }0:{2

nba nn