now our picture looks like turing machines

1

Turing Machines

And the languages they handle

Now our picture looks like

Regular Languages

FiniteLanguages

Deterministic Context Free Languages

Context Free Languages

We’re going to start to look atlanguages out here

Turing Machine

Accepting a string A string x is accepted by a TM, if

Starting in the initial configuration With x on the input tape The machine eventually ends up in an

accepting state.

I.e. q0x a* αpβ and p ∈F

Turing Machine

Running a Turing Machine The execution of a TM can result in 3

possible cases: The machine “halts” in an accepting state

(ACCEPT) The machine “halts” in a non-accepting state

(REJECT) The machine goes into an “infinite loop”

(REJECT but keeps us guessing!)

Turing Machine

Language accepted by a TM The language accepted by a TM is the set

of all input strings x on which the machinehalts in an accepting state.

Questions?

Computation with TuringMachines

Characteristic function of a set: For any language L, the characteristic function is

defined as: 1 if x ∈ L 0 otherwise

If a TM computes the characteristic function of alanguage, it will always halt with either a 1 or 0left on the tape

This is stronger than saying that x is accepted by the TMsince if x ∉ L, for a TM that accepts L, there is noguarantee that the the machine will halt.

2

TMs and languages

Two classes of languages that involveTMs: Languages, L accepted by TM

Halts on x ∈L, “rejects” on x ∉L

Languages, L recognized by a TM TM computes the characteristic function of L Halts on all x Give yes/no answer as to whether x ∈L

TMs and languages

Two classes of languages that involveTMs: A language is recursively enumerable if

there is a TM that accepts it A language is recursive if there is a TM that

recognizes it.

TMs and languages

First observation: Every recursive language is also recursively

enumerable Modify the TM that recognizes the language so

that it goes into a “nowhere state” just beforeplacing the 0 on the tape indicating that astring is not in the language.

TMs and languages

Recursive

Recursively

Enumerable

Is theresomething inhere

Is theresomething outhere

TMs and languages

Game plan1. Show that there exists a language that is

not recursively enumerable2. Show that there exists a language that is

recursively enumerable but not recursive.

A language that is not recursivelyenumerable

Sorry, no pumping Lemma for TMs

Instead we turn to thepower of contraction!

3



Show that there exists a language that is notrecursively enumerable The encoding provides each TM with a unique

“serial” number. That “serial number” can be considered the decimal

equivalent of the binary encoding of the TM Mi = the TM whose code is wi , wi represented in binary L(Mi) is the language accepted by Mi

Note: if wi is not a valid TM encoding, we define Mi to bethe TM with 1 state and no transitions. In this case L(Mi)= ∅.


So far… For i=0, 1, 2, …

wi is the binary representation of i Mi is the TM that has wi as it’s encoding L(Mi) is the language accepted by Mi

Diagonalization Language

Diagonalization Language (Ld) Set of all strings wi such that wi ∉L(Mi) All strings w that are not accepted by the

TM with w as it’s encoding. All encodings for TMs that don’t accept

their own encoding when input


Why is it called diagonalization? Consider all pairs (i,j).

i represents TM “serial numbers” j represents decimal representations of binary

string (i, j) represents if j is accepted by the TM with

“serial number” i.


Ld is the complement of the diagonal

4

The punch line

Ld is not recursively enumerable Proof by contradiction

Assume Ld was recursively enumerable thenthere exists a TM, M that accepts Ld. Let Mhave “serial number” i, for some i.

Ld = L (Mi)

Now ask: is wi in Ld?

Miwi

YES

NO

The punch line

YES? If yes, then

We just claimed that Mi will accept wi. This contradicts the definition of Ld. Ld = Set of all strings wi such that wi ∉L(Mi)

Miwi

YES

The punch line

NO? Then, we are claiming that Mi will not accept wi.

Ld = Set of all strings wi such that wi ∉L(Mi) Thus wi ∈ Ld

Can’t be both in Ld and not in Ld. CONTRADICTION (Nomad’s about to explode!)

Ld must not be recursively enumerable.

Miwi

NO


Questions?

Back to countability A language that is not recursively enumerable

It can be shown that the power set 2S of aninfinite set (I.e. the set of all subsets of S) isinfinitely uncountable.

More specifically, the power set of the set ofstrings composed of symbols from an alphabet Σ(what is a language again?) is infinitelyuncountable.

This power set is merely the set of all languagesover Σ.

Back to countability A language that is not recursively enumerable

Countably infinite sets are smaller thanuncountably infinite sets.

The set of TMs is countable The set of languages is uncountable. There are just not enough TMs to go around!

There are more languages than TMs, thus theremust be some languages that have no TMs thataccept them.

5

RE but not recursive

2. Show that there exists a languagethat is recursively enumerable but notrecursive.

Helps to show a nice fact about thecomplements of recursive languages.

Complements of RecursiveLanguages

If L is recursive, then L’, its complement isalso recursive If L is recursive, there exists a TM, M, that, on a

given input x Always halts Will leave 1 on the tape if x ∈ L Will leave 0 on the tape if x ∉ L.

Build a new TM, M’ that simulates M except Will leave 0 on the tape if x ∈ L Will leave 1 on the tape if x ∉ L.

M’ accepts L’

Universal Language

Universal Language (Lu) Set of all strings wi such that wi ∈L(Mi) All strings w that are accepted by the TM

with w as it’s encoding. All encodings for TMs that do accept their

encoding when input

Is Lu recursively enumerable?

Universal Language Is Lu recursively enumerable?

Is there a TM that will always halt for wi ∈L(Mi)

Yes! The Universal TM, Tu

Running Tu on wi Will halt if wi is accepted by the TM with encoding wi

Will reject otherwise.

Lu is recursively enumerable.

Universal Language Is Lu recursive? Assume that it is:

Then Lu’ is also recursive Lu = Set of all strings wi such that wi ∈L(Mi) Lu’ = Set of all strings wi such that wi ∉L(Mi) = Ld

We just showed Ld not to be recursively enumerable. Since all recursive languages are also recursively

enumerable, then Ld is certainly not recursive. Contradiction! Lu must not be recursive!

TMs and languages

Recursive

Recursively

Enumerable

Is theresomething inhere

Is theresomething outhere

YES – Lu

YES – Ld

6

Reality check

Recursive vs. Recursively Enumerable Language that is not RE


Language that is RE but not recursive Universal language.

Questions?

Now we have 2 pictures…thisone

Regular Languages

FiniteLanguages

Deterministic Context Free Languages

Context Free Languages

And this one…

Recursive

Recursively

Enumerable

How do these 2 relate

Unrestricted grammars

To answer this we’ll have to takeanother look at grammars.

Grammars

Let’s formalize this a bit: A grammar is a 4-tuple: (V, T, P, S) where

V is a set of variables T is a set of terminals P is a set of production rules V and T are disjoint (I.e. V ∩ T = ∅) S ∈V, is your start symbol

Grammars Let’s formalize this a bit:

Production rules We say that γ can be derived from α in one step:

A → β is a rule α = α1A α2

γ = α1 β α2

α ⇒ γ

We write α ⇒* γ if γ can be derived from α in zero or moresteps.

7

Unrestricted Grammars With unrestricted grammars, there is no

restriction on the length of the left hand sideof a production.

The only rule is that the left hand side mustcontain at least 1 varaible Example:

ABC → aB Ba → ACA aAa → b

Unrestricted grammar Example

L = { aibici | i ≥ 1 } note: this is not a CFL S → Α1BCS1 | A1BC (1) S1 → ABCS1 | ABC (2) BA → AB (3) CA → AC (4) CB → BC (5) cC → cc (6) bC → bc (7) bB → bb (8) aB → ab (9) aA → aa (10) Α1 → a (11)

Unrestricted grammar Derive aabbcc

S → Α1BCS1 (1) → Α1BCABCS (2) → Α1BCABCABC (2) → aBCABCABC (11) → aBACBCABC (4) → aABCBCABC (3) → aABCBACBC (4) → aABCABCBC (3) → aABACBCBC (4)


→ aABACBCBC → aAABCBCBC (3) → aAABBCCBC (5) → aAABBCBCC (5) → aAABBBCCC (5) → aaABBBCCC (10) → aaaBBBCCC (10) → aaabBBCCC (9)


→ aaabBBCCC → aaabbBCCC (8) → aaabbbCCC (8) → aaabbbcCC (7) → aaabbbccC (6) → aaabbbccc (6)

Questions?

Context Sensitive Grammar Context Sensitive Grammars

Productions α → β where α contains at least 1 variable And | α | ≤ | β |

A variable can only be replaced in the context of othersymbols

A language derived from a context sensitive grammar is acontext sensitive language.

The last example was a context sensitive language

8

Context Sensitive Grammar Do Context Sensitive Languages have a

corresponding machine ? Of course, all language classes do. Linear Bounded Automata

Like a TM except Has two additional symbols 〈 and 〉 The LBA’s starting configuration is (q0, 〈 x 〉 ) The machine cannot move left of the 〈 or right of the 〉 An LBA can only use n cells on the tape where n is the size of the

input string.

It can be shown that: Every Context Free Language is Context Sensitive

By definition of the grammars Every Context Sensitive Language is Recursive

Minor modification to turn an LBA into a TM that alwayshalts.

There is a recursive language that is not ContextSensitive One of those strange diagonal type languages. Captain Kirk → NOMAD → BOOM.

Our complete picture:

Regular

Finite

Deterministic Context Free

Context Free

Context Sensitive

RecursiveRecursively Enumerable

Context Sensitive Languages Fun facts

Context Sensitive Languages are closed under Union, Intersection, and Concatenation… If L is context sensitive, so is L+

Complement…Open question for quite some time Turns out to be yes.

Still unknown if every CSL can be accepted by aDETERMINISTIC Linear Bounded Automata.

Context Sensitive Languages

More fun facts Most “practical” languages are context

sensitive. Programming languages Spoken languages

It also can be shown: Every recursively enumerable language

can be generated by an unrestrictedgrammar.

In fact, Chomksy (the grammar guy),set out to define the four languageclasses: Regular, CF, CS, Recursively Enumerable By just using grammars.

9

Theory Hall of Fame Noam Chomsky

The Grammar Guy 1928 – b. Philadelphia, PA

PhD – UPenn (1955) Linguistics

Prof at MIT (Linguistics) (1955 -present)

Probably more famous for hisleftist political views.

Chomsky Hierarchy (1956,1959)

Questions?

now our picture looks like turing machines

Documents