1 introduction to computability theory lecture11: the halting problem prof. amos israeli
Post on 22-Dec-2015
227 views
TRANSCRIPT
![Page 1: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/1.jpg)
1
Introduction to Computability Theory
Lecture11: The Halting Problem
Prof. Amos Israeli
![Page 2: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/2.jpg)
In this lecture we present an undecidable language.
The language that we prove to be undecidable is a very natural language namely the language consisting of pairs of the form where Mis a TM accepting string w:
The Halting Problem
2
wMwMATM acceptingTM a is ,
wM ,
![Page 3: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/3.jpg)
Since this language requires to decide whether the computation of TM M halts on input w, it is often called The Halting Problem.
Theorem
The halting problem is Turing Recognizable.
The Halting Problem
3
![Page 4: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/4.jpg)
Consider a TM U that gets a pair as input and simulates the run of M on input w. If M accepts or rejects so does U. Otherwise, U loops.
Note: U recognizes ATM ,since it accepts any pair
, that is: any pair in which M accepts input w.
Proof
4
wM ,
LwM ,
![Page 5: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/5.jpg)
On the previous lecture, we detailed the simulation of a DFA by a TM.
Simulating one TM by another, using the encoding of the first TM is a very similar process. In the next slide we review the main characteristics of TM N simulating TM M, using M’s encoding <M>.
Simulating an Input TM
5
![Page 6: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/6.jpg)
TM N works as follows:
1. Mark M’s initial state and w’s initial symbol as the “current state” and “current head location”.
2. Look for M’s next transition on the description of its transition function.
3. Execute M’s transition.
Simulating an Input TM
6
![Page 7: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/7.jpg)
4. Move M’s “current state” and “current head location” to their new places.
5. If M’s new state is a deciding state decide according to the state, otherwise – repeat stages 2-5.
Simulating an Input TM
7
![Page 8: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/8.jpg)
Our goal in this lecture is to prove that ATM is
not decidable. The proof uses a common mathematical technique known as Diagonalization.
Diagonalization was first used by Cantor when he found a way to distinguish between several types of infinite sets.
Diagonalization
8
![Page 9: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/9.jpg)
Cantor dealt with questions like:
How many natural numbers are there? Infinity!
How many real numbers are there? Infinity!
Does the amount of natural numbers equal to the amount of real numbers?
How is the size of infinite sets measured?
Cardinality
9
![Page 10: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/10.jpg)
Cantor’s answer to these question was the notion of Cardinality.
The cardinality of a set is a property marking its size.
Two sets has the same cardinality if there is a correspondence between their elements
Cardinality
10
![Page 11: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/11.jpg)
At this point of the lecture, think about a correspondence between sets A and B as 2 lists: A list of A’s elements and in parallel a list of B’s elements. These 2 lists are juxtaposed so that each element of A corresponds to a unique element of B.
Intuitive Notion of Correspondence
11
![Page 12: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/12.jpg)
Clearly, the cardinality of A is equal to the cardinality of B.
How about the cardinality of infinite sets?
Example
12
4,3,2,1A 8,6,4,2B8642
4321
B
A
![Page 13: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/13.jpg)
How about the cardinality of infinite sets?
Is the cardinality of natural numbers larger than the cardinality of even natural numbers?
Intuitively, the cardinality of any set should be larger that the cardinality of any of its proper subsets. Alas, our intuition of sets is driven by our daily experience with finite sets.
Example
13
![Page 14: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/14.jpg)
So let us try to create a correspondence between the natural numbers the even natural numbers?
Indeed defines the wanted correspondence between the 2 sets.
Example
14
,...,...,4,3,2,1 nN
,...2,...,8,6,4,2 nEN ...2...642
......321
nEN
nN
nnf 2
![Page 15: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/15.jpg)
So the cardinality of N is equal to the cardinality of EN.
Example
15
,...,...,4,3,2,1 nN
,...2,...,8,6,4,2 nEN ...2...642
......321
nEN
nN
![Page 16: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/16.jpg)
This last example suggests the notion of Countable Sets:
A set A is countable if it is either finite or its cardinality is equal to the cardinality of N.
A cool way of looking at countable sets is:“A set is countable if a list of its elements can be created”.
Countable Sets
16
![Page 17: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/17.jpg)
“A set is countable if a list of its elements can be created”.
Note: This list does not have to be finite, but for each natural number i, one should be able to specify the i-th element on the list.
For example, for the set EN the i-th element on the list is 2i .
Countable Sets
17
![Page 18: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/18.jpg)
We just proved that EN, the set of even natural numbers is countable. What about the set of rational numbers?
Is the set Q of rational numbers countable?
Can its elements be listed?
Countable Sets
18
![Page 19: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/19.jpg)
Theorem
The set of rational numbers is countable.
Proof
In order to prove this theorem we have to show how a complete list of the rational numbers can be formed.
The set of Rationals is Countable
19
![Page 20: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/20.jpg)
Recall that each natural number is defined by a pair of natural numbers.
One way to look
at the Rationals
is by listing them
in an infinite Rectangle.
The set of Rationals is Countable
20
5/54/53/52/51/5
5/4
5/3
5/2
5/1
4/43/42/41/4
4/33/32/31/3
4/23/22/21/2
4/13/12/11/1 ………
………………………
………
………………………………………………...
![Page 21: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/21.jpg)
How can we form a list including all these numbers?
If we first list
The first row –
We will never
reach the second.
The set of Rationals is Countable
21
5/54/53/52/51/5
5/4
5/3
5/2
5/1
4/43/42/41/4
4/33/32/31/3
4/23/22/21/2
4/13/12/11/1 ………
………………………
………
………………………………………………...
![Page 22: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/22.jpg)
5/54/53/52/51/5
5/4
5/3
5/2
5/1
4/43/42/41/4
4/33/32/31/3
4/23/22/21/2
4/13/12/11/1
The set of Rationals is Countable
22
One way to do it is to start from the upper left corner,and continue in this fashion
![Page 23: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/23.jpg)
Note that some rational numbers appear more than once. For example: all numbers on the main diagonal are equal to 1, so this list is not final.
In order to compute the actual place of a given rational, we need to erase all duplicates, but this is a technicality…
The set of Rationals is Countable
23
![Page 24: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/24.jpg)
Can you think of any infinite set whose elements cannot be listed in one after the other?
Well, there are many:
Theorem
The set of infinite binary sequences is not countable.
So perhaps all sets are countable
24
![Page 25: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/25.jpg)
Assume that there exists a list of all binary sequences. Such a list may look like this:
Uncountable Sets
25
10001
1
1
1
0
0111
0000
0011
1101 ………
………………………
………
…………………….................
![Page 26: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/26.jpg)
But can you be sure that all sequences are in this list?
In fact, There exist many sequences that are not on the list:
Uncountable Sets
26
10001
1
1
1
0
0111
0000
0011
1101 ………
………………………
………
…………………….................
![Page 27: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/27.jpg)
Consider for example S=0,0,1,1,0,… . The sequence S is formed so that 1st elt. Of 1st seq.
2nd elt. Of 2nd seq.
3rd elt. Of 3rd seq.
And so on …
Uncountable Sets
27
10001
1
1
1
0
0111
0000
0011
1101 ………
………………………
………
……………………...........
1S
2S
3S
![Page 28: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/28.jpg)
In general: The i-th element of S, differs from the i-th element of the i-th sequence in the list. Note: the i-th element of the i-th sequence in the list is always the element on the diagonal.
Can the sequence S appear on the list?
Uncountable Sets
28
iS
![Page 29: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/29.jpg)
?...01100
........................
......10001
......10111
......10000
......10011
......01101
5
4
3
2
1
S
S
S
S
S
S
In this case,
Uncountable Sets
29
Assume there exists an index j such that jSS
jSjS j
jSjS jBut by definition:
Contradiction!!
![Page 30: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/30.jpg)
For any sequence on the list, , the sequence differs from by its element on the diagonal, that is: , so cannot be on the list.
For obvious reasons, this technique is called Diagonalization.
Uncountable Sets
30
jSS jS
jSjS j S
![Page 31: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/31.jpg)
We just used diagonalization to prove that the set of infinite binary sequences is uncountable.
Can a a similar proof for the set of real numbers?
Uncountable Sets
31
![Page 32: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/32.jpg)
Corollary
Some Languages are not Turing-recognizable.
Proof
For any (finite) alphabet, , the set of (finite) strings , is countable. A list of all elements in is obtained by first listing strings of length 1, then 2, …, then n…
Turing Unrecognizable Languages
32
*
*
![Page 33: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/33.jpg)
The set of all TM-s is also countable because every TM, , can be described by its encoding , which is a string over . So the set of TM-s corresponds to a subset of .
Note: Here we use the (unproven but correct) fact that the cardinality of a set is always not greater then the cardinality of any of its supersets.
Proof (cont.)
33
MM
*
![Page 34: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/34.jpg)
Since each TM recognizes exactly a single language, a list of all TM-s can be used as a list of all recognizable languages.
If we show that the set of languages over is uncountable, we can deduce that at least a single language is not on the list, that is: it is not recognized by any TM.
Proof (cont.)
34
![Page 35: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/35.jpg)
We have already seen that the set of infinite binary sequences is uncountable. If we form a correspondence between the set of languages and the set of infinite binary sequences we will show that the set of languages is uncountable.
Proof (cont.)
35
![Page 36: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/36.jpg)
Consider a fixed list l of all words . The correspondence is formed as follows: For every infinite binary sequence S, corresponds the language:
QED
Proof (cont.)
36
*
1 where| isilSL
![Page 37: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/37.jpg)
So far we proved the existence of a language which is not Turing recognizable. Now we continue our quest to prove:
Theorem
The language
is undecidable.
The Language ___ Is Undecidable
37
TMA
wMwMATM acceptingTM a is ,
![Page 38: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/38.jpg)
Before we start the proof let us consider two ancient questions:
Question1:
Can god create a boulder so heavy that god cannot lift?
The Language ___ Is Undecidable
38
TMA
![Page 39: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/39.jpg)
Question2:
In the small town of L.J. there is a single barber:
Over the barber’s chair there is a note saying:“I will shave you on one condition: Thaw
shall never shave thyself!!!”
Who Shaves the Barber?
The Language ___ Is Undecidable
39
TMA
![Page 40: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/40.jpg)
Assume, by way of contradiction, that is decidable and let H be a TM deciding .
That is
Define now another TM new D that uses H as a subroutine as follows:
Proof
40
TMA
wMreject
wMacceptwMH
rejects if
accepts if ,
TMA
![Page 41: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/41.jpg)
Define now another TM new D that uses H as a subroutine as follows:
D=“On input where N is a TM:
1. Run H on input .
2. Output the opposite of H’s output namely: If H accepts reject, otherwise accept.“
Proof
41
M
MM ,
![Page 42: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/42.jpg)
Note: What we do here is taking advantage of the two facts:
Fact1: TM M should be able to compute with any string as input.
Fact2: The encoding of M, , is a string.
Proof
42
M
![Page 43: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/43.jpg)
Running a machine on its encoding is analogous to using a compiler for the computer language Pascal to, that is written in Pascal, to compile itself.
As we recall from the two questions self-reference is sometimes means trouble (god forbid…)
Proof
43
![Page 44: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/44.jpg)
What we got now is:
Consider now the result of running D with input . What we get is:
Proof
44
M accepts if
M ejects if
Mreject
MacceptMD
r
D
D accepts if
D ejects if
Dreject
DacceptDD
r
![Page 45: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/45.jpg)
So if D accepts, it rejects wand if it rejects it accepts. Double Trouble.
And it all caused by our assumption that TM H exists!!!
Proof
45
D accepts if
D ejects if
Dreject
DacceptDD
r
![Page 46: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/46.jpg)
1. Define .
2. Assume that id decidable and let H be a TM deciding it.
3. Use H to build TM D that gets a TM encoding and behaves exactly opposite to H’s behavior, namely:
Proof Review
46
wMwMATM acceptingTM a is ,
TMA
M
M accepts if
M ejects if
Mreject
MacceptMD
r
![Page 47: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/47.jpg)
4. Run TM D on its encoding and conclude:
Contradiction.
Proof Review
47
D accepts if
D ejects if
Dreject
DacceptDD
r
D
![Page 48: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/48.jpg)
The following table describes the behavior of each machine on its encoding:
So Where is the Diagonalization?
48
acceptacceptM
M
acceptacceptacceptacceptM
acceptacceptM
MMMM
4
3
2
1
4321
![Page 49: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/49.jpg)
This table describes the behavior of TM H. Note: TM H rejects where loops.
So Where is the Diagonalization?
49
rejectrejectacceptacceptM
rejectrejectrejectrejectM
acceptacceptacceptacceptM
rejectacceptrejectacceptM
MMMM
4
3
2
1
4321
iM
![Page 50: 1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d775503460f94a59ecd/html5/thumbnails/50.jpg)
Now TM D is added to the table…
Proof Review
50
???
4
3
2
1
4321
acceptacceptrejectrejectD
acceptrejectrejectacceptacceptM
rejectrejectrejectrejectrejectM
acceptacceptacceptacceptacceptM
acceptrejectacceptrejectacceptM
DMMMM