1

The Halting Problem and Decidability

How powerful is a TM?

Any program in a high level language can be simulated by a

TM.

Any algorithmic procedure carried out by a human or a

computer can be carried out by a TM.

“Algorithm” is an informal notion – but we have identified it with

a TM.

Questions

Are there problems that cannot be solved by an algorithm?

(existence)

What are some example problems?

(witness)

The plan:

1. Universal Turing Machines

2. The Halting Problem

3. Undecidability/unsolvability

2

Universal Turing Machines

Interpreting and running programs is algorithmic. It is is a

difficult task, but nevertheless can be carried out by an

algorithm (e.g. a compiler).

SO

We can design a (universal) TM which “compiles” or “runs”

other TMs. This is the universal TM – it takes as input a TM

and its input tape, and runs the TM on that input.

e(M)*x UTM outputs fM(x)

iff M computes fM(x)

i.e. it halts

input loops

iff M loops on x

If M is a TM, then encode it by e(M). x is the input for M, * is a separator symbol.

There are lots of ways to encode a TM, i.e. to construct e(M). Here is one example.

NB. A TM may be very large, and have many input symbols, but they are all finite, hence a finite encoding.

3

Encoding a Turing Machine

Consider a TM M with states Q and alphabet (the input alphabet).

Let A = {a1,a2, …} both infinite sets

St = {q1,q2,…}

Such that Q C St and C A.

Encode every symbol ai in A by e(ai)= ai, where i is the binary

representation of i. For example, a5 is encoded by a101. Similarly, qi

encodes qi.

Assume a1 is the marker <, a2 is a blank, and q1 is the start state.

Represent every halting state by a blank. So the transition (q,a),( ,a)

is a transition into a halting state (trust me).

The TM is represented by a sequence of transitions of form

((qi,aj),(qk,t)) where t is either ar (a symbol), or -> or <-. Note, qk

might be blank.

For example, the transition ((q5,a4),(q6,->)) is represented by

((q101,a100),(q101,->)).

Assume the sequence of transitions is ordered lexicographically. (Doesn’t matter which ordering, pick one).

Encode the input string in the same way.

4

Encoding a Turing Machine

Now, to simulate M, assume a 3-tape version for the UTM U.

1. Tape 1 has e(M)*e(x)

2. Copy e(M) on to tape 2

3. Shift e(x) to leftmost position of tape 1.

Tape 1 will be used to simulate the I/O of M. Note that tape 1head

does not look at a symbol from x, but an encoding of it, e.g. a101

instead of a5. So, before any simulation step assume head is a first

symbol of encoding. E.g. a101.

4. Tape 3 is used to hold qi – the encoding of the current state qi in M.

To simulate a step in M, assume U is in the state (corresponding to )

qj in M (qj on tape 3), and tape 1 head points at ai (tape 1). Look up

transition for (qj,ai) on tape 2 and simulate that transition, leaving new

state on tape 3 and move tape 1 head accordingly.

This concludes simulation of M by U.

5

Undecidability/unsolvability

Undecidability/unsolvability – shades of the same thing.

A problem is unsovable if there exists no Turing machine to compute

it

Question1 . Are there problems that cannot be solved by an

algorithm? (existence)

Yes. Proof:

1. Show all TMs can be enumerated

2. Show all partial functions cannot be enumerated. (Diagonalisation)

1. Encode every TM M by a unique number n(M) (Godel

numbering)

i) Encode M by e, as for UTMs. ie consider e(M). Convert every

transition to binary, then concat all together, with inputs. Result is one

long string which can be interpreted as a number.

ii) Enumerate all the TMs as follows

Tk is M iff n(M) = k

is T otherwise, where T is a special non-halting machine

6

Undecidability/unsolvability

2. There exists a (partial) function f:N->N that is not (partial) Turing

computable, i.e. there is a function without a corresponding machine.

Proof: Diagonalisation.

For every k, let k be the function computed by Tk.

Now construct a “diagonal” function f:

f(x) = (x) +1 if x is in dom((x))

= 0 otherwise

e.g. if (3) = 6, then f(3) = 7.

0 1 2 3 …

(0) f(0)<> (0)

(1) f(1)<> (1)

(2) f(2)<> (2)

.

.

.

Clearly, f differs from every at every single value. So f isn’t in the enumeration.

Thus there exists a function which has no corresponding TM!

Let’s consider a more concrete example.

7

The Halting Problem

Problem: Design a machine H such that

outputs 1

e(M)*e(x) H iff M halts given input x

outputs 0

Input iff M does not halt given input x

Theorem: The Halting problem is unsolvable

Proof: By contradiction. Assume H exists and then derive a contradiction.

Construct the following machine H’:

e(M) copier e(M)*e(M) modified loops iff

H M halts given input (e(M))

outputs 0 iff

M does not halt

H’ given input e(M))

Modified H – just like H, except instead of outputting 1, loops forever.

Now, let e(H’) be the input to H’.

If H’ loops => output is 0 (by defn of H’ when it outputs 0)

If H’ outputs 0 => loops (by defn of H’ when it loops)

CONTRADICTION! So H cannot exist.

8

Implications

There is no algorithm which can take an arbitrary program and arbitrary input, and tell you whether or not the program will halt.

You may be able to show this for a particular program, or even a sub-class of programs, but there is no general strategy for all programs.

E.g.

While b do ….

For i=1 to x do ….

The world of computing is littered with (interesting!) unsolvable problems!

9

Decision problems

Unsolvable problems: yes/no solutions

Examples

1. Does z = x+y, (for integers x,y,z)

2. Does TM M halt given x as input.

3. Does TM M halt for very input x in *.

4. Does TM M halt for given input .5. Are two TMs equivalent.

6. Does a diaphantine equation have a solution. (Hilbert’s 10 th problem)

A DE has form p(x1,…xn) = 0

e.g. x2+y2 = z2

x2-2 = 0

7. Is a predicate calculus statement valid. (1936, Church)

10

Reduction

Arguments by reduction:

Defn A decision problem reduces to ’, ->’, if an algorithm to solve ’ can be used directly to solve .

->’ iff

e(I) TM e’(I’)

I is instance of , I’ is instance of ’.

----------------------------------------------------------------------------------------------------

Theorem: If ->’ and ’ is solvable, then so is ’ .

Corollary: If ->’ and is unsolvable then so is ’.

e(I) TM e’(I’) ’ 1 iff I’ in Y(’) iff 1 in Y()

->’ TM 0 iff I’ in N(’) iff 1 in N()

11