the power and the limits of computation

The Power and the Limits of Computation

Elaine Rich

The Earliest Digital Computers

1945 ENIAC

Stored 20 10-digit decimal numbers

The IBM 7090A dual 7090 system at NASA in about 1962.

Could store 32,768 36-bit words.

That’s about .00015 gigabytes.

Cost:

about $3,000,000.

or $19,794,000 2005 dollars

The Earth SimulatorThe Earth Simulator (ES) is a project of Japanese agencies to develop

a 40 TFLOPS system for climate modeling. The ES site is a new location in an industrial area of Yokohama, an

hour drive west of Tokyo. The facility became operational in late 2001.

Hardware

The ES is based on: 5,120 (640 8-way nodes) 500 MHz NEC CPUs

8 GFLOPS per CPU (41 TFLOPS total) 2 GB (4 512 MB FPLRAM modules) per CPU (10 TB total)

shared memory inside the node 640 × 640 crossbar switch between the nodes

16 GB/s inter-node bandwidth 20 kVA power consumption per node

The iPod Nano

2005

Can store 4 gigabytes (1000 songs).

Cost: about $250.

Compute Power Increases Over Time

From Hans Moravec, Robot Mere Machine to Transcendent Mind 1998.

Moore’s Law

What Can We Do With All that Power?

Overheard in ACES last week: “Genomics has turned biology into an information science.”

Is there anything we can’t do?

The Traveling Salesman Problem

Finding a Solution to the TSP

Given n cities:

•n choices for a starting point.•n-1 for the next city•n-2 for the next city

For a total of n! paths to be considered.

Finding a Solution to the TSP

Given n cities:

•n choices for a starting point.•n-1 for the next city•n-2 for the next city

For a total of n! paths to be considered.

We notice that it doesn’t matter where we start (since we need to make a loop).

And the cost is the same forward or backward. So we can cut the number of paths down to:

(n – 1)!/2

The Growth Rate of n!n! = n(n-1)(n-2)…(1)

2 2 11 4790016003 6 12 62270208004 24 13 871782912005 120 14 13076743680006 720 15 209227898880007 5040 16 3556874280960008 40320 17 64023737057280009 362880 18 12164510040883200010 3628800 19 243290200817664000011 39916800 36 3.6 1041

Putting that Rate into Perspective

Speed of light: 3 108 m/sec

Width of a proton: 10-15 m

If we perform one operation in the time light crosses a proton, we can perform: 3 1023 ops/sec

Seconds since the big bang: 3 1017

Operations since the big bang: 9 1040

Compared to 36! 3.6 1041

The Post Correspondence Problem

i X Y1 b bbb2 babbb ba3 ba a4 bbbaa babbb

The Post Correspondence Problem

i X Y1 b bbb2 babbb ba3 ba a4 bbbaa babbb

2

X b a b b b

Y b a

The Post Correspondence Problem

i X Y1 b bbb2 babbb ba3 ba a4 bbbaa babbb

2 1

X b a b b b b

Y b a b b b

The Post Correspondence Problem

i X Y1 b bbb2 babbb ba3 ba a4 bbbaa babbb

2 1 1

X b a b b b b b

Y b a b b b b b b

The Post Correspondence Problem

i X Y1 b bbb2 babbb ba3 ba a4 bbbaa babbb

2 1 1 3

X b a b b b b b b a

Y b a b b b b b b a

The Post Correspondence Problem

i X Y1 10 1012 011 113 101 011

i X Y1 1101 12 0110 113 1 110

The Post Correspondence ProblemA program to solve this problem:

Until a solution is found do:Generate the next candidate solution.Test it. If it is a solution, halt and report yes.

So, if there are say 4 rows in the table, we’ll try:1 2 3 4 1,1 1,2 1,3 1,4 1,5 2,1 …… 1,1,1 ….

The Post Correspondence ProblemA program to solve this problem:

Until a solution is found do:Generate the next candidate solution.Test it. If it is a solution, halt and report yes.

So, if there are say 4 rows in the table, we’ll try:1 2 3 4 1,1 1,2 1,3 1,4 1,5 2,1 …… 1,1,1 ….

But what if there is no solution?

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

A Tiling Problem

Programs Debug Programs

read nif 2*int(n/2) = n then print “even” else print “odd”

read nresult = 1for i = 2 to n do

result = result * iprint result

Given an arbitrary program, can it be guaranteed to halt?

A Problem That Cannot be Solved in Any Amount of Time

read nif 2*int(n/2) = n then print “even” else print “odd”

read nresult = 1for i = 2 to n do

result = result * iprint result

Given an arbitrary program, can it be guaranteed to halt?

result1

i= 2 2 i= 3 6i= 4 24i= 5 120

Other Kinds of Loops

result = 0count = 0until result > 100 do

read n count = count + 1 result = result+nprint count

Other Kinds of Loops

result = 0count = 0until result > 100 do

read n count = count + 1 result = result+nprint count

Suppose all the inputs are positive integers.

Other Kinds of Loops

result = 0count = 0until result > 100 do

read n count = count + 1 result = result+nprint count

Suppose some of the integers are negative.

The Halting Problem Is Not Solvable

Suppose that the following program existed:

Halts(program, string) returns:•True if program halts on string•False otherwise

The Halting Problem is Not Solvable

Consider the following program:

Trouble(string) = If Halts(string, string) then loop forever else halt.

Now we invoke Trouble(<Trouble>).

What should Halts(<Trouble>, <Trouble>) say? If it:•Returns True (Trouble will halt): Trouble loops•Returns False (Trouble will not halt): Trouble halts

So there is no answer that Halts can give that does not lead to a contradiction.

Other Unsolvable Problems

PCP:

•We can encode a <program>,<input> pair as an instance of PCP so that the PCP problem has a solution iff <program> halts on <input>.

•So if PCP were solvable then Halting would be.

•But Halting isn’t. So neither is PCP.

Other Unsolvable ProblemsTiling:

•We can encode a <program>,<input> pair as an instance of a tiling problem so that there is an infinite tiling iff <program> halts on <input>.

00010000111000000111110000000000000 0001000011101000011111000000000000000010000111011000111110000000000000 …

•So if the tiling problem were solvable then Halting would be.

•But Halting isn’t. So neither is the tiling problem.