jeff edmonds york university building a computer from sand layers of abstractionlayers of...

Jeff EdmondsYork University

Building a ComputerFrom Sand

• Layers of Abstraction• Building an AND Gate• AND/OR/NOT Circuits• Building a CPU• Circuit Depth• Parsing/Compiling• Neural Nets• Quantum Machines• Humans

Please feel free to ask questions!

Please give me feedbackso that I can better serve

you.

Thanks for the feedback that you have given me already.

Layers of Abstraction

Roumani-CSE 4

Roumani-CSE 5

SEMICONDUCTOR

Roumani-CSE 6

Roumani-CSE

Select between two alternatives A and B

7

Roumani-CSE 8

Roumani-CSE

CPU

DRAM

I/O

9

Roumani-CSE

Loader

Linker

Memory Manager

I/O Controller

Process Manager

0x3c0110010x342400280x201002260x000040200x000048200x3c0110010x002908210x8c2a00000x515000060x212900040x292a00280x1540fffa0x3c0110010x342400310x200200040x0000000c0x03e00008

boolean found = false;for (int i = 0; i < 10 && !found; i++){ found = (target == list[i]);}

la $a0, yes addi $s0, $0, 550 add $t0, $0, $0 add $t1, $0, $0lbl: lw $t2, list($t1) beq $t2, $s0, ok addi $t1, $t1, 4 slti $t2, $t1, 40 bne $t2, $0, lbl la $a0, nook: addi $v0, $0, 4 syscall jr $ra

boolean found = list.contains(target);

Select * from list where key = target;

boolean found = service.invoke(target);

10

Roumani-CSE 11

The Computing Universe

Vision | Robotics | AI | HCI | DB | Sim | Bio | DC | QC

Vision | Robotics | AI | HCI | DB | Sim | Bio | DC | QC

Roumani-CSE 12

Areas of Study

Lets do it again more slowly.

AND

yx

z

Building an AND Gate

Electricity can’t jump large gap from Cathode to Plate.It can jump from Cathode to Grid.

And if it starts flying, then it keeps going to the Plate.Large jump happens iff power

to Grid AND to Plate

AND

yx

z

Building an AND Gate

AND

yx

z

Building an AND Gate

Building an AND Gate

Building an AND Gate

Four electrons

in the outer ring.

Fourholes

in the outer ring.

Same as Carbon

Building an AND GateSame as Carbon

Building an AND GateSame as Carbon

Building an AND GateSame as Carbon

Each Carbon/Siliconbonds with four others.

Building an AND GateSame as Carbon

Building an AND GateSame as Carbon

Building an AND Gate

Electricity does not flow through it because all the electrons are happy.

Building an AND Gate

Electricity flows through it because this extra electron

moves.

Dope it with what is one to the right in the periotic table,

i.e. one extra electron.Called N

Building an AND Gate

Electricity flows through it because this extra hole

moves.

Dope it with what is one to the left in the periotic table,

i.e. one extra hole.Called P

Building an AND Gate

Electricity flows from N to Pextra electron to extra hole.

N P

Electricity does not flow from P to N extra hole to extra electron

Building an AND Gate

Once the electricity startsthe electrons keep tunneling through!!!

N P

Building an AND Gate

Tunneling happens iff power across PN

ANDacross PNP

N P

Building an AND Gate

Tunneling happens iff power across PN

ANDacross PNP

AND

yx

z

Building an AND Gate

x3x2x1

OR

ORANDAND

OR

NOT

• A circuit is a directed acyclic graph of and/or/not gates

0 1 0

• An input X assigns a bit to each incoming wire.

0 0

0

0 0

1

The bits percolate down to the output wires.

And/Or/Not Circuits

• Clearly circuits compute.• Any function f(X) of n bits can be computed

with a nonuniform circuit of size O(2n).

And/Or/Not Circuits

X f(X)

000000 0

000001 1

000010 0

000011 0

000100 1

2n

f(x)

n

= Cn

And/Or/Not Circuits

X f(X)

000000 0

000001 1

000010 0

000011 0

000100 1

2n

n

Outputs 1iff

X = 000001

¬x1¬x2 ¬x3 …xn

AND

¬x1 ¬x2 ¬x3 x4 … ¬xn

AND

Outputs 1iff

X = 000100

x1 x2 ¬x3 … ¬xn

AND

Outputs 1iff

X = 110..0

…

Repeat this for every value of X

for which f(X)=1.

And/Or/Not Circuits¬x1¬x2 ¬x3 …xn

AND

¬x1 ¬x2 ¬x3 x4 … ¬xn

AND

x1 x2 ¬x3 … ¬xn

AND…

Outputs 1 iff f(X)=1.

OR

Multiplexor

r bits to be addressed

y1

yr

x1 xlogrlog r bitsforms address x

yx The addressed bit

Multiplexor

And/Or/Not Circuits

¬x1¬x2 ¬x3 …xlogr

AND

¬x1 ¬x2 ¬x3 x4 … ¬xlogr

AND

x1 x2 ¬x3 … ¬xlogr

AND…

Multiplexor outputs addressed bit yx.

OR

Multiplexor

All r value that might get addressed.

AND

y1

AND

yi

AND

yr

Get the bit addressed.

Circuit Size ≈ 2logr = r = size of table.

And/Or/Not Circuits

• Time is kept by a clock.• Value xt may change through time.• But when clock is high, current value

is stored in memory.• When clock is low, it remembers

previous value.• Output of memory is this value stored.

Memory

xt

xt

Clockxtxt-1

And/Or/Not Circuits

• Time is kept by a clock.• Value xt may change through time.• But when clock is high, current value

is stored in memory.• When clock is low, it remembers

previous value.• Output of memory is this value stored.

Memory

xt

zt

Clockyt

zt = yt

xt if clock=1if clock=0

¬c

AND

OR

c

AND

xt+1

yt

And/Or/Not Circuits

Building a CPURAM

Add *B to A

38

10

accumulator

A

State of CPU & Computer at time t.register

B program counter

PC

Building a CPU

10

accumulator

A

State of CPU & Computer at time t.register

B program counter

PCM

ultiplexor

RAM

Add *B to A

38

Add *B to A

Command

Building a CPU

10

accumulator

A

State of CPU & Computer at time t.register

B program counter

PCM

ultiplexor

RAM

Add *B to A

38

Add *B to A

Command

38

*B

Building a CPU

10

accumulator

A

State of CPU & Computer at time t.register

B program counter

PC

Multiplexor

RAM

Add *B to A

38

Add *B to A

Command

38

*B

38

Building a CPU

10

accumulator

A

State of CPU & Computer at time t.register

B program counter

PC

Multiplexor

RAM

Add *B to A

38

Add *B to A

Command

38

*B

3810

Building a CPU

10

accumulator

A

State of CPU & Computer at time t.register

B program counter

PC

RAM

Add *B to A

38

Add *B to A

Command

3810

Add

48New A

Building a CPU

10

accumulator

A

State of CPU & Computer at time t.register

B program counter

PC

RAM

Add *B to A

38

IncNew PC

48New A New B

Save A to *B

Building a CPU

10

accumulator

A

State of CPU & Computer at time t.register

B program counter

PC

RAM

Add *B to A

38

New PC48

New A New B

Save A to *B

Clock48

register

B

Building a CPU

48

accumulator

A

State of CPU & Computer at time t.register

B program counter

PC

RAM

Add *B to A

38

Save A to *B

register

B

Save A to *B

Command

Reverse

Multiplexor

48

Building a CPU

48

accumulator

A

State of CPU & Computer at time t.register

B program counter

PC

RAM

Add *B to A

38

Save A to *B

48

Much slowerClock

Circuits Depth• The depth of a circuit:

• is the length of the longest path from an input to an output.

• It indicates evaluation time.• It relates to parallel

computation time.

x3x2x1

OR

ORANDAND

OR

NOT

0 1 0

0

0 0

0 0 1• The Size of a circuit:• is the number of gates.• It relates to sequential

computation time.

x1 ¬x2 x3 …¬xn

AND

Any function f(X) of n bits can be computed with a circuit of size 2n.

¬x1 x2 ¬x3 … xn

AND

x1 x2 ¬x3 … ¬xn

AND

Outputs 1 iff f(X)=1.

…

OR

1

If you allow arbitrary fan.But we assume 2 inputs.

Circuits Depth

x1 ¬x2 x3 …¬xn

AND

Any function f(X) of n bits can be computed with a circuit of size 2n.

¬x1 x2 ¬x3 … xn

AND

x1 x2 ¬x3 … ¬xn

AND

Outputs 1 iff f(X)=1.

…

OR

O(log(n))

O(n)

Circuits Depth

O00110100

H01001000

E01000011

L01001101

L01001101

qstart

Some smallcircuit

Some smallcircuit

Some smallcircuit

Some smallcircuit

Some smallcircuit

qstart x1 x2 x3 x4 x5 ….

• If a TM can compute it in time T(n), • then it uses at most T(n) cells of tape,• then the size of the circuit is O(T(n)×T(n)).• The circuit is constructed in a uniform way

from many copies of the same small circuit. O(T(n))

Circuits Depth

How to add 2 n-bit numbers.

**

**

**

**

**

**

**

**

**

**

**

+

How to add 2 n-bit numbers.

**

*

**

**

**

**

**

* **

**

**

**

**

+

How to add 2 n-bit numbers.

**

*

**

**

**

**

* **

* **

*

**

**

**

**

+

How to add 2 n-bit numbers.

**

*

**

**

**

* **

* **

*

* **

*

**

**

**

**

+

How to add 2 n-bit numbers.

**

*

**

**

* **

* **

*

* **

*

* **

*

**

**

**

**

+

How to add 2 n-bit numbers.

**

*

* **

*

* **

*

* **

*

* **

*

* **

*

* **

*

* **

*

* **

*

* **

*

***

*

+*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

+*

*

*

**

*

Takes O(n) time.Even with parallel help.

O(n) circuit depth.

How to add 2 n-bit numbers.

How to add 2 n-bit numbers.

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

+*

*

*

**

*

xi = ith input bit of X. yi = ith input bit of Y.

zi = ith output bit of Z.

ci = ith carry bit.

zi = lowBit( ci+xi+yi ) +

xi yi ci

ci+1 zi ci+1 = highBit( ci+xi+yi )

How to add 2 n-bit numbers.

c1 x1 y1

+z1

x0 y0 c0

+z0

c2 x2 y2

+z2

c3 x3 y3

+z3

c4 x4 y4

+z4

c5 x5 y5

+z5

c5 x5 y5

+z6 z7

O(n) circuit depth.

We need to compute the carries ci sooner.

Is O(n) depth intrinsic to adding or can it be done with less depth?

**

**

**

**

**

**

**

**

**

**

**

+ **

Previous alg I was taught Little extra thought.

Size (# of gates)

Depth (path leng)

O(n)

O(n)

O(n2)

O(logn)

O(n)

O(logn)

How to add 2 n-bit numbers.

Trade off between size and depth! Win-Win

**

**

**

**

**

**

**

**

**

**

**

+ **

At level l= 1,2,3,…,logn from the circuit inputs.Break the input bits into n/d blocks of size d = 2l.

n/d = 3 blocks d = 4 bits per blocks

How to add 2 n-bit numbers.

Xd0

Yd0

Xd1

Yd1

Xd2

Yd2

n/d = 3 blocks d = 4 bits per blocks

Note 0 ≤ Xli ≤ 2d-1

Value of 2d gives a carry out of these d bits.Xd

i + Ydi ≥ 2d, then there is definitely a carry

to the next block. We say the block “Generates” a carry. Wire gd

i = true.Xd

i + Ydi ≥ 2d-1, then there is a carry out

only if there is a carry in. We say the block “Propagates” a carry. Wire pd

i = true.

1

How to add 2 n-bit numbers.

**

**

**

**

**

**

**

**

**

**

+

AND

yixi

g1i

1n/1 = 12 blocks d = 1 bits per blocks

Wire g1i = block “Generates” a carry (ie. one out for sure)

xi+yi ≥ 21 = 2

g1i = And( xi, yi )

Wire pdi = block “Propagates” a carry (ie. one out if one in)

xi+yi ≥ 21-1 = 1 pd

i = Or( xi, yi )

1

*1

OR

yixi

p1i

10

How to add 2 n-bit numbers.

n/d = 3 blocks d = 4 bits per blocks

Wire gdi = block “Generates” a carry (ie. one out for sure)

iff first half generates a carry and second half propagates it or second half generates it = (gd/2

2i and pd/22i+1) or (gd/2

2i+1)

Wire pdi = block “Propogates” a carry (ie. one out if one in)

iff first half propagates a carry and second half propagates a carry = (pd/2

2i and pd/22i+1)

1

How to add 2 n-bit numbers. 1

Xd/22i+1

Yd/22i+1

Xd/22i

Yd/22i

1 1

n/d = 3 blocks d = 4 bits per blocks How to add 2 n-bit numbers.

Xd/22i+1

Yd/22i+1

Xd/22i

Yd/22i

gdi pd

i

gd/22i+1 pd/2

2i+1 gd/22i pd/2

2i

gdi+1 p

di+1

gd/22i+3 pd/2

2i+3 gd/22i+2 pd/2

2i+2

Xd/22i+1

Yd/22i+1

Xd/22i

Yd/22i

g2di/2 p

2di/2

Little extra thought.

Size

Depth

n+n/2+n/4+…1 = O(n)

O(logn)

n/d = 3 blocks d = 4 bits per blocks

Suppose you know the carries between blocks of size d.Then there is a carry between these block of size d/2, iff carry at beginning and propagate or generate.

How to add 2 n-bit numbers. *

Xd/22i+1

Yd/22i+1

Xd/22i

Yd/22i

* 1 1

Compute ALL carries between bits.

How to add 2 n-bit numbers.

*

Little extra thought.

Size

Depth

n+n/2+n/4+…1 = O(n)

O(logn)

* * * * * * * * * * * * * * * *

Compute ALL carries between bits.

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

+*

*

*

**

*

xi = ith input bit of X. yi = ith input bit of Y.

zi = ith output bit of Z.

ci = ith carry bit.

zi = lowBit( ci+xi+yi )

+

xi yi ci

zi

How to add 2 n-bit numbers.

Little extra thought.

Size

Depth

O(n)O(1)

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

*

**

*

+*

*

*

**

*

How to add 2 n-bit numbers.

Previous alg I was taught Little extra thought.

Size (# of gates)

Depth (path leng)

O(n)

O(n)

O(n2)

O(logn)

O(n)

O(logn)

How to multiply 2 n-bit numbers.

* * * * * * * * * * * * * * * *

* * * * * * * * * * * * * * * *

* * * * * * * * * * * * * * * *

* * * * * * * * * * * * * * * *

* * * * * * * * * * * * * * * *

* * * * * * * * * * * * * * * *

x1 x2 x3

xn

Add n 2n-bit

numbers

O(n logn) circuit depth.

How to multiply 2 n-bit numbers.

x3

+

x1 x2

+

x4

+x5

+x6

+x7

+

O(logn) circuit depth.

Can we multiply in O(log2n) depth?

Can we multiply in O(log2n) depth?

How to multiply 2 n-bit numbers.

Can we multiply in O(logn) depth?

x11 x12 x9 x10 x7 x8 x5 x6 x3 x4 x1 x2

++++++

+++

+

6.

How to multiply 2 n-bit numbers.

Output:

Input: s=6*8+((2+42)*(5+12)+987*7*123+15*54)

Parsing/Compiling

Context Free Grammar:

Input: Java Code

Parsing/Compiling

Output: MARIE Machine Codesimulating the Java code.

x3x2x1

×

+ × +

/

-

• An arithmetic circuit has +, -, ×, & / gates.

3 5 7

• An input X assigns a real number to each incoming wire.

8 21

-14

168 -12

12

Arithmetic Circuits

The real numbers percolate down to the output wires.

Neural Nets

• Inputs x1, x2 , x3 , …, xn and output y are binary.• Weights w1 , w2 , w3 , …, wn are real numbers

(possibly negative).• y = 1 iff Σi wi×xi ≥ T• The neural net learns by adjusting weights wi.

y

Threshold

T

x1 x2 x3 … xn w1 w2 w3 … wn

Neural Nets

Quantum MachinesWhat about Quantum Machines?• Based on quantum mechanics,

at each point in time a quantum TM is in the super-position of any number of configurations of normal TMs. • In time T, it can only flip T quantum coins so can only be

in the super-position of 2T configurations.• Hence, can be simulated by a normal TM in time 2T×T

time.• Hence, Quantum Machines can’t compute more,

just sometimes faster.• Factoring can be done in poly-time, ie 6=2×3.• It is believed that NP-complete problems still take

exponential time.

Human

What about the human brain? Can it compute more than a TM?• Science: • The brain is just an elaborate machine (neural net).• Hence can’t do any more than a TM.

• New Age:• Quantum mechanics is magical.• The brain is based on quantum mechanics.• Hence, the brain can do much more than a machine.• But we already showed QM can’t compute more.

• Religion:• The human has a soul.• Hence, the brain can do much more than a machine.

End