ist 4 information and logic
TRANSCRIPT
IST 4Information and Logic
mon tue wed thr fri31 M1 1= todayT
7 M1
14 1 2M2x= hw#x out
= todayT
oh oh
oh oh
21 2
28 M2
x= hw#x due
idt
oh oh
h h28 M2
5 312 3 4Mx= MQx out
midtermsoh
oh oh
oh = office hours oh
12 3 419
26 4 5
Mx= MQx out
Mx= MQx dueoh oh
oh oh
26 4 52 5
Toh
oh
oh
oh
Tuesday 6/3 2:30pm –
1 D d l h G f f B G Pl
Tuesday, 6/3, 2:30pm
1. Diana Ardelean - The Gift of Becoming a GO Player
2. Shalini Majumdar - The Gift of Companionship
3. Alex Geffner-Mihlsten - The Gift of Names and Lives
4. Sumana Mahata - The Gift of Laughter
5. Parth Shah - My Mother has the Greatest of all Gifts!
6 Tim Menninger – Colors6. Tim Menninger Colors
7. Nasser Al Rayes - Boo (The Gift of Being Tall)
8 Sophia Yuenchih Chen The Gift of Uniqueness8. Sophia Yuenchih Chen - The Gift of Uniqueness
Thursday 6/5 2:30pm –
1 W ll N G L
Thursday, 6/5, 2:30pm
1. Willis Nguy - A Greater Loss
2. Sarah Brandsen - The Woman Who Cannot Forget
3. Angela Gui - The Persistence of Language
4. Jiyun Ivy Xiao - How I Trained My Memoryy y y y
5. Ankit Kumar - The Vedic Oral Tradition
6 Leon Ding - Chess Memory6. Leon Ding Chess Memory
7. Nancy Wen - Scent of a Memory
8 Grace Lee A Memory about Art 8. Grace Lee - A Memory about Art
Last LectureGates and circuits AON: AND, OR, Not
LT: Linear ThresholdLT: Linear Threshold
SYM LT1THSYM LT1TH
5AON *
4
2
LT-l
LT-nl
*
*
* = it is optimal Exponential gap in size
Last LectureGates and circuits AON: AND, OR, Not
LT: Linear ThresholdLT: Linear Threshold
SYM LT1THSYM LT1TH
5AON *
4
2
LT-l
LT-nl
*
*
* = it is optimal Exponential gap in size
neural circuits and logicgsome historysome history...
Being Homeless and Interdisciplinary Research
Warren McCulloch W l PiWarren McCulloch1899 - 1969
Walter Pitts1923 - 1969
Neurophysiologist, MD Logician, Autodidactg ,
Computing with neural circuits: a connection between logic and neural networks, 1943
Warren McCulloch arrived in early 1942 to the University of Chicago, invited Pitts, who was homeless, to live with his family
h ll h d P ll b d In the evenings McCulloch and Pitts collaborated. Pitts was familiar with the work of Leibniz on computing.They considered the question of whether the nervous system is a kind of universal computing device as described by Leibnizuniversal computing device as described by Leibniz
This led to their 1943 seminal neural networks paper:A Logical Calculus of Ideas Immanent in Nervous Activity
Warren McCulloch W l Pi
ImpactWarren McCulloch
1899 - 1969 Walter Pitts1923 - 1969
Neurophysiologist, MD Logician, Autodidactg ,
This led to their 1943 seminal neural networks paper:p pA Logical Calculus of Ideas Immanent in Nervous Activity
Neural networks d Logic Ti MNeural networks and Logic Time Memory
Threshold Logic and Learning
State Machines
Warren McCulloch W l Pi
ImpactWarren McCulloch
1899 - 1969 Walter Pitts1923 - 1969
Neurophysiologist, MD Logician, Autodidactg ,
This led to their 1943 seminal neural networks paper:
Neural networks d TiLogic M
p pA Logical Calculus of Ideas Immanent in Nervous Activity
Neural networks and TimeLogic Memory
Threshold Logic State Machines
TodayToday
State MachinesState Machines
Summaryy
St t M hiState Machinesdefinition
State Diagram
/1
State Diagram
1 /
a b/0
/0
a b0
1 /10/01
states – labeled vertices
i i di d dinputs – labels on edges that correspond to the symbols that transitions – directed edges correspond to the symbols that trigger the transitions
outputs – labels on edges that outputs – labels on edges that correspond to the symbols that are generated by the transitions
St t M hiState Machinesstream of bits in stream of bits out
State Diagram
/11 /
a b/0
/0
a b0
1 /10/01
state machine
symbols in symbols out
State Diagram
/11starting state /
a b
starting state
/0
/0
a b0
1 /10/01
state machine
0 0a
State Diagram
/11starting state /
a b
starting state
/0
/0
a b0
1 /10/01
state machine
1 1ab
State Diagram
/11starting state /
a b
starting state
/0
/0
a b0
1 /10/01
state machine
1 0ba
St t M hiState MachinesThe nosy professor
State Diagram for ???
/11starting state /
a b
starting state
/0
/0
a b0
1 /10/01
00
11
01
01
10
11
01
01
01
10
000 1 1 1 0 1 1 1 1 0 0
a a b b b a b b b b a a
State Diagram for ???
/11starting state dd/
a b
starting state
even
odd
/0
/0
a b0
1 /10/01
00
11
01
01
10
11
01
01
01
10
000 1 1 1 0 1 1 1 1 0 0
a a b b b a b b b b a a
XOR of the incoming sequence…..output 1 if saw an odd number of 1’s
State Diagram for XOR
/11starting state
g
/
a b
starting state
even odd
/0
/0
a b0
1 /10/01
Q: How to implement it using a syntax box / logic circuit?
St t M hiState Machinessynthesis
An Architecture for a State Machine
/11starting state /
a b
starting state
even oddHow do we represent a and b?
/0
/0
a b0
1 /10/01
linputs outputslogiccircuit
inputs outputs
functions of
state
functions of inputs and current state
An Architecture for a State Machine
/11starting state /
a=0 b=1
starting state
even odd
/0
/0
a=0 b=10
1 /10/01
linputs outputslogiccircuit
inputs outputs
functions of
state
functions of inputs and current state
State Machine for XOR
/11starting state /
a=0 b=1
starting state
even odd
/0
/0
a=0 b=10
1 /10
i
Computing the next state?/01
What is the function?
currentstate
inputs 10
XOR
What s th funct on?
state0
1
0 1
1 0
XOR
State Machine for XOR
/11starting state /
a=0 b=1
starting state
even odd
/0
/0
a=0 b=10
1 /10
i
Computing the output?/01
What is the function?
currentstate
inputs 10
XOR
What s th funct on?
state0
1
0 1
1 0
XOR
State Machine for XOR
/11starting state /
a=0 b=1
starting state
even odd
/0
/0
a=0 b=10
1 /10
i
Computing the output/01
Computing the next state
currentstate
inputs 1010
XOR XORstate
0
1
0 1 0 1
1 0 1 0
State Machine for XOR
/11starting state /
a=0 b=1
starting state
even odd
/0
/0
a=0 b=10
1 /10/01
linputs outputlogiccircuit
inputs outputXOR
state
State Machine for XOR
/11starting state /
a=0 b=1
starting state
even odd
/0
/0
a=0 b=10
1 /10/01
linputs output0 0
logiccircuit
inputs outputXOR
0
State Machine for XOR
/11starting state /
a=0 b=1
starting state
even odd
/0
/0
a=0 b=10
1 /10/01
linputs output1 1
logiccircuit
inputs outputXOR
0
State Machine for XOR
/11starting state /
a=0 b=1
starting state
even odd
/0
/0
a=0 b=10
1 /10/01
linputs output1 0
logiccircuit
inputs outputXOR
1
St t M hiState Machinessynthesis of an adder
2 symbol adderc
d1 d2
c
s
digit 1 digit 2digit 1 digit 2
2 symbol addercarry carry
sum
0 1
0 1
0
0 1
1
sum
2 symbol adderc
d1 d2
c
0 0 11 1 0
0 1 01 0 1
y
s 0 1
0 1
0 0 01 0 1
0 1
0 0 11 1 1
carry
1 0 1 1 1 1
1 1 1 0 0 0 1 1
2 symbol adder1
0
0 2 symbol adder0
1
0 2 symbol adder0
1
1 2 symbol adder1
0
0
1101 + 1001 = 10110digit 1 digit 2
2 symbol addercarry carry
sum
1 1 1 0 0 0 1 1
2 symbol adder1
0
0 2 symbol adder0
1
0 2 symbol adder0
1
1 2 symbol adder1
0
0
1101 + 1001 = 10110digit 1 digit 2
2 symbol addercarry carry
sum
1 1 1 0 0 0 1 1
2 symbol adder1
0
0 2 symbol adder0
1
0 2 symbol adder0
1
1 2 symbol adder1
0
0
1101 + 1001 = 10110 1 1
digit 1 digit 2
2 symbol adder
0 sum
1 1 1 0 0 0 1 1
2 symbol adder1
0
0 2 symbol adder0
1
0 2 symbol adder0
1
1 2 symbol adder1
0
0
1101 + 1001 = 10110 0 0
digit 1 digit 2
1 1
2 symbol adder
1 sum0
1 1 1 0 0 0 1 1
2 symbol adder1
0
0 2 symbol adder0
1
0 2 symbol adder0
1
1 2 symbol adder1
0
0
1101 + 1001 = 10110 1 0
digit 1 digit 2
01 01
2 symbol adder
1 sum10
1 1 1 0 0 0 1 1
2 symbol adder1
0
0 2 symbol adder0
1
0 2 symbol adder0
1
1 2 symbol adder1
0
0
1101 + 1001 = 10110 1 1
digit 1 digit 2
101 001
2 symbol adder
0 sum110
1 1 1 0 0 0 1 1
2 symbol adder1
0
0 2 symbol adder0
1
0 2 symbol adder0
1
1 2 symbol adder1
0
0
1101 + 1001 = 10110 0 0
digit 1 digit 2
1101 1001
2 symbol adder
1 sum0110
1 1 1 0 0 0 1 1
2 symbol adder1
0
0 2 symbol adder0
1
0 2 symbol adder0
1
1 2 symbol adder1
0
0
1101 + 1001 = 10110 0 0
digit 1 digit 2
1101 1001
2 symbol adder
1 sum10110
1 1 1 0 0 0 1 1
2 symbol adder1
0
0 2 symbol adder0
1
0 2 symbol adder0
1
1 2 symbol adder1
0
0
1101 + 1001 = 10110digit 1 digit 2
1101 1001
2 symbol adderWhat is the newingredient
memoryingredient
in the box???
sum10110
1 1 1 0 0 0 1 1
2 symbol adder1
0
0 2 symbol adder0
1
0 2 symbol adder0
1
1 2 symbol adder1
0
0
1101 + 1001 = 10110digit 1 digit 2
1101 1001
one bit = carry
2 symbol adderHow many bits do we memory
y
need to remember?
sum10110
1 1 1 0 0 0 1 1
2 symbol adder1
0
0 2 symbol adder0
1
0 2 symbol adder0
1
1 2 symbol adder1
0
0
1101 + 1001 = 10110digit 1 digit 2
1101 1001
one bit = carry
2 symbol adderThe STATE of the
box is represented by memory
y
the bits of the memory
Two statessum10110
Two statescarry = 0carry = 1
State Diagram for Addition
The STATE of the The STATE of the MACHINE is represented
b th bit f th
Two statescarry = 0
1by the bits of the Memory
carry = 12 symbol adder
Possible inputs01
??Possible inputs00
010 1
01
10
11
State Diagram for Addition
Two statescarry = 0
1carry = 12 symbol adder
Possible inputs01
11Possible inputs00
010 1
01
10
11
State Diagram for Addition
Two statescarry = 0
1carry = 12 symbol adder
Possible inputs10
Possible inputs00
01
11
0 101
10
??11
State Diagram for Addition
Two statescarry = 0
1carry = 12 symbol adder
Possible inputs10
Possible inputs00
01
11
0 101
10
0011
State Diagram for Addition
Two statescarry = 0
1carry = 12 symbol adder
Possible inputs11
Possible inputs00
010 1?? ??
01
10
0011
State Diagram for Addition
Two statescarry = 0
1carry = 1
Possible inputs2 symbol adder
11Possible inputs00
010 100
01 01
1101
10
0110
011000
11
State Diagram for Addition
Two statescarry = 0
1
2 symbol adder
carry = 1
Possible inputsWhat is the output?
/011Possible inputs00
01
/1
/0
/0
/10 100
01 01
1101
10
/1/1
/1 /0/001
10
011000
11
State Diagram for Addition
/011 /
0 1
/1
/0
/1 /0/0
/10 100
01 0110
11
00/1 /1 /010 1000
Q: How to implement it using a logic circuit?
State Machine for Addition
/011 /
0 1
/1
/0
/1 /0/0
/10 100
01 0110
11
00/1 /1 /010 1000
linputs outputlogiccircuit
inputs output
functions of
state
functions of inputs and current state
State Machine for Addition
/011 /
0 1
/1
/0
/1 /0/0
/10 100
01 0110
11
00/1 /1 /010 1000
iComputing the next state
currentstate
inputs 111001002 symbol adder
state0
1
0 0 0 1
0 1 1 1MAJority
State Machine for Addition
/011 /
0 1
/1
/0
/1 /0/0
/10 100
01 0110
11
00/1 /1 /010 1000
linputs outputlogiccircuit
inputs output
MAJority
state
MAJority
State Machine for Addition
/011 /
0 1
/1
/0
/1 /0/0
/10 100
01 0110
11
00/1 /1 /010 1000
iComputing the output
currentstate
inputs 111001002 symbol adder
state0
1
0 1 1 0
1 0 0 1XOR
State Machine for Addition
/011 /
0 1
/1
/0
/1 /0/0
/10 100
01 0110
11
00/1 /1 /010 1000
linputs outputXORlogiccircuit
inputs output
functions of
XOR
state
functions of inputs and current state
MAJority
State Machine for Addition
2 symbol adder
linputs outputXORlogiccircuit
inputs output
functions of
XOR
state
functions of inputs and current state
MAJority
State Machine for Addition
ymbo
l add
er2
sy
linputs outputXORlogiccircuit
inputs output
functions of
XOR
state
functions of inputs and current state
MAJority
Pl in ith St t M hinPlaying with State Machines
State Diagram for ???
/0
g
1starting state /
a b
starting state
/0
a b
1/1
/00
1/00/0
/0
0
1
/00
c d
/11
01
10
00
00
11
00
10
10
01 ist
/01starting state
a c d b d c a b a c/
a b
starting state
/0
a b
1/1
/00
1/00/0
/0
0
1
/00
c d
/11
Key to this ProgressAbstractions in Information SystemsAbstractions in Information Systems
We know how to synthesize: going from left to rightWe know how to synthesize: going from left to right
Claude ShannonG B l
Physics =Syntax =
u1916-2001George Boole
1815-1864
yrelay circuits
yBoolean algebra
reasoning
Key to this ProgressAbstractions in Information SystemsAbstractions in Information Systems
We know how to synthesize: going from left to rightWe know how to synthesize: going from left to rightHowever, it is hard for us t l i to analyze: going from right to left
Physics =Syntax = ystate
machines
ystate
diagramsreasoning
Even or odd number of 1s Even or odd number of 0s
/01starting state /
a b
starting state
/0
a b
1/1
/00
1/00/0
/0
0
1
/00
c d
/11
Even or odd number of 1s Even or odd number of 0s
/01starting state /
a b
starting state
/0
a b
1/1
/00
1/00/0
/0
0
1
/00
even number
c d
even numberof 1sand dd b
/11
odd numberof 0s
Key to this ProgressAbstractions in Information SystemsAbstractions in Information Systems
We know how to synthesize: going from left to rightWe know how to synthesize: going from left to right
A clear problem definition coupled Systematic process...p
Physics =Syntax = ystate
machines
ystate
diagramsreasoning
Represent a, b, c and dusing binary strings - 2 bits
State DiagramTo State Machine
Write the lookup table for the next state as a function ofth curr nt st t nd input
To State Machine
a b the current state and input
Write the lookup table for the output as a function of
a b
the output as a function ofthe current state and input
Implement the two functionsc d Implement the two functions
logicinputs outputs
c d
logiccircuit
inputs outputs
state
St t M hiState Machineshistory...
History on FSM (Finite State Machines)
Warren McCulloch1899 - 1969
Walter Pitts1923 - 1969 1943: A Logical Calculus of Ideas Immanent in 1899 1969 1923 1969 g f mm
Nervous Activity
Stephen C. Kleene 1909-1994
David A. Huffman 1925-1999
1951: Representation of Events in Nerve Nets and Finite Automata
1953: The Synthesis of Sequential Switching Circuits
Edward F. Moore, 1956: Gedanken-Experiments on Sequential Machines
George H. Mealy, 1955: A Method for Synthesizing Sequential Circuits
Shannon
It is all about PEOPLE…Shannon
1916-2001McCarthy 1927- 2011
John McCarthyB.S. in Mathematics, Caltech, 1948Ph.D. in Mathematics, Princeton, 1951
1956Claude Shannon1916-2001
John McCarthy 1927- 2011
7/29/2004, Palo Alto, CA
John von Neumann 1903-1957
Warren McCulloch1899 - 1969
1956Claude ShannonHixon Symposium at Caltech, 1948Cerebral Mechanisms in Behavior
1916-2001John McCarthy
1927- 2011
Cerebral Mechanisms in Behavior
von Neumann’s lecture: The central and logical Theory of Automata-comparisons between computing machines and living organisms
In the discussion following the lecture.
-comparisons between computing machines and living organisms
McCulloch:I confess that there is nothing I envy Dr. von Neumann more than the fact that the machines with which he has to cope are than the fact that the machines with which he has to cope are those for which he has, from the beginning, a blueprint of what the machine is supposed to do and how it is supposed to do itdo it.
Unfortunately for us in the biological sciences-or, at least, in y gpsychiatry - we are presented with an alien, or enemy's, machine. We do not know exactly what the machine is supposed to do and certainly we have no blueprint of itsupposed to do and certainly we have no blueprint of it.
A Final Question... almost...
Can a brain be simulated by simulated by a computer?
Answers:
Of course it can! It is a stupid questionOf course it can! It is a stupid question...
Not sure... probably yes... I thought that computers t thican compute anything...
Not my brain...
... seriously, I think that it is a valid question...
Gottfried Leibniz1646-1716 Leibniz
Information and Logic g
Characteristica Universalis:Characteristica UniversalisLeibniz ‘s goal was to develop an alphabet of human thought, a universal symbolic human thought, a universal symbolic language (characteristic) to describe nature
300 l300 year later,it is still a dreamit is still a dream...
The appearance of life is the first Information Megamorphosisg p
The appearance of the human brain is the second Information Megamorphosisthe second Information Megamorphosis
The Final Question!What will be
the third Information Megamorphosis?the third Information Megamorphosis?
Progress happens with
lthe introduction of new languagesDNA l l h i tib iDNA
spoken language
molecular switches associations
state machinesevolution
brain
written language
b b bilit
stochastic relays
l i
memory
number systems
mathematics proofs neural gates
probability
synthesis
analysis
mathematics p f
syllogismBoolean algebra
relays AON gatesalgebra
abstractions
axioms Boolean algebra
syntax boxesalgorizmsabacus axioms
The Babylonians knew everything!
E h if ! Mother’s day is the most
DNA l l h i tib i
Everyone has a gift! Mother s day is the most important day for the year!Smile!
DNA
spoken language
molecular switches associations
state machinesevolution
brain
written language
b b bilit
stochastic relays
l i
memory
number systems
mathematics proofs neural gates
probability
synthesis
analysis
mathematics p f
syllogismBoolean algebra
relays AON gatesalgebra
abstractions
axioms Boolean algebra
syntax boxesalgorizmsabacus axioms