Cognitive Computing….Computational Neuroscience

Jerome SwartzThe Swartz Foundation

May 10, 2006

Large Scale Brain Modeling• Science IS modeling

• Models have power– To explain

– To predict

– To simulate

– To augment

Why model the brain?

Brains are not computers …• But they are supported by the same physics

Energy conservation Entropy increase Least action Time direction

• Brains are supported by the same logic,

but implemented differently…– Low speed; parallel processing; no symbolic software layer;

fundamentally adaptive / interactive; organic vs. inorganic

Brain research must be multi-level

• Scientific collaboration is needed– Across spatial scales

– Across time scales

– Across measurement techniques

• Current field borders should not remain

boundaries… Curtail Scale Chauvinism!

…both scientifically and mathematically• To understand, both theoretically and practically,

how brains support behavior and experience

• To model brain / behavior dynamics as Active requires– Better behavioral measures and modeling

– Better brain dynamic imaging / analysis

– Better joint brain / behavior analysis

… the next research frontier• Brains are active and multi-scale / multi-level• The dominant multi-level model: Computers

… with their physical / logical computer hierarchy – the OSI stack

– physical / implementation levels

– logical / instruction levels

( = STDP)

A Multi-Level View of Learning

LEARNING at a LEVEL is CHANGE IN INTERACTIONS between its UNITS,implemented by INTERACTIONS at the LEVEL beneath, and by extensionresulting in CHANGE IN LEARNING at the LEVEL above.

IncreasingTimescale

Separation of timescales allows INTERACTIONS at one LEVEL to be LEARNING at the LEVEL above.

Interactions=fastLearning=slow

LEVEL UNIT INTERACTIONS LEARNING

society organism behaviour

ecology society predation, symbiosis

natural selection

sensory-motorlearning

organism cell spikes synaptic plasticity

cell

protein molecular forces gene expression,protein recycling

voltage, Ca bulk molecular changessynapse

amino acid

synapse protein direct,V,Ca molecular changes

( = STDP)

A Multi-Level View of Learning

LEARNING at one LEVEL is implemented byDYNAMICS between UNITS at the LEVEL below.

IncreasingTimescale

Separation of timescales allows DYNAMICS at one LEVEL to be LEARNING at the LEVEL above.

Dynamics=fastLearning=slow

LEVEL UNIT DYNAMICS LEARNING

society organism behaviour

ecology society predation, symbiosis

natural selection

sensory-motorlearning

organism cell spikes synaptic plasticity

cell

protein molecular forces gene expression,protein recycling

voltage, Ca bulk molecular changessynapse

amino acid

synapse protein direct,V,Ca molecular changes

T.Bell

What idea will fill in the question mark?

physiology (of STDP)

physics of self-organisation

probabilistic machine learning

?(STDP=spike timing-dependent plasticity)

-unsupervised probability density estimation across scales

- the smaller (molecular) models the larger (spikes)…. suggested by STDP physiology, where information flow from neurons to synapses is inter-level….

? = the Levels Hypothesis: Learning in the brain is:

T.Bell

network of 2 brains

network of neurons

network of macromolecules

network of protein complexes(e.g., synapses)

Networks within networks

1 cell1 brain

Multi-level modeling:

ICA/Infomax between Layers.(eg: V1 density-estimates Retina)

2

• within-level• feedforward• molecular sublevel is ‘implementation’• social superlevel is ‘reward’• predicts independent activity• only models outside input

retina

V1

synaptic weights

x

y

Infomax between Levels.(eg: synapses density-estimate spikes)

1

• between-level• includes all feedback• molecular net models/creates• social net is boundary condition• permits arbitrary activity dependencies• models input and intrinsic together

all neural spikes

all synaptic readout

synapses,dendrites

t

y

pdf of all spike timespdf of all synaptic ‘readouts’

If we canmake thispdf uniform

then we have a model constructed from all synaptic and dendritic causality

ICA transform minimises statisticaldependence between outputs. The bases produced are data-dependent,not fixed as in Fourier or Wavelettransforms.

T.Bell

The Infomax principle/ICA algorithms T.Bell

Many applications (6 international ICA workshops)…

• audio separation in real acoustic environments (as above)

• biomedical data-mining -- EEG,fMRI,

• image coding

Cognitive Computing…Computational Neuroscience