plasticity and learning lecture 10. i.introduction ii.synaptic placticity rules − the basic hebb...

46
Plasticity and Learning LECTURE 10

Upload: shannon-newman

Post on 27-Dec-2015

267 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Plasticity and Learning

LECTURE 10

Page 2: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

I. Introduction II. Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules − Anti-hebbian rules − Timing-based Rules

Page 3: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Introduction

Hebb’s postulate

“When an axon of cell A is near enough to excite cell Bor repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased”.

Donald O. Hebb (1949)

The theory is often summarized as "cells that fire together, wire together".

Page 4: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

• Conditioning: - The first attempt to model conditioning in terms of synaptic change. - Behavior ---?--- neural mechanisms

• Development: - The formation and refinement of

neural circuits need synaptic elimination.

- Axonal or synaptic competition in neuromuscular junctions and visual system (Consumptive and interference competition)

Page 5: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

• Long term potentiation (LTP) - Long term depression ( LTD )

- Changes that persist for tens of minutes or longer are generally called LTP and LTD. It lasts for hours in vitro and days and weeks in vivo - The longest-lasting forms appear to require protein synthesis. - First found in Hippocampus - The physiological basis of Hebbian learning - Properties and mechanisms of long-term synaptic plasticity in the mammalian brain may relate to learning and memory. - Inhibitory synapses can also display plasticity, but this has been less thoroughly investigated both experimentally and theoretically

Page 6: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

I. Introduction II. Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules − Anti-hebbian rules − Timing-based Rules

Page 7: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

The basic Hebb rule

)(atytaxdt

tdwi

iw 0 )()(

)(

post

iwpre i

: the firing rates of the pre- and postsynaptic neurons

1. Local mechanism2. Interactive mechanism3. Time-dependent mechanism

yxi and

learning rate

Page 8: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

The basic Hebb rule is unstable

022 2 2

2

yydt

d

dt

dww xw

ww

w

post

wpre

ydt

dw x

w

xw

jjjjj

jjj

xwyxxf

xfwydt

dy

have we,)( Assume

)(

1. The processes of synaptic plasticity are typically much slower than the neural activity dynamics.

2. If, in addition, the stimuli are presented slowly enough to allow the network to attain its steady-state activity during training,

Page 9: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Theoretically, an upper saturation constraint must be imposed to avoid unbounded growth. But experimentally,

Page 10: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Is it due to that the basic Hebb rule has no LTD? Let’s add LTD by introducing the covariance rule

LTP and LTD at the Schaffer collateral inputs to the CA1 region of a rat hippocampal slice

Page 11: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

I. Introduction II. Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules − Anti-hebbian rules − Timing-based Rules

Page 12: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

The covariance rule

)( yw ydt

d xw

ydt

dw )( xx

w

x

postsynaptic threshold, e.g.

presynaptic threshold, e.g.

y

wxxxx

xxw

))((

)( ydt

dw

the input covariance matrix,

Page 13: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

• By the basic Hebb rule, synapses are modified whenever correlated pre- and postsynaptic activity occurs. Such correlated activity can occur purely by chance, rather than reflecting a causal relationship that should be learned. To correct for this, the covariance rather than correlation-based rule is often used by network models

ydt

dw )( xx

w

• Although the covariance rule allows LTD and reflects a causal pre- and postsynaptic relationship it is still unstable due to positive feedback

)( yydt

dw x

w (homosynaptic depression)

(heterosynaptic depression)

Page 14: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

The covariance rules, like the Hebb rule, are unstable and non-competitive

0)2(

)(2

)(2 2

2

2

2

yyydt

d

yy

ydt

d

dt

d

w

ww

w

xw

xxww

ww

ydt

dw )( xx

w

Average above equation over the training period:

post

wpre

Competition can be introduced to allowing threshold to slide as follows

Page 15: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

I. Introduction II. Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules − Anti-hebbian rules − Timing-based Rules

Page 16: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

BCM Rule

)( yw yydt

d xw

Bienenstock, Cooper and Munro (1982) proposed an alternative for which there is experimental evidence where the postsynaptic threshold is dynamic

w

yy y

dt

d

2y

LTP

LTD

0

Postsynaptic activity

One example:

Usually set:

Hebb rule covariance rule

BCM rule

Page 17: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

- This is again unstable if is fixed.

- However, if the threshold is allowed to grow faster than v we get stability.

LTP

LTD

0

Postsynaptic activity

- Here competition between synapses appears since strengthening some synapses results in threshold increasing meaning that it is harder for others to be strengthened

- depends on postsynaptic activity. For instance, the threshold for LTP decreases when postsynaptic activity is low (y ↓ ↓)

Page 18: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Synaptic weight normalization

constant

constant

2

jj

jj

w

w

- It is a more direct way of enforcing competition

- Idea is that postsynaptic neuron can only support a certain amount of total synaptic weight so strengthening one leads to weakening others

- 2 types: subtractive normalisation and multiplicative normalisation

Page 19: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Subtractive normalisation

jj

xi

iw

jj

jj

jj

xw

xN

yyx

dt

dw

wxxN

yy

dt

d

:or

) ;( wnxnnxw

- It is easy to prove that the total increase in the weights is 0.

constant j

jw

Page 20: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Evidences for BCM rule

- The field potentials evoked in layer III by layer IV stimulation in slices of visual cortex prepared for light-deprived and control rats 4-6 weeks old

- The effects can be reserved by as little as two days of light exposure before slice preparation

Evidence for a sliding threshold: It is easier to obtain LTP in the cortex of dark-reared animals and it is harder to induced LTD in these cortices

Page 21: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Experimental evidences for constant total synaptic weights

- Low- and high-frequency BLA stimuli (LFS, HFS) are known to, respectively, produce homosynaptic NMDA dependent LTD and LTP in ITC cells.

- Whether LFS and HFS also produce inverse heterosynaptic modifications is unclear.

Page 22: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

(Royer and Pare 2003, Nature)

• intercalated (ITC) neurons of the amygdala: 中间神经元 • the basolateral amygdala (BLA): 基底外侧杏仁核 • an array of closely spaced (~150 μm) stimulating electrodes

• slices of the amygdala

• guinea-pigs (3–5 weeks old)

Page 23: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Plot of EPSP amplitude and rise time versus stimulation site(Royer and Pare 2003, Nature)

• Homosynaptic LTP was induced with HFS paired to postsynaptic depolarization. Postsynaptic depolarization was achieved by applying short (2ms) depolarizing current pulses (0.2 nA) timed so that BLA-evoked EPSPs would occur just before or during current-evoked spikes

Page 24: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

LTD induction produces heterosynaptic LTP

Left: Difference between pre- and post-LFS response profiles (EPSP amplitudes) for one cell (top) and average of all cellsRight:Time course of changes in response amplitude

(Royer and Pare 2003, Nature)

Page 25: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Result is similar with high frequency stimuli

(Royer and Pare 2003, Nature),

Page 26: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

- Their results showed that the activity-dependent enhancement or depression of particular inputs to intercalated neurons is accompanied by inverse modifications at heterosynaptic sites, which contributes to total synaptic weight stabilization

- The inverse homo- versus heterosynaptic plasticity seems to be a cell- wide event, which needs an intracellular signaling system that can render synapses ‘aware’ of each other or of the mean neuronal activity.

- How do unstimulated inputs detect the stimulation frequency at the stimulated pathway?

Page 27: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

I. Introduction II. Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules − Anti-hebbian rules − Timing-based Rules

Page 28: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Non-Hebbian forms of synaptic plasticity

• They modify synaptic strengths solely on the basis of pre- or postsynaptic firing, are likely to play important roles in homeostatic, developmental, and learning processes

• Homeostatic plasticity - It allows neurons to sense how active they are and to adjust their properties to maintain stable function - Loosely defined, a homeostatic form of plasticity is one that acts to stabilize the activity of a neuron or neuronal circuit in the face of perturbations, such as changes in cell size or in synapse number or strength, that alter excitability. - A large number of plasticity phenomena have now been identified (e.g., synaptic scaling and homeostasis of intrinsic excitability of neurons)

Page 29: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Synaptic scaling

− A form of synaptic plasticity that adjusts the strength of all of a neuron's excitatory synapses up or down to stabilize firing, avoiding quiescence and hyper-excitation at the level of individual neurons.

− Current evidence suggests that neurons detect changes in their own firing rates through a set of calcium-dependent sensors

− Review paper: Gina G. Turrigiano. The Self-Tuning Neuron: Synaptic Scaling of Excitatory Synapses. Cell 135: 422-435, October 31, 2008

Page 30: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

(Turrigiano 1999, TINS)

A model of multiplicative scaling through the removal of AMPA receptors

Page 31: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

• Activity can also modify the intrinsic excitability and response properties of neurons

• Models of such intrinsic plasticity show that neurons can be remarkably robust to external perturbations if they adjust their conductances to maintain specified functional characteristics

• Intrinsic and synaptic plasticity can interact in interesting ways. For example, shifts in intrinsic excitability can compensate for changes in the level of input to a neuron caused by synaptic plasticity.

Homeostasis of intrinsic excitability of neurons

Page 32: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Homeostasis of intrinsic excitability of neurons

Theoretical and experimental work suggests that intracellular Ca2+ concentration might regulate the balance of inward and outward currents generated by a neuron

(Turrigiano 1999, TINS)

Page 33: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Anti-Hebbian plasticity

• It causes synapses to decrease (rather than increase) in strength when there is simultaneous pre- and postsynaptic activity.

• It is believed to be the predominant form of plasticity at synapses in mormyrid electric fish and those from parallel fibers to Purkinje cells in the cerebellum

• Anti-Hebbian modification tends to make weights decrease without bound

)(atytaxdt

tdwi

iw 0 )()(

)(

Page 34: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

I. Introduction II. Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules − Anti-hebbian rules − Timing-based Rules

Page 35: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Timing-Based Rules

LTP is induced by repetitive stimulation with positively correlated spike times of post and pre-synaptic neuron

LDP is induced by repetitive stimulation with negatively correlated spike times of post and pre-synaptic neuron

Page 36: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Spike Timing Dependent Plasticity (STDP)

An intracellular recording of a pair of cortical pyramidal cells in a slice experiment

(Markram et al., 1997)

Page 37: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

- LTP and LTD of retinotectal synapses recorded in vivo in Xenopus tadpoles

(Zhang et al., 1998)

Page 38: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

• Simulating the spike-timing dependence of synaptic plasticity requires a spiking model (e.g. Integrate-and-Fire Models). However, an approximate model can be constructed on thebasis of firing rates

)( )(

)]()()()()()([0

HH

txtyHtxtyHddt

dwii

iw

where

• Note above equation is based on a Hebbian rule

• The STDP rule describes an asymmetric learning rule

LTP LTD

Page 39: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

• About H(τ): a function like the solid line in previous figure.

.0 if ,

0 if ,)(

/

/

teA

teAtH

t

t

Page 40: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Sequence learning based on STDP

• The Timing-Based plasticity rule is applied throughout a training period during which the stimulus being presented moves to the right and excites the different neurons in the network sequentially

• After the training period, the neuron with sa = 0 receives strengthened input from the sa =−2 neuron and weakened input from the neuron with sa = 2

Page 41: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

• If the same time-dependent stimulus is presented again after training, the neuron with sa = 0 will respond earlier than it did prior to training

• The training experience causes neurons to learn a time sequence

Page 42: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Another example on time sequence learning in place fields

(Mehta et al. 1997; 2000)

• Place field is negatively skewed after experience

Page 43: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

A variety in plasticity

• Different cortical regions, such as hippocampus and visual cortex have somewhat different forms of synaptic plasticity.

(Abbott and Nelson 2000, Nature)

Page 44: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

A few properties of LTP and LTD

Long-term plastic changes can be induced in about 1 s or less (i.e. within a rather short period, similar to short-term plasticity)

The induced change in synaptic weight typically lasts for hours (if no further changes are induced)

The longest-lasting forms appear to require protein synthesis

Page 45: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Three types of training procedures

• Unsupervised (or sometimes self-supervised) learning. - A network responds to a series of inputs during training solely on the basis of its intrinsic connections and dynamics

• Supervised learning - A desired set of input-output relationships is imposed on the network by a ‘teacher’ during training. - Networks that perform particular tasks can be

constructed in this way

• Reinforcement learning - It is somewhat intermediate between these cases. - The network output is not constrained by a teacher, but evaluative feedback on network performance is provided in the form of reward or punishment

Page 46: Plasticity and Learning LECTURE 10. I.Introduction II.Synaptic placticity rules − The basic Hebb rule − The covariance rule − BCM Rule − Non-Hebbian rules

Homework

1. 与基本 Hebb 学习律比较 , BCM 学习律在哪些方面做了改进 ? 意义何在 ?

2. 举例说明稳态可塑性 (Homeostatic plasticity) 。

3. 如果要实现神经网络的时间序列学习,需要采用那种学习律?为什么?