Learning Rules 1

Computational Neuroscience 03

Lecture 8

Last week showed how to model neurons and networks of neurons using firing rate models.

And we then discussed how to add them together to form networks of neurons

).( rwFvdt

dvr

Where the weight vector is replaced by a matrix. Also often replace feedforward input with a vector

Feedforward and Recurrent networks

)()( rMhFvrMrWFvdt

dvr

Recurrent networks can also do this but have much more complex dynamics than feedforward nets. Also more difficult to analyse

Much analysis focuses on looking at the eigenvectors of the matrix M

Can show for instance that networks can exhibit selective amplification if there is one dominant eigenvector (cf PCA)

Or if an eigenvalue is exactly equal to 1 and others < 1can get integration of inputs and therefore persistent activity as activity does not stop when input stops

But how are we to generate such precise weight changes? Need some synaptic modification rules

Hebb’s postulateHebb's postulate of learning (or simply Hebb's rule) (1949), is the following:

"When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth processes or metabolic changes take place in one or both cells such that A's efficiency as one of the cells firing B, is increased".

This rule forms the basis of much of the research on role of synaptic plasticity in memory and learning

Has been generalised to include decreases of strength when neuron A repeatedly fails to be involved in activation of B and generally look at the correlation or covariance of activities of pre-and postsynaptic neurons

Learning in the hippocampus

Here see examples of long-term potentiation and depression (LTP and LTD).

High frequency stimulation induces LTP while long-lasting low frequency leads to LTD

‘Long Term’ refers to changes > 10 mins

Learning RulesWe will consider 3 types of learning and will focus on Hebbian type learning though others exist (modifcations on pre/postsynaptic activity only etc)

• Unsupervised learning: Network responds during training solely as a result of its connections and intrinsic dynamics. Net self-organises in a manner dependent on inputs and synaptic plasticity rule

• Supervised learning: Here the network also has a “teacher” in the form of a set of desired ‘target’ outputs for each input. Not especially biologically plausible, but good for existence proofs

• Reinforcement learning: Somewhere in between the 2. The net does not know the target output but gets feedback via reward punishment

Unsupervised Learning

Start with single postsynaptic neuron and a linear activation function. As synaptic changes will be much longer time scale than these dynamics firing rate equation reduces to:

uvdt

wdw

uwvdt

dvr .

Start with simplest Hebbian style plasticity for a single neuron:

uN

bbbuwuwv

1

.

As u, v denote firing rates vu can be interpreted as a probability that pre- and post fire together.

Each different set of u is known as an input pattern. As weight changes are slow, rather than summing all changes separately can average the input patterns and thus compute the average change.

To do this use < > to denote averages over the ensemble of input patterns. Thus get:

uvdt

wdw

Remembering that v = w.u this gives the correlation-based rule

uN

bbbb

bww wQ

dt

dwwQ

dt

wd

1''' :or .

Where Q is correlation matrix: Qbb’ = <ubub’>

So suppose u1= (1, 0) and u2= (0, 1) then

5.00

05.0Q

01.0

015.0

1.0

1.0

5.05.0

5.011.0

1wQw

w

Alternatively u1= (1, 1) and u2= (1, 0) then

5.05.0

5.01Q

Suppose w=10 and w=(0.1,0.1) and we have the 2nd matrix then:

As wn+1= wn + w, w1=(0.115, 0.11), w2=(0.13, 0.12) ….

Which leads to instability and uncontrolled growth of w

eg for 2 patterns u1 and u2, Qij = ½(u1i u1

j + u2i u2

j)

W=(0.1, 0.1)

W=(0.1, -0.3)

Outcome dependent on eiegenvectors of Q and initial conditions, but unstable

To avoid unbounded growth can impose a saturation constraint.

However, this means all weights go to max or min and thus we have no competition between different synapses

This means that the neuron cannot distinguish between presynaptic inputs

Also since u, v are firing rates and therefore positive, Hebb rule only describes LTP.

However, earlier figure showed that synapses can depress in strength if presynaptic activity is accompanied by a low level of postsynaptic activity

Can also get results where opposite is true

Analysis focuses on looking at eigenvectors of Q the correlation matrix of the input vectors

For instance, can show that after training Hebbian rule leads to v e1.u for arbitary vector u and that weight vector expressed as a sum of eigenvectors is dominated by e1 ie w e1

That is v is projection of u onto the principal eigenvector of Q

Eg for a Q with principal eigenvector (1, -1)/sqrt(2) we would expect w to end up as (wmax,0) or (0, wmax).

However because of saturation constraints can get (wmax, wmax)

Covariance ruleTo get LTD can introduce a postsynaptic or presynaptic threshold:

uvdt

wdvw )(

Below thresholds get depression, above potentiation. A convenient choice for the thresholds is the average pre/post synaptic input <u> or <v>. If we now replace v with w.u we get the covariance rule

.wCdt

wdw

Where C is the covariance matrix of the input data ie:

)( uw uvdt

wd

]))([( )-()-(2 TuuuuEuuuuuuuC

Note that as <v> keeps changing we need to keep updating the postsynaptic threshold while the presynaptic one is independent of the weights

Although both average to the same thing, they do have differences.

The postsynaptic threshold means that only modifies weights for non-zero presynaptic activities. If v is below threshold then this results in homosynaptic depression

Alternatively, presynaptic threshold reduces the strength of inactive synapses for v>0: heterosynaptic depression

Although the covariance rule allows LTD it is still unstable due to positive feedback

Also we do not have competition, but this can be introduced to allowing threshold to slide as follows

As covariance rule allows LTD without postsynaptic/presynaptic activity, Bienenstock, Cooper and Munro (82) proposeed an alternative for which there is experimental evidence where the postsynaptic threshold is dynamic

BCM rule

)( vw vuvdt

wd

This is again unstable if is fixed.

However, if we allow the threshold to grow faster than v we get stability. For instance use as low pass filtered version of v2

vv v

dt

d 2

Usually set to be less than w so that changes in faster than changes in v

Now get competition between synapses since strengthening some synapses results in threshold increasing meaning that it is harder for others to be strengthened

A more direct way of enforcing competition is through synaptic normalisation

Idea is that postsynaptic neuron can only support a certain amount of total synaptic weight so strengthening one leads to weakening others

Can either hold the sum of weights constant if all are +ve or –ve or can constrain the sum of squares of the weights (cf ANN network pruning)

2 types: subtractive normalisation and multiplicative normalisation

Synaptic normalisation

Where n is a vector of ones of length Nu so n.u is the sum of all the inputs u.

Thus the second term is simply a vector Nu long with the same values in ie (k, k, ….., k) whose sum over all the elements is equal to the sum over all the elements of vu. Thus the total increase in the weights is 0

This rule must be augmented by a saturation constraint to prevent the weights becoming negative, that if a weight becomes zero, it is not moved downwards

Also, without upper saturation often leads to all weights bar one being zero. Note also rule involves global knowledge of weights

Subtractive normalisation

uw N

nunvuv

dt

wd ).(

Where is a +ve constant: known as Oja’s rule (1982)

This rule is more local than previous as it only involves the weight in question and pre-and post synaptic activities.

However, its form is based on theoretical arguments rather than experimental data

Previous rule was rigid as it had to be satisfied at all times whereas this is more dynamic with |w|2 gradually relaxing to 1/

This induces competition as if one weight increases, the maintenance of constant length of the weight vector forces others to decrease

Multiplicative normalisation

wvuvdt

wdw

2

Both Hebbian and Oja’s rule run for long enough generate vectors parallel to principal eigenvector of correlation matrix as in A

This is basically principal component analaysis (PCA) which is theoretically the optimal in terms of retaining info way to encode high dimensional info onto lower dimensional subspace

However, B shows what happens if input vectors don’t have zero mean (as in real systems), but this problem is alleviated by using covariance-based rules

Previous rules don’t take timing into account

Can be crucial since if pre-synaptic spike occurs after postsynaptic get LTD rather than LTP

Timing based rules

dtutvHtutvHvdt

wdw ))()()()()()((

0

Therfore need to integrate over time as in the following:

Where H is a function like the solid line in previous figure

Such functions still require saturation constraints but timing can generate competition

Can extend the rules defined previously to nets with multiple postsynaptic neurons. In these networks the output rates are:

Multiple Postsynaptic Neurons

vMuWv Thus where 1)( MIKuKWv

And the Hebbian rule becomes:

KWQuvdt

dWw

Can also use feature-based models where the net is indexed by input features rather than buy individual inputs

Network models can have adaptive feedforward wieghts and fixed recurrent ones, or vice versa or both layers can be adaptive

Can get competition through mainly inhibitory recurrent connections

Can then get eg self-organising maps and elastic nets where K can have forms like: