a supervised learning approach based on stdp and ... -...

A supervised learning approach based on STDPand polychronization in spiking neuron networks

Hélène Paugam-Moisy1, Régis Martinez1 and Samy Bengio2

1LIRIS - CNRS - Université Lumière Lyon 2Lyon, France

http://liris.cnrs.fr

2IDIAP Research InstituteMartigny, Switzerland

http://www.idiap.chSamy is now at Google

ESANN 2007 - April, 27

http://liris.cnrs.fr

http://www.idiap.ch

Motivations Problematics Network architecture Learning mechanisms Results (1) Polychronization Results (2) Conclusion

Plan

1 Motivations

2 Problematics

3 Network architecture

4 Learning mechanisms

5 Results (1)

6 Polychronization

7 Results (2)

8 Conclusion

Régis Martinez A supervised learning approach based on STDP and polychronization in spiking neuron networks 2 / 31


Plan

1 Motivations

2 Problematics



5 Results (1)

6 Polychronization

7 Results (2)

8 Conclusion



Motivation

In Spiking Neuron Networks (SNNs), informationprocessing is based on the times of spike emissions.

SNNs are a very powerful new generation of artificial neuralnetworks but efficient learning in SNNs is not straightforward.

A current track is to simulate the synaptic plasticity, as canbe observed by neurobiologists [Bi and Poo,1998] but thismethod lacks supervised control of learning.



Theoretical fundations

Theoretically, the use of delays increases the learningcapacity of SNNs...[Maass, 1997] [Schmitt, 1999]

... but delays are rarely used in SNN models

Recent advances in neural networks (ESN [Jaeger, 2001],LSM [Maass et al, 2002]) give interesting results

The concept of polychronization emphasizes theimportance of delays for explaining neural activity[Izhikevich, 2006]



Plan

1 Motivations

2 Problematics



5 Results (1)

6 Polychronization

7 Results (2)

8 Conclusion



Problematics

A better computational power is a good point, but what about thelearning algorithm ? How to take advantage of the computationalpower of delays ?

We take advantage of polychronous groups activations tomonitor activity in the networkWe define a supervised1 learning mechanism to control thecomputational power of a SNN

Polychronization will help us monitor and understand the networkactivity.

1simplest way for us to show that polychronization can actually be a reliableinformation coding



Plan

1 Motivations

2 Problematics



5 Results (1)

6 Polychronization

7 Results (2)

8 Conclusion



The model

Maintains biological plausibility within the internal networkNeuron model : Spike Response Model (SRM0)[Gerstner 1997]Inspired from LSM/ESN architectures :- input layer of spiking neurons- recurrent randomly connected internal network- output layer which supports a supervised learning rule

.

.

.

class 2

class 1

2 output cellsK input cells

Internal network

M internal cells

input connections

internal connections

output connectionswith adaptable delay



The model

.

.

.

class 2

class 1

M internal cells

input connections


with adaptable delay

K input cellsInternal network

2 output cells

output connections

Input layer (stimulation layer) :10 neuronsInput injection



The model

.

.

.

class 1


Internal network

M internal cells

input connections



class 2

Internal Network :100 neurons, 80% excitatory, 20% inhibitoryRandom recurrent topologyConnection delays fixed (but randomly chosen) between 1and 20 ms



The model

.

.

.

class 2

class 1


Internal network

M internal cells


input connections


Output layer :2 neurons : one for each target classrecieves a connection from each internal neuron



The model

.

.

.

class 2

class 1


Internal network

M internal cells

input connections



Tested on a classification task

Two input patterns :Target neuron must fire before non-target neuron

20 msTime [ms]

20 msTime [ms]

Inpu

t neu

rons

Inpu

t neu

rons

Stimulation pattern 1 Stimulation pattern 2



Plan

1 Motivations

2 Problematics



5 Results (1)

6 Polychronization

7 Results (2)

8 Conclusion



A two scale learning algorithm

.

.

.

class 2

class 1


Internal network

M internal cells

input connections



1 Unsupervised learning : Spike Time Dependent Plasticity(STDP) within the internal network (ms time scale) [Kempteret al., 1999]

2 Supervised mechanism : delay adaptation on outputconnections (at each input presentation) based on a margincriterion [Vapnik, 95]



1. Unsupervised learning algorithm

Unsupervised learning : Spike Time Dependent Plasticity(STDP) within the internal network (ms time scale)

Temporal hebbian rule, suitable for SNNsAt the synaptic level (local mechanism)Depending on activity going through the synapseCausality based on spike emissions order



2. Supervised learning algorithm

.

.

.

class 2

class 1


Internal network

M internal cells

input connections



After the presentation of a given input pattern p,If target/non-target spikes order is OKANDIf margin between target/non-target spikes > ε

Then : pattern is well classifiedOtherwise,• for target neuron : decrement the delay (−1ms)• for non-target neuron : increment the delay (+1ms)



Plan

1 Motivations

2 Problematics



5 Results (1)

6 Polychronization

7 Results (2)

8 Conclusion



Simulation protocol

Initial noisy stimulation : noise presented during 300 msLearning phase : alternated presentation of two patternsGeneralization phase : alternated presentation of the twonoisy patterns

NB : One presentation every 100 ms



Initialization phase

0

20

40

60

80

100

120

0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000

Neu

ron

ID

Time [ms]



Learning phase observation

Decreasing internal activity (STDP)Activity pattern different from an input to the otherMargin evolution

0

20

40

60

80

100

120

8500 8600 8700 8800 8900 9000 9100



Generalization performance

Error rate with noise 4 : 4%Error rate with noise 8 : 19%Hard to discriminate by human

0

20

40

60

80

100

120

18900 19000 19100 19200 19300 19400 19500 19600



Plan

1 Motivations

2 Problematics



5 Results (1)

6 Polychronization

7 Results (2)

8 Conclusion



Polychronization [Izhikevich, 2006]

Definition : neuron interactions characterized by spike timesfollowing a precise temporal pattern, depending on delays.

Example :

N2

15ms

8ms

N1

N3Time [ms]

8 ms

15 ms

N3

N2

N1

If N1 emits a spike at t, and N3 at t + 7, then N2 emits a spikeat t + 15.

A set of such interacting neurons is called a polychronousgroup.



Scanning for supported polychronous groupsStructurePolychronous groups are supported by the topology.

connections between neuronsdelays of the connections

A given topology = a particular set of supportedpolychronous groupsEach neuron can be involved in several polychronous groups

To find all supported polychronous groups, we use the samealgorithm as [Izhikevich 2006].

Dynamicsset of supported polychronous groups 6= set of activatedpolychronous groups



Plan

1 Motivations

2 Problematics



5 Results (1)

6 Polychronization

7 Results (2)

8 Conclusion



Polychronous groups activations

0

20

40

60

80

100

60 65 70 75 80 85 90 95 100

Perc

enta

ge o

f ac

tivat

ion

# polychronous groups

activations in response to class 1activations in response to class 2

0

20

40

60

80

100

60 65 70 75 80 85 90 95 100Pe

rcen

tage

of

activ

atio

n

# polychronous groups

activations in response to class 1activations in response to class 2

Figure: Activation ratio from 2000 to 5000 ms, and then from 8000 to11000 ms.



Plan

1 Motivations

2 Problematics



5 Results (1)

6 Polychronization

7 Results (2)

8 Conclusion



Conclusion

Algorithm easy to implementThe learning seems to work on a classification taskEasily explained by polychronizationActivity easily monitored with polychronous groupsInternal network is no longer a black-box contrary to ESNand LSM



Perspectives

Topology

Dynamics

Polychronousgroups STDP

Complex network analysis :Are polychronous groups the (or a part of the) link betweentopology and dynamicsHow far ?



Thank you for listening.

Questions !

A supervised learning approach based on STDPand polychronization in spiking neuron networks– Hélène Paugam-Moisy, Régis Martinez and Samy Bengio


Appendix

Plan9 Appendix

Work in progressReservoir Computing perspectivesGroupes polychrones sur 100 neuronesModèle SRM0Modèle SRM1Forme d’un PPSRéseau expérimentalSensibilité à un motif spécifiqueFenêtre STDP EurichFenêtre STDP classiqueFenêtre STDP MeunierStabilité du classifieurCodage temporelArchitectureActivation des groupes polychronesActivité neuronalePG detectionThe model proposedOriginal problemDifference with synfire chainNetwork activity


9 AppendixWork in progressReservoir Computing perspectivesGroupes polychrones sur 100 neuronesModèle SRM0Modèle SRM1Forme d’un PPSRéseau expérimentalSensibilité à un motif spécifiqueFenêtre STDP EurichFenêtre STDP classiqueFenêtre STDP MeunierStabilité du classifieurCodage temporelArchitectureActivation des groupes polychronesActivité neuronalePG detectionThe model proposedOriginal problemDifference with synfire chainNetwork activity

Work in progress

Use larger inputs : encouraging tests with USPS dataset

2 versus 7 : 96% success on train set, 93% on test set3 versus 8 : 89% success on train set, 86% on test set

Switch to more than two classesExtend model with persistant activity

retour

Appendix

Reservoir Computing perspectives : Open questions

Might there be links with reservoir computing. Indeed, sometheoretical properties exists : point-wise separation, universalapproximation, echo state properties...But still difficulties to investigate what’s going on in the reservoir(refering to special session)Polychronous groups can be a reliable way

to analyse dynamics of a spiking neuron reservoirto find optimal topologies (structures)

retour


Groupes polychrones sur 100 neurones

0

20

40

60

80

100

0 10 20 30 40 50 60 70 80

Neu

rone

s

Temps

0

20

40

60

80

100

0 10 20 30 40 50 60 70 80

Neu

rone

s

Temps

[48] 18,24,80 (0,11,11) ==> 37 (16) — [49] 19,31,43 (3,0,5)==> 6 (6)

0

20

40

60

80

100

0 10 20 30 40 50 60 70 80

Neu

rone

s

Temps

0

20

40

60

80

100

0 10 20 30 40 50 60 70 80

Neu

rone

s

Temps

[50] 19,55,76 (0,11,13) ==> 70 (16) — [51] 21,52,76 (7,7,0)==> 11 (12)

retour

Modèle SRM0 (used)

uj(t) = η(t − tfj )︸︷︷︸

A : refractory periode

+∑

i

wij ε(t − tfi − dij)︸︷︷︸

B : excitatory potential

retour

Modèle SRM1

uj(t) = η(t − tfj )︸︷︷︸

A : refractory periode

+∑

i

wij

∑f

ε(t − tfi − dij)︸︷︷︸

B : excitatory potential

retour

Forme d’un PPS

0

0.2

0.4

0.6

0.8

1

-1 0 1 2 3 4 5

Ups

p [m

V]

Temps [ms]

exp(-x/Tau)

retour

Réseau expérimental

retour

Sensibilité à un motif spécifique

1

2

3

4

5

d = 21

d = 15

d = 8

d = 1

t+6 t+13 t+20 t+21

21

15

8

8

1

temps [ms]

Neuronesdu sac

arrivée au neurone de sortie 1

t

Neurons

1

2

3

4

5

Neurone de sortie 1

d = 8

retour

Fenêtre STDP Eurich

retour

Fenêtre STDP classique

1.0

−1.0

20 mstpost − tpre [ms]

−20 ms

décalage temporel de la synapse (tpost − tpre)

augmentation du poids

Delta W

retour

Fenêtre STDP Meunier

Si ∆W ≤ 0, le poids est augmenté :wij ← wij + α ∗ (wij − wmin) ∗∆W

Si ∆W ≥ 0, le poids est diminué :wij ← wij + α ∗ (wmax − wij) ∗∆W

1.0

POTENTIATION

DEPRESSION −0.5

t

W

10ms 20ms

100ms

1.0

t

W

20ms−20ms

−0.25DEPRESSION DEPRESSION

POTENTIATION

+infini−infini

retour

Stabilité du classifieur

0

5

10

15

20

25

30

0 100000 200000 300000 400000

Stab

ilite

des

repo

nses

Temps

Diagramme de Stabilite des reponses

0

5

10

15

20

25

0 100000 200000 300000 400000St

abili

te d

es re

pons

es

Temps

Diagramme de Stabilite des reponses

retour

Codage temporel

cd

ba a

bc

dtt−1

Neurones

Temps

Codage temporel

Composantes du vecteur

Codage en intensité

Vecteur numérique

Intensité

Vague de spikes dans un intervallede codage temporel

retour

Architecture

.

.

.

class 2

class 1


Internal network

M internal cells

input connections



retour

Activation des groupes polychrones

0

5

10

15

20

25

30

35

40

45

50

55

60

65

70

75

80

85

90

95

100

105

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 12000 13000 14000 15000 16000 17000 18000 19000 20000

Poly

chro

nous

gro

ups

t [ms]

retour

Activité neuronale

retour

PG detection

To find all supported polychronous groups, we use the samealgorithm as [Izhikevitch 2006]. It consists in scanning for spiketime combination of all groups possible of 3 neurons (i.e.combinatorial quiestions), so that the spikes would trigger thefiring of one or more impacted neurons, taking axonal delays intoaccount.

Il est possible de procéder de même en cherchant plus dedéclencheurs, mais la complexité est accrue: O(np), avec pnombre de déclencheurs. retour

The model proposed

.

.

.

class 2

class 1

M internal cells

input connections


with adaptable delay

K input cellsInternal network

2 output cells

output connections

Input layer (stimulation layer) :10 neuronsOutgoing connection probability : 0.1Delay to central assembly : 0 ms

retour

The model proposed

.

.

.

class 1


Internal network

M internal cells

input connections



class 2

Central assembly :100 neurons, 80% excitators, 20% inhibitorsRandom topologyReccurent connection probability : 0.3Recurrent connections delay from 1 to 20 msSpike Time Dependent Plasticity (STDP)

retour

The model proposed

.

.

.

class 2

class 1


Internal network

M internal cells


input connections


Output layer :2 neurons : one for each target classIncoming connection probability : 1 (central assemblycompletely projected)Adaptable delays of input connections (all initialized to 10ms)

retour

Initial work

Originally : problem for learning binary patternsSpike responses : all or nothingSolution : allow diversity in axonal delays

0

100

200

300

400

500

0 20 40

Neu

rone

s

Temps

Diagramme de Pulses

retour

Difference with synfire chainin Synfire Chains and Catastrophic Interference – J. Sougnéand R. French (2001) :

when an initial neuron, A, fired, a second neuron, B,would fire 151ms later, followed by a third neuron, C,that would fire 289ms later with a precision across trialsof 1 ms

in Polychronization : computation with spikes – E. Izhikevich(2006) :

Synfire chains describe pools of neurons firingsynchronously, not polychronously. Synfire activity relieson synaptic connections having equal delays or nodelays at all. Though easy to implement, networkswithout delays are finite-dimensional and do not haverich dynamics to support persistent polychronousspiking.

retour

Initialization phase

0

20

40

60

80

100

120

0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000

Neu

ron

ID

Time [ms]

retour

Learning phase observation

Decreasing internal activity (STDP)Activaty pattern different from an input to the otherMargin evolution

0

20

40

60

80

100

120

2300 2400 2500 2600 2700 2800 2900 3000 3100 3200 8500 8600 8700 8800 8900 9000 9100

retour

Generalization performance

Error rate with noise 4 : 4%Error rate with noise 8 : 19%

0

20

40

60

80

100

120

18500 18600 18700 18800 18900 19000 19100 19200 18900 19000 19100 19200 19300 19400 19500 19600

retour

a supervised learning approach based on stdp and ... -...

Documents