The Perceptron

( )K tD

0 T

Affe

ren

ts

Vthr

V rest

tmax 0 tD

What does a neuron do?

spike

no spike

Affe

ren

ts

0 Ttmax-

VthrNull

We consider a simplified case: input is synchronous

Affe

ren

ts

Alternatively, input is constant

The perceptron

j jj

h W X 11 sgn

2 Y h

1X 2X 3X 4X

Y1W 4W

11 sgn

2

Y W X

Geometrical interpretation

11 sgn

21

sgn cos2

Y W X

1X 2X

Y

1W 2W

W

1W

2W

X

1X

2X

The perceptron

The Perceptron categorizes the space of inputs into inputs that should evoke a response and inputs that should not evoke a response

Constraints on possible categorizations

1 11 sgn 1 sgn

2 2

i i

i

Y W X W X

1X

2X

Constraints on possible categorizations

1X

2X

1 11 sgn 1 sgn

2 2

i i

i

Y W X W X

Constraints on possible categorizations

1X

2X

1 11 sgn 1 sgn

2 2

i i

i

Y W X W X

Constraints on possible categorizations

1X

2X

22 1 0 X X

Solution: change of coordination

21X

2X

22 1 0 X X

Solution: change of coordination

More complicated rules can be realized if an additional non-linear layer is added

Deerinck 2002

Llinas 1975

Ramon Y Cajal

Llinas 1975

Increasing network capacity?

The perceptron Learning algorithm

1 11 sgn 1 sgn

2 2

i i

i

Y W X W X

The perceptron Learning algorithm

• Algorithm starts with an arbitrary set of weights

• Examples are presented one by one

• If the Perceptron correctly classifies the example no change in synaptic weights

• If the Perceptron does not correctly classify the example then make a Hebbian change in weights:

The perceptron Learning algorithm

• If the example is to be classifies as ‘1’:

i i iW W X

• If the example is to be classifies as ‘0’:

i i iW W X

Perceptron.m

Hebbian plasticity and unsupervised learning

1X 2X 3X 4X

Y1W 4W

Unsupervised learning in linear neurons

Y W X

Hebbian plasticity

1 i i iW n W n X n Y n

Wi(n+1) = efficacy of synapse i after n updates

1X 2X 3X 4X

Y1W 4W

Y W X

is the plasticity rate

Geometrical interpretation

1X 2X

Y

1W 2W

W

1W

2W

X

1X

2X