Nucleus

Dendrite

Soma

AxonMyelin

Schwann Cell

Axon TerminalNode

NEURON

Soma: central part of neuron. Contains nucleus protein synthesisDendrites: input, some outputAxon: carries signal away from somaAxon terminal: releases neuro-chemical to communicate with other neurons

10m

Excitatory neurons: excite their target neurons. Excitatory neurons in the brain are often glutamatergic.

Inhibitory neurons: inhibit their target neurons. Inhibitory neurons are often interneurons. The output of some brain structures (neostriatum, globus pallidus, cerebellum) are inhibitory. The primary inhibitory neurotransmitters are GABA and glycine.

Modulatory neurons evoke more complex effects termed neuromodulation. These neurons use such neurotransmitters as dopamine, acetylcholine, and serotonin.

Image of pyramidal neurons in mouse cerebral cortex expressing greeen fluorescent protein. The red staining indicates GABAergic interneurons.

(Chemical) SYNAPSE: Electrical signal causes releases of chemicalsthat flow across

synapse and generate

electrical signalin next neuron.

Complexity of human brain and enormous number of possible states:

Synapses in brain: young children 1016 (10,000,000,000,000,000)

Stabilizes in adulthood by ½ to < 5 x1015

20 nm

TERMINOLOGY

Synapse is asymmetric:PRE-SYNAPTIC neuron secretes neurotransmitter chemicalsPOST-SYNAPTIC neuron binds neurotransmitters at receptors

Pre-synaptic neuronRelease of neurotransmitter triggered by the arrival of a nerve impulse Action potential

Cellular secretion, exocytosis, is very rapid

Pre-synaptic nerve terminal has docked vesicles docked at Membrane containing neurotransmitter

Arriving action potential produces influx of calcium ions through voltage-dependent, calcium-selective ion channels.

Calcium ions trigger biochemical cascade: vesicles fuse with presynapticmembrane and release their contents to the synaptic cleft.

Post-Synaptic neuron

Receptors on opposite side of synaptic gap bind neurotransmitter molecules

Respond by opening nearby ion channels in the post-synaptic cell

Ions rush in or out and change local transmembrane potential

Resulting change in voltage is called postsynaptic potential

Result is excitatory, in the case of depolarizing currents

inhibitory in the case of hyperpolarizing currents

Whether a synapse is excitatory or inhibitory depends on what type(s) of ion channel conduct the post-synaptic current display(s), which in turn is a function of the type of receptors and neurotransmitter employed at the synapse.

Following fusion of the synaptic vesicles and release of transmitter molecules into the synaptic cleft, the neurotransmitter is rapidly cleared from the space for recycling by specialized membrane proteins in the pre-synaptic or post-synaptic membrane.

This “re-uptake" prevents “desensitization" of post-synaptic receptors

Re-uptake ensures succeeding action potentials will elicit the same size post-synaptic potential (PSP)

Necessity of re-uptake and the phenomenon of desensitization in receptors and ion channels means that the strength of a synapse may diminish as a train of action potentials arrive in rapid succession

Frequency dependence synapses

Sounds bad but nervous system exploits this property for computational purposes, and tunes synapses through Protein phosphorylation; size, number and replenishment rate of vesicles

Example: serotonin re-uptake inhibitors (SSRIs) inhibit serotonin reuptake

NeurotransmittersCategorized into three major groups: (1) amino acids (primarily glutamic acid, GABA, aspartic acid &

glycine)

(2) peptides (vasopressin, somatostatin, neurotensin, etc.)

(3) monoamines (norepinephrine NA, dopamine DA & serotonin 5-HT) plus acetylcholine (ACh).

The major "workhorse" neurotransmitters of the brain are glutamic acid (=glutamate) and GABA.

Neurotransmitters can be broadly classified into small-molecule transmitters and neuroactive peptides.

Around 10 small-molecule neurotransmitters are known: acetylcholine, 5 amines, and 3 or 4 amino acids (depending on exact definition used), Purines, (Adenosine, ATP, GTP and their derivatives) are neurotransmitters.

Over 50 neuroactive peptides have been found, among them hormones such as

LH or insulin that have specific local actions in addition to their long-range signalling properties.

Single ions, such as synaptically-released zinc, are also considered neurotransmitters by some.

It is important to appreciate that it is the receptor that dictates the neurotransmitter's effect.

Some examples of neurotransmitter action:

•Acetylcholine - voluntary movement of the muscles

•Norepinephrine - wakefulness or arousal

•Dopamine - voluntary movement and emotional arousal

•Serotonin - memory, emotions, wakefulness, sleep and temperature regulation

•GABA (gamma aminobutyric acid) - motor behaviour

•Glycine - spinal reflexes and motor behaviour

•Neuromodulators - sensory transmission-especially pain

Acetylcholine

Serotonin

Dopamine

Neuronal Patterns 

1.      How a neuron can act as a decision-making device; 2.      In what sense does a neuron recognizes a pattern; 3.      How do input weights and threshold interact to determine the set of patterns recognized by

a neuron; 4.      Limitations of the linear discrimination performed by neurons. 

Model of a neuron: McCulloch-Pitts neuron

Not biologically neurons per se, but embody fundamental properties of biologically based information processing (e.g., the weighted summation of excitatory inputs is converted to a binary signal).

Able to recognize patterns in its environment based on the features of a given stimulus

Cognitive processes that are modeled using these neurons are pattern recognition and decision-making.

Understanding of how the brain can produce cognition.

Use a geometric space (Euclidean distances) to quantify features and patterns

A feature (e.g., a person’s weight, height, hair length, bank account, or grade point average) is a quantitative aspect of the total pattern we associate with that individual.

A feature can be an axis in a higher dimensional geometric space.

Qualitative features can also be valued, as zero or one, for the absence or presence of a feature (e.g., ‘has a driver’s license’)

We will only use non-negative values to quantify a feature that corresponds to simple interpretations of a neuron’s output, i.e., spike or no spike, spike frequency, or inverse interpulse interval.

Ordered set of such features is called a n-dimensional vector as a point in n-dimensional space.

Re: vector is a point in a geometric space and an ordered set of scalar values

Because these values are ordered, a vector can be called a pattern

Any pattern can be expressed as a vector

Terminologywhat goes into a network: input, input pattern, input vector

neural network : an organized system of neurons

Operate on patterns in very high-dimensional spaces—e.g., many neurons in the neocortex receive 5,000-20,000 excitatory inputs and thus process patterns in 5,000-20,000 dimensions.

Each neuron is part of a network that processes information in an even higher dimensional space.

For organized, functionally delimited brain regions, such as human forebrainthe dimension ranges from 500,000 to 5,000,000 (high dimensional vector) but

Similarity still based upon separation distance

Individual Neurons Make Decisions.

Synapses made between individual presynaptic inputs and postsynaptic neuron transform that input.

Neuron’s threshold for action potential generation plays important role in determining a neuron’s decision.

Assume neuron is dedicated to recognizing a family of closely related similar) patterns

Each active input to a postsynaptic neuron contributes to the excitation of that postsynaptic neuron in proportion to the synaptic strength or weight, which connects that presynaptic input to the postsynaptic cell.

Triggering of a postsynaptic neuron: neuronal decision-making is a voting system

Each presynaptic input neuron casts its vote for the firing of the postsynaptic neuron, either by firing its own action potential or by not firing.

The postsynaptic neuron tallies the votes over the set of inputs—a ‘no’ vote is a zero and a ‘yes’ vote has some positive value (= 1 in simplest model).

If tally is large enough, postsynaptic neuron declares ‘yes’ and it fires

If the tally is not large enough, the postsynaptic neuron says ‘no’ and does not fire.

But, not all votes are equal neuronal vote is not a true democracy.

Some ‘yes’ votes count more than others do.

Example, an affirmative decision by a postsynaptic neuron may require only a relatively small number of ‘yes’ votes from its heavily weighted inputs.

Use McCulloch-Pitts neurons to study pattern recognition

Neurons function in discrete time intervals and have no memory of their past excitation.

i.e. assume that successive input sets, and the postsynaptic decisions they drive, occur in discrete, sequential time intervals (an input set contains one or more input patterns).

Postsynaptic neuron makes a decision at each time interval based solely on its current input weights, its active inputs, and its threshold.

Net synaptic excitation of a McCulloch-Pitts neuron is a linear function of its inputs.

Each synaptic connection scales its axon’s value and then the postsynaptic neuron adds up these scaled values.

Each postsynaptic neuron has its own distinct summation process, which converts the current set of active synapses into a scalar value termed the internal excitation or net input excitation (or just excitation for short).

However, the output of such a McCulloch-Pitts neuron is either zero or one.

Crude binary encoding of the internal excitation:neuron fires (output of one) when its net input excitation exceeds a certain value called ‘threshold.’

Threshold is the minimum value of the sum of the weighted active inputs needed for the postsynaptic neuron to fire.

All neurons in model operate on same temporal cycle.

One computational cycle of a network for the weighted summation of active inputs and the thresholding to occur.

On the next computational cycle of the simulation, this process is repeated.

Remember that a postsynaptic neuron does not remember what happened on the last cycle (note that we use the phrase ‘computational cycle’ to describe computations by the network model)

NEURON NETWORK

Assess performance on pattern recognition problems

Look at simple network:

Single postsynaptic neuron receives connections from three different presynaptic inputs.

Computational elements of a McCulloch-Pitts neuron

Neuron, including its inputs, is a parallel computational device

A neuron with n inputs performs n multiplications and n-1 additions at each discrete time step.

The multiplications are performed in parallel—each synapse multiplies the incoming excitation from each input by its weight.

McCulloch-Pitts neurons: these products are summed instantaneously with the total ending up in the cell body.

Neuron then compares this summated excitation with its threshold, and this comparison decides whether or not the neuron fires.

On the next time step, the whole process begins again.

Re: a McCulloch-Pitts neuron forgets its summated internal excitation from one time step to the next.

The minimum, quantitative description of a McCulloch-Pitts neuron requires:

1. The set of all synaptic weights which defines the potency of each presynaptic input to the neuron

2. The threshold for firing this neuron  Recapitulate: a McCulloch-Pitts postsynaptic (output) neuron

1. Has no memory from one time step to the next

2. Within a time step, it linearly sums its inputs as scaled by their respective synaptic weights

3. It fires (output=1) if the value of its internal excitation exceeds a preset threshold value of excitation, otherwise it does not fire (output=0).

A single McCulloch-Pitts neuron with: three inputs (1, 2, and 3 that take on values x1, x2, and x3, respectively).

Each synapse is a connection formed between a presynaptic axon (input line) and the single postsynaptic dendrite (indicated by the darkened circular symbol, .

The indicates the summation of internal excitation (analogous to the altered polarization of neuronal cell body).

The output of the postsynaptic neuron is another axon that transmits any action potentials of the postsynaptic neuron.

postsynaptic dendrite

presynaptic

Summarize: McCulloch-Pitts neuron consists of:  Its synaptic weights that scale its excitatory inputs,  A summation process that adds up the scaled inputs,  A spike generator that crudely encodes the value of this summation.

Two Types of Variables in Simulations1. Variables whose values you choose 2. Variables whose values the simulation generates

In the first case: we parameterize the computational elements

In the second case: variables are continuously changed at successive time steps by the computational steps of the simulation.

 Parameterized variables that we choose for each postsynaptic neuron includes: 1.      the value of each synaptic weight, and2.      the firing threshold.

The category of the calculated, changing values of a neuron includes time itself and1.      the internal excitation of each neuron, and2.      the neuron’s output (fire or not fire).

Simple Simulation: computational parameters that you specify are fixed for any one simulation (pattern recognition), and there is no synaptic modification (no learning).

Internal excitation—a linear computation

What happens when an input vector arrives at a postsynaptic neuron? The internal or postsynaptic excitation of a McCulloch-Pitts neuron is the linearly scaled sum of its inputs

Individual synaptic weights specify the scaling on a connection-by-connection basis (the terms synaptic weight and synaptic strength are interchangeable).

Example: input at time step one

synaptic weights of a postsynaptic neuron named j

Figure on next slide

x 110

0 .5

w j

0 .10. 30 . 2

A single McCulloch-Pitts neuron

On one time step the inputs take on values xi where i{1,2,3}

Each synapse, wij, is formed by a presynaptic input line i connecting to postsynaptic

neuron j.

Here each input xi(t) makes a single synapse with the postsynaptic neuron called j.

Inputs x depend on state of presynaptic neurons and are time dependent.

To keep notation simple, the time notation, t, has been suppressed in diagram.

Excitation (next slide)

EXCITATION

yi(1): the internal excitation of postsynaptic neuron i at time step one (reverse subscripts meaning)

Excitation: the sum of the inputs scaled by their respective synaptic weights

3

10.2.0.100.10.20.50.300.1111

=jijji =++=++=)w(x=)(y

ij

n

=jji wx=y

1

Time notation is suppressed

Notation for a synapse: wij woutput,input (note subscript notation)

Synapse is the connection between two neurons requires two integers to specify

Subscript j indicates a specific input neuronSubscript i indicates a specific output neuron

This example has three synapses:  w11, the synapse connecting input neuron 1 with output neuron 1;

w12, the synapse between input neuron 2 and output neuron 1;

w13, the synapse between input neuron 3 and output neuron 1.

 

THRESHOLD

Threshold: parameter of a neuron that defines the minimum excitation needed for that neuron to fire.

Logical operation that compares two valuesif excitation of neuron is greater than its threshold value (logical true) then that neuron fires an action potential; if it is false, the neuron does not fire.

PATTERN RECOGNITION 

Pattern: x-vector giving state (x values) of group of pre-synaptic neurons

Pattern Recognition: Post-synaptic neuron fires in response to input from x-vector

THOUGHT PROCESS (Consciousness?)Parameterize the synaptic weights and the threshold of the post-synaptic neuron to recognize a pattern(s) successfully

First Example (trivial)All synaptic weights (w) are positive and threshold=0

neuron fires in response to all input patterns (even x=0)

Trivial and uninteresting because neuron can only make one decision — it recognizes everything (fires at every time step)

neuron transmits no information (response is independent of input pattern x)

Neuron is not wrong, just boring

y

w1

w2 w3 w4

x1 x2 x3 x4

y1

x2x1

y2

w11 w22

w12w21

How to parameterize model to recognize different patterns

Example: Two different simple patterns x1>x2 or x1<x2

Pattern recognition: neuron y1 fires if x1>x2 neuron y2 fires if x1<x2

To recognize x-patterns, Parameters that can be varied are: weights (w) and thresholds (h)

Define problem: which parameter is fixed versus which do you need to determine

Re: xw.y i.e.

n

ijiji xwy

1

Example: assume w are pre-determined based upon considerations such as physiological information

Method for determining thresholds h1 and h2 to recognize patterns

Extremely simple example:

Normalize w and x each to 1 i.e. xi2 =1 and i wij

2 = j wij2 = 1

(This is not a real restriction, just normalization)

Simple weights

Neuron x1 can only excite y1 w11=1, w21=0

Neuron x2 can only excite y2 w12=0, w22=1

1

0

0

1

22

12

21

11 =

w

w

w

w

y1

x2x1

y2

w11 w22

Draw unit circle and place w1j and w2j

Next: on circumference, locate point that corresponds to boundary between two patterns

Boundary conditions xb: x1=x2 and x12+x2

2=1 x12+x1

2=2x12=1

x1=1/2=0.707 x2=0.707 tan=.707/.707=1 =45o

Drop perpendicular from xb to w1 vector: intersection is h1=(1) cos

Drop perpendicular from xb to w2 vector: intersection is h2=(1) sin

w1=(1,0)

w2=(0,1)

x1

x2

xb

45o

h1= 1/2 =.707

h2= 1/2 =.707

Click mouse on screen for slide to evolve:

Graphical Method becomes complicated for wij0, and hard to visualize for > 2 input neurons (> 2-D graph)

General Mathematical Way for Determining Thresholds

Relax restrictions that required i wij2 =j wij

2 =1 but maintain xi2 =1 (can relax later)

Still have basic relationship between input (x), weights w, and output excitation y

i.e.

n

jjiji xwy

1

Calculating thresholds h requires knowledge of boundary input vector xb

Iff xb is known then thresholds h are calculated from

i.e.

n

jbjiji xwh

1

xbi are elements of boundary vector

Example: next page

h=w•xb

y=w•x

Determine h1 and h2 for a network

y1 to recognize the pattern (fire) if : x1>2x2 and

y2 to recognize the pattern (fire) if : x1<2x2 xb =

but

New wiring diagram (weights) with all (forward) connections

y1

x2x1

y2

w11 w22

w12w21

h1=w11xb1+ w12xb2 h2=w21xb1+ w22xb2

Example: w11=0.75 w12=0.23 h1 = (0.75)(.894)+(0.23)(0.447) = 0.773 w21=0.45 w22=0.62 h2 = (0.45)(.894)+(0.62)(0.447) = 0.679

Check: x1=0.80 x2=0.60 x1<2x2 x1=0.95 x2=0.31: x1>2x2

y1= (0.75)(.80)+(0.23)(0.60) = 0.738 <h1 does not fire y1=0.784 firesy2= (0.45)(.80)+(0.62)(0.60) = 0.732 >h2 fires y2=0.62 no fire

.447

894.

Click mouse on screen for slide to evolve: