A STOC!XASTIC MODEL O F N O N L I N m R L Y

I N T E R A C T I N G NEURONS

T e r r e n c e J'oseph Se jnowski

A D I S S E R T A T I O N

PRESENTED TO THE

FACULTY O F PRINCETON m I V E R S I T Y

I N CANDIDACY FOR THE DEGREE

O F DOCTOR OF PHILOSOPHY

RECOMMENDED FOR ACCEPTANCE BY THE

DEPARTMENT O F

P H Y S I C S

iii

A Stochastic Hodel of N o n l i n e a r 1 y I n t e r a c t i n g Neurons

T e r r e n c e J o s e p h ~e jnovsti

Neurons i n t h e v e r t e b r a t e n e r v o u s s y s t e m r e s p o n d t o

i n p u t s nonlinearly, are h i g h l y and p r e c i s e l y interconnected,

a n d h a v e a r a n d o m l y f l u c t u a t i n q c o m p o n e n t i n their membrane

potentials, T h e H a r t L i n e - R a t l i f f model of the Limlus

r e t i n a i s g e n e r a l i z e d t o i n c o r p o r a t e t h e s e three fundamental

p r o p e r t i e s , B e c a u s e n o t a l l n e u r o n s p r o d u c e a l l - o r -none

action p o t e n t i a l s , t h e membrane p o t e n t i a l r a t h e r than t h e

f i r i n g r a t e i s taken a s t h e p r i m a r y variable i n the model,

However, the a v e r a g e f i r i n g rate, w h i c h is a n i r a p o r t a n t

m e a s u r e m e n t in many experiments, p l a y s a k e y r o l e i n

c o u p l i n g t h e equation which g o v e r n s t h e a y e r a g e membrane

p o t e n t i a 1s w i t h t h e e g a a t i o n w h i c h g o v e r n s the c o r r e l a t i o n s

b e t w e e a membrane p o t e n t i a l s .

Hany n e u r o n s i n v i s u a l c o r t e x r e s p o n d b e s t t o special

p a t t e r n s of l i g h t which a r e c a l l e d " t r i g g e r f e a t u ~ e s " . The

p r o p e r t i e s of s u c h " t r i g g e r f e a t u r e s " a r e examined i n a

v a r i a t i o n a l a n a l y s i s o f t h e s t e a d y - s t a t e e q u a t i o n f o r t h e

a v e r a g e r 3 m b r a n e p o t e n t i a l s . T h i s e q u s t i o n a l w a y s h a s a t

l e a s t one s o l u t i o n a n d an e x a m p l e i s a n a l y z e d w h i c h

d e m o n s t r a t g s b i f u r c a t i o n s t o m u l t i p l e s t a b l e s o l u t i o n s and

h y s t e r e s i s b e t w e e n t h e m . A r e c e n t c o o p e r a t i v e model of

b i n o c u l a r d e p t h p e r c e p t i o n (Piar r and P o g g i o , 1975) is

i d e n t i c a l v i t h the s t e a d y - s t a t e e q u a t i o n analyzed here; a

t ime-dependen t g e n e r a l i z a t i o n of t h e a o d e l is g i v e n which

s h o u l d have b e t t e r p s r f o r n a n c e v i t h dynamic i n p u t .

I f t h e membrane p o t e n t i a l s of a g r o u p o f n e u r o n s have

G a u s s i a n d i s t r i b u t i o n s , which i s a r e a s o n a b l e a s s u m p t i o n i n

a h i q h l y i n t e r c o n n e c t e d a r e a s u c h a s c e r e b r a 1 c o r t e x , then

the e q u a t i o n f o r the c o v a r i a n c e s between membrane p o t e n t i a l s

u n e x p e c t e d l y becomps l i n e a r a n d r e p r e s e n t s a l i n e a r f i l t e r .

I f t h i s s t a t i s t i c a l a s s u m p t i o n i s e x p e r i m e n t a l l y con f i rmed ,

t h e n control and corarnunication t h e o r y i n s y s t e m s e n g i n e e r i n g

n a y have d i r e c t a p p l i c a t i o n s i n some p a r t s o f t h e b r a i n .

T h e model is a p p l i e d t o motor l e a r n i n g i n t h e

c e r e b e l l u n and a p r e d i c t i o n is made c o n c e r n i n g t h e

u n d e r l y i n q p l a s t i c i t y : t h e c h a n g e i n s y n a p t i c strength

between a p a r a l l e l f i b e r and a P u r k i n j e c e l l s h o u l d be

p r o p o r t i o n a l t o t h e c o v a r i a n c e between i m p u l s e s i n the

p a r a l l e l fiber and the P u r k i n j e ce l l ' s c l i m b i n g f i b e r .

Un l ike p r e v i o u s p r o p o s a l s f o r s y n a p t i c p l a s t i c i t y i n t h e

c e r e b e l l u m , t h i z p r e d i c t i o n cequires b o t h f a c i l i t a t i o n and

d e p r e s s i o n t o o c c u r ( u n d e r d i f f e r e n t c o n d i t i o n s ) a t t h e same

s y n a p s e ,

Bcknowled geaents

I am indebted t o John 53heeler uho made t h e i n i t i a t i o n

of this w r k possible. I want to t h a n k ray parents and a l l

my fxiends for their support and encouragement daring its

completion, and I am e s p e c i a l l y gra te fu l t o David Bender,

Alan G e l p e r i n , Charles Gross, John Bopfie ld, Bruce Knight,

J r . , and Murray Lampert f o r their generous h e l p .

i a b l e of C o n t e n t s

A b s t r a c t

Acknowledgements

T a b l e o f C o n t e a t s

L i s t o f I l l u s t r a t i o n s

I n t r o d u c t i o n

Neurons and Synapses

Slow P o t e n t i a l s a n d Stochastic V a r i a b i l i t y

The LimuZus R e t i n a

Gene ra l i z i nq t h e LimuZus n o d e l

I. T h e Nonlihear model

Neurons and tf21icrocircuits19

Q u a n t i t a t i v e Model

I Averaqe Hembrane Potentials i n S t e a d y State

S t o c h a s t i c Hodel

S t e a d y - S t a t e Solutions

Feature D e t e c t i o n

Stereopsis

iii

v

vi

v i i i

111. C o r r e l a t i o n s between neabr ane P o t e n t i a l s

C o r r e l a t i o n s

C o v a r i a n c e E q u a t i o n

S t a t i o n a r y Case

Heuronal Filters and Memory

IV. Hotor L s a r n i n g i n the Cerebellum

C e r e b e l l a r C o r t e x

S t o r i n g Covariance

Synaptic P l a s t i c i t y

Lonq- term Memory

D i s c u s s i o n

A p p e n d i x : P o i n t - P r o c e s s n o d e l

References

vii

82

v i i i

List of I l L u s t r a t i o n s

Page F i g u r e

G o l q i - s t a i n e d n e u r o n s R s p r e s e n t a t i v e v e r t e b r a t e neuron E E G r e c o r d i n g s A c t i c n p o t e n t i a l Chemical s y n a p s e s Slow p o t e n t i a l p r o c e s s i n g V e r t e b r a t e r e t i n a : anatomy V e r t e b r a t e r e t i n a : physiology S i n g l e neuron i n c a t v i s u a l cortex P r og neuomuscula r j u n c t i o n LimuZus o n m a t i d i u ~ LimZus r e t i n a r e s p o n s e t o l i g h t LimuZus r e t i n a lateral i n h i b i t i o n LinnZus r e t i n a r e s p o n s e t o s t e p i l l u ruinat ion ~imuZus r e t i n a r e s p o n s e t o rnodul3ted input LimuZus r e t i n a t r a n s f e r f u n c t i o n s Hembrane p o t e n t i a l a s a f u a c t i o n o f time F i r i n g r a t e a s a function of ~ e m b r a n e p o t e n t i a l S c h e m a t i c drawing of two i n t e r a c t i n g n e u r o n s Averaqe f i r i n g rate and its d e r i v a t i v e G e n e r a l tuo-neuron model S o l u t i o n s of t h e two-neuron model H y s t e r e s i s i n t h e t w o-neuron model Random-dot s t e r e o g r a r~ D e p t h l o c a l i z a t i o n Geometrg of t h e d i s p a r i t y a l g o r i t h n Summary o f the physical and s t a t i s t i c a l v a r i a b l e s Block d i a g r a m of a n e u r o n a l f i l t e r Single n e u r o n s i n cat l a t e r a l g e n i c u l a t e n u c l e u s G r c s s cerebellum B a s i c e l e m e n t s i n t h e c e r a b e l l a r c o r t e x Patch of P u r l t i n j e c e l l d e n d r i t e C o r r e l a t i o n s i n t u o climbing fibers instantaneous r a t e u A f t e r p o t e n t i a l f o l l o w i n g a n a c t i o n p o t e n t i a l

X n t r o d u c t i o e

The n e u r c n a l b a s i s of a n i m a l b e h a v i o r i s l a r g e l y a

mystery d e s p i t e t h e s i g n i f i c a n t p r o g r e s s t h a t has b e e n ~ a d e

i n o u r knowledge o f t h e b r a i n ' s s t r u c t u r e , Each component

of t h e n e r v o u s s y s t e m h a s b e e n p a i n s t a k i n g l y studied, b u t

o n l y r e c s n t l y h a s i t been p o s s i b l e t o e x p l o r e i n t e g r a t e d

b e h a v i o r d u r i n g w h i c h t h e components of t h e n e r v o u s system

work t o g r t h s r i n c o n c e r t . The t u o q u e s t i o n s a d d r e s s e d by

this s t u d y a r e f i r s t , what p a r a l l e l c o m p u t a t i o n s can the

n e r v o u s system make b a s e d on o u r p r e s e n t knowledge of i t s

s t r u c t u r e , and second . how can these c o m p u t a t i o n s be

i n v e s t i g a t e d with e x i s t i n g e x p e r i m e n t a l t e c h n i q u e s ?

T h e complex i ty of t h e n e r v o u s system is u n p r e c e d e n t e d

and o u r u n d e r s t a n d i n g of l a r g e - s c a l e p a r a l l e l c o m p u t a t i o n is

limited. One vay t o d e v e l o p a n i n t u i t i o n f o r p a r a l l e l

p r o c e s s i n g i s t c s t u d y t h e p r o p e r t i e s of simple n o n l i n e a r

ne tworks - t h e c o n c e p t u a l and a a t h e e a a t i c a l i n s i g h t s g a i n e d

c o u l d sarve aE a basis f o r more r e a l i s t i c models . T h e

s i m p l i c i t y o f a model i s by no means a drawback i f its

b e h a v i o r r e s e m b l e s t h e r e g u l a r i t i e s e x h i b i t e d by a complex

sys t em. An e x a a p l e from s t a t i s t i c a l mechan ic s is t h e

two-d imens iona l I s i n g model f o r f errornagnet ism. A

two-dimensional l a t t i c e o f s p i n s w i t h n e a r e s t n e i g h b o r

i n t e r a c t i o n is a caricature of t h r e e - d i m e n s i o n a l a tomic

structure. However, t h e I s i n g model h a s t h e t v i n virtues of

a n e x a c t a n a l y t i c s o l u t i o n and a p h a s e t r a n s i t i o n semaekably

s i m i l a r t o the e x p e r i m e n t ally o b s e r v e d ferromagnetic

t r a n s i t i o n . B i o l o g i c a l s y s t e m s , a n d e s p e c i a l l y t h e n e r v o u s

s y s t e m , a r e c o n s i d e r a b l y more c o m p l e x t h a n t h i s e x a m p l e , b u t

t h e same s t r a t e g y c o u l d be u s e f u l , a n d certainly the

s i m p l e s t models s h o u l d b e t h o r o u g h l y s t u d i e d b e f o r e

a t t e m p t i n g t o s t u d y m o r e c o a p l i c a t e d ones .

G e n e r a l a g r e e m e n t has n o t y e t b e e n r e a c h e d a b o u t w h a t

among t h e r i c h v a r i e t y o f c h e m i c a l a n d e lec t r ica l detail in

t h e n e r v o u s s y s t e m is a o s t i ~ p o r t a n t f o r its f u n c t i o n . Each

a p p r o a c h , e x p ~ r i n e n t a l a s well as theoretical, makes d r a s t i c

a s s u m p t i o n s t o limit t h e s c o p e of e n q u i r y . A brief review

of n e r v e c e l l s a n d t h e i r e l e c t r i c a l p r o p e r t i e s i s g i v e n h e r e

t o p r o v i d e a b a c k g r o u n d f o r t h i s s t u d y .

Neurons and S y n a p s e s

I n 3906 t h e N o b e l P r i z e i n P h y s i o l o g y a n d n e d i c i n e was

s h a r e d by S a n t i a g o ~ a r n 6 n y C a j a l a n d C a m i l l o G o l g i f o r t h e i r

c o n t r i b u t i o n s t c n e u r o h i s t o l o g y . G o l g i h a d i n t r o d u c e d a

s i l v e r me thod f o r s t a i n i n g w h o l e nerve c e l l s i n 1873, b u t

h i s work u a s i g n o r e d f o r 1 4 years u n t i l C3 j a l , among o t h e r s ,

i m p r o v e d G o l q i l s m e t h o d and s y s t e m a t i c a l l y a p p l i e d i t t o

t i s s u e s f r o @ a w i d e r a n g e o f t h e animal kingdom. Cajalgs

F i g . 1. D r a u i n q s of mammalian n e u r o n s s t a i n e d by the

Golgi method: a) P u r k i n je cell from t h e h u m a n cerebel lua.

b) P y r a m i d a l c e l l f r o m t h e r a b b i t cerebral c o r t e x . c) fiotor

neu ron from the c a t s p i n a l cord ( f r o n Ca j a l , 1 9 1 1 ) .

careful d e s c r i p t i o n s and e x t e n s i v e i l l n s t r a t i o n s l a i d t h e

f o u n d a t i o n £01 modern neuroanatomy, and h i s b o l d

i n t e r p r e t a t i o n s of what h e s a 4 have p roved remarkably

a c c u r a t e .

I n t h e " b l a c k r e a c t i o n " , a s the Golgi method is

somet imes c a l l e d , s i l v e x i o n s taken up b y a neuron are

r educed t o a k l a c k precipitate which d r a m a t i c a l l y o u t l i n e s

t h e n e u r o n , However, t h e s t a i n is c a p r i c i o u s : by some

unknown mechanism o n l y a few n e u r o n s a r e s t a i n e d , but t h e s e

a r e s t a i n e d i n t h e i r e n t i r e t y , Although many s p e c i a l i z e d

stains have been i n t r o d u c e d , the Golg i method has n o t been

s u p e r s e d e d i n t h e c l a r i t y and c o m p f e t e n e s s with w h i c h t h e

mcrphology o f n e u r o n s c a n be v i s u a l i z e d , a s i l l u s t r a t e d in

Piq . 1 .

The a v a r d i n q o f t h e Nobel P r i z e i n 1906 h e a t e d a

v i q o r o u s d e b a t e which had begun 2 5 y e a r s e a r l i e r and would

n o t b e settled u n t i l 50 y e a r s l a t e r . I r o n i c a l l y , Go lg i and

Ca j a l were l e a d i n g spokesmen f o r t h e oppos ing g r o u p s , G o l g i

f a v o r e d the idea t h a t n e r v e c e l l s were j o i n e d i n a r e t i c u l a r

ne twork k y c o n t i n u a u s p r o t o p l a s m . T h e main e v i d e n c e f o r

t h i s v i e % u a s t h e e x i s t e n c e o f fibrils w i t h i n n e r v e cells

which a p p e a r e d to c r o s s between c e l l s , C a j a l s u p p o r t e d the

oppos ing v i e w that n e r v e ce l l s a r e c o a t i q u o u s but not

c c n t i n u o u s . Accord inq t o t h e "neu r on d o c t r i n e " , n e r v e cells

were functional u n i t s and their zones o f c o n t a c t were n o t

Cell body

Dendrite ,

Initial segment J Ah Myelin sheath +I'

iPSP lnpu t

v-

Ourput A electrical Secretion upl ling or

F i g . 2. a ) S c h e m a t i c i l l u s t r a t i o n of a representative

v e r t l a b r a t e nau ron . b ) F u n c t i o n a l i d e a l i z a t i o n of a neuron

a c c o r d i n g t o the c l a s s i c a l model. T h e arrows i n d i c a t e

t y p i c a l e l e c t r i c a l r e s p o n s e s : t o p - excitatory (EPSP) and

i n h i b i t o r y (I F s P ) p o s t s y n a p t i c p o t e n t i a l s ; midd le - format i o n o f t h e a c t i o n p o t e n t i a l a t t h e spike-initiating

zone: bottom - p r o p a g a t i o n of t h e a c t i o n p o t e n t i a l a l o n g

t h e axon (from Kandel, 1 9 7 6 ) .

c o n t i n u o u s l y j o i n e d , The l e n g t h a n d b i t t e r n e s s of the 75

y e a r d e b a t e a r e d i f f i c u l t t o imag ine now t h a t d i r e c t

e l e c t r o n - m i c r o g r a p h i c e v i d e n c e e x i s t s p r o v i n g t h a t , with

o n l y special e x c e p t i o n s , the t3neuron d o c t r i n e " i s c o r r e c t .

I n the l a s t 2 5 y e a r s t h e zones of c o n t a c t be tween neurons,

c a l l e d synapses, have b e e n the f o c u s o f i n t e n s i v e s t u d y .

Neurons a r e t h e basic u n i t s i n t h e brain and a re

q r e a t e r i n v a r i e t y t h a n any o t h e r cell t y p e in t h e body.

The d i v e r s i t y of s h a p e s and s i z e s i s matched b y t h e complex

b u t s p e c i f i c p a t t e r n o f t h e i r s y n a p t i c i n t e r c o n n e c t i o n s ,

Certain f e a t u r e s of t h e ne rvous sys tem, however , a p p e a r t o

b e u n i v e r s a l i n t h e a n i m a l kingdom, s u c h a s t h e fo rms of

e l e c t r i c a l s i q r a l i n g used f o r communicat ion w i t h i n and

between neurons, which w i l l b e b r i e f l y summar ized here.

More d e t a i l e d i n t r o d u c t o r y a c c o u n t s a r e g i v e n by Katz

( 1 9 6 6 ) , S t e v e n s (1966) , Aidley (197 I ) , K u f f l e r and N i c h o l l s

(1976), Randa l (19761, and Bul lock (1977) .

A l thaugh n c s i n g l e t y p e of n e u r o n is t y p i c a l o f a l l the

neurons i n t h e v e r t e b r a t e n e r v o u s s y s t e a , t h e i d e a l i z e d

neu ron i n Piq. 2 combines c h a r a c t e r i s t i c s common t o s e n s o r y

n e u r o n s , which t r a n s d u c e p h y s i c a l s t i m u l i i n t o e lec t r ica l

s i g n a l s , motor n e u r o n s , which i n n e r v a t e n u s c l e s , and many

t y p e s of i n t e r n e u r o n s , which m e d i a t e be tween neurons .

Accord ing t o t h e p r i n c i p l e of Ifdynamic p o l a r i z a t i o n N ,

i n t r o d u c e d b y Ca j a l ( 1 9 11) , i n f o r m a t i o n w i t h i n a a e u c o n

Guinea pig

a w w M

Rabbil

Cot

Monkey

Human infant

Human adult ,

occ.

Fiq. 3 , B E G recordinqs from s e v e r a l vertebrates, The

c a l i b r a t i o n bar in all recordings represents one second.

The human r e c o r d i n q s a r e f rom s c a l p electrodes; all o the r

recordinqs a r e froa e l ec trodes on the s u r f a c e of cerebral

c o r t e x . The bottom r e c o r d i n g shows the b l o c k a g e of t h e

a lpha r h y t h m when t h e s u b j e c t opened both eyes (from

Horrell, 1967).

f l o w s f r o m r e c e ~ t i v e z o n e s , u s u a l l y c a l l e d d e n d r i t e s , toward

t h e t r a n s r e r i t t i n q e l e u e n t , c a l l e d the axon. The eel1 bodies

o f n e u r o n s a r e t y p i c a l l y between 10p and 50p i n d i a m e t e r ,

b u t t h e branching d e n d r i t e s o f t en e x t e n d s e v e r a l millimeters

from t h e c e l l body and some a x o n s a r e o v e r a m e t e r l ong .

T h e e l e c t r i c a l s i g n a l s c o l l e c t e d b y d e n d r i t e s a r e s p a t i a l l y

and t e m p o r a l l y summed b y g raded e l e c t r i c a l p o t e n t i a l s w i t h i n

t h e neu ron . I n c o n t r a s t w i th e l e c t r i c a l l y p a s s i v e

d e n d r i t e s , the axon p r o d u c e s an a c t i v e , a l l - o r - n o n e ,

r e g e n e r a t i v e e v e n t , c a l l e d t h e a c t i o n p o t e n t i a 1, which

t y p i c a l l y p r o p a g a t e s a t speeds between 0. 1 and 700 a/sec

depend ing on t h e s i z e and p r o p e r t i e s o f t h e axon. Synonyms

f o r t h e a c t i o n p o t e n t i a l a r e i m p u l s e and spike.

The f irst e l e c t r i c a l r e c o r d i n g from a n exposed b r a i n

was made by Eiichard Ca ton i n 1875. Hans Berge r i n 7929

shoved t h a t s i m i l a r m i c r o v o l t p o t e n t i a l s c o u l d be r e c o r d e d

on t h e surface of t h e s k u l l . T h e marked rhy thms o b s e r v e d i n

t h e s e r e c o r d i n g s , a s shown in F i g . 3, va ry t h r o u g h o u t t h e

b r a i n , a r e c o r r e l a t e d w i t h b e h a v i o r a l s t a t e s , and a r e

r o u t i n e l y used t o d i a g n o s e b r a i n i n j u r y . Because the

s o u r c e s o f t h e r h y t h m s i n t h e e l e c t r o e n c e p h a l o g r a m (EZG) a r e

not known and because the EEG r e p r e s e n t s t h e pooled a c t i v i t y

o f many n e u r o n s , i t i s n o t a s u s e f u l a s e l e c t r i c a l

r e c o r d i n g s from i n d i v i d u a l neu rons .

The s u r f a c e of a neu ron , l i k e t h a t of a n y ce l l , is a

l i p i d b i l a y e r membrane which can s u p p o r t a p o t e n t i a l

difference, c a l l e d t h e membrane p o t e n t i a l . I n t h e absence

o f disturbance a r e s t i n g p o t e n t i a l o f a p p r o x i m a t e i y 60 my,

w i t h the i n s i d e n e q a t i v e , i s s a i n t a i n e d a c r o s s the aembrane

o f most neu rons . dembranes a r e s e l e c t i v e l y permeable t o

p a r t i c u l a r ions, s u c h a s sod ium, po t a s s ium, and c h l o r i n e ;

the r e s t i n g p o t e n t i a l is produced hy a m e t a b o l i c pump w h i c h

keeps the c o n c e n t r a t i o n of sod ium w i t h i n a n e u r o n low, and

t h e c o n c e n t r a t i o n o f p o t a s s i u m h i g h , r e l a t i v e t o t h e

e x t r a c e l l u l a r medium.

The membrane p o t e n t i a l of a s i n g l e n e u r o n can be

recorded ui t h a l a i c r o p i p e t t e . B finely-drawn glass

c a p i l l a r y w i t h a t i p of a b o u t 0. l p is f i l l e d w i t h

e l e c t r o l y t e and usad t o i m p a l e a n e u r o n . The membrane

u s u a l l y r e s e a l s a r o u n d the m i c r o p i p e t t e and t he

i n t r a c e l l u l a r membrane p o t e n t i a l c a n be r e l i a b l y a o o i t o r e d

f c r many h o u r s . Examples o f i n t r a c e l l u l a r l y r e c o r d e d

membrane p o t e n t i a l s a r e shown i n P igs . 5 and 8. The weak

e x t r a c e l l u l a r p o t e n t i a l s t h a t a r e f o u n d i n t h e vicinity o f a

neu ron can be r e c o r d e d w i t h a n e x t r a c e l l u l a r metal

m i c r o e l e c t r o d e , A f i n e t u n g s t e n wire u i t h i t s t i p e t c h e d t o

a b o u t 1 ,u was used t o r e c o r d the i m p u l s e s s h o r n i n P i g . 9.

Host o f c u r d e t a i l e d knowledge a b o u t t h e e l e c t r i c a l

p r o p e r t i e s of nembranes is b a s e d on the l a r g e n e r v e f i b e r s

( u p t o 1 m m i n d i a m e t e r ) t h a t t h e s q u i d u s e s t o i n i t i a t e

r a p i d e s c a p e moversent, ?fie pioneering work o n t h e giant

axon of t h e s q u i d i n t h e 1950s, and i n p a r t i c u l a r t h e

i n v e s t i q a t i o n s of A l a n Hodgkin and Andreu Huxley on t h e

a c t i o n p o t e n t i a l , h a s s i n c e p r o v e n r e l e v a n t t o many

different t i s s u e s , such as v e r t e b r a t e n e r v e fibers and heart

muscle. An action p o t e n t i a l is a s u d d e n spike i n the

membrane p o t e n t i a l w h i c h o c c u r s i n some membranes when a

t h r e s h o l d p o t e n t i a l i s r e a c h e d a n d which lasts f o r a few

~ i l l i s e c o n d s , a s shown i n F i g . 4a. T h e a c t i o n potential is

t r i q g e r e d by v o l t a g e - s e n s i t i v e c h a n n e l s i n t h e membrane: t h e

p e r m e a b i l i t y f o r s o d i u a i o n s is t r a n s i e n t l y increased d u r i n g

t h e r i s i n q p h a s e ( a p o s i t i v e s h i f t o f t h e raembrane potential

c a l l e d a d e p o l a r i z a t i o n ) , and t h e p e r m e a b i l i t y for p o t a s s i u m

i o n s is increased d u r i n g t h e f a l l i n g p h a s e (a n e g a t i v e s h i f t

o f t h e membrane potential c a l l e d a h y p e r p o l a r i z a t i a n ) , a s

i n d i c a t e d i n F i q . 4b.

A t mos t s y n a p s e s a c h e m i c a l n e u r o t r a n s m i t t e r i s u s e d t o 0

c o m m u n i c a t e across the 200 A s e p a r a t i o n , c a l l e d the synaptic

c l e f t , b a t n e s n t h e p r e s y n a p t i c membrane and t h e p o s t s y n a p t i c

eembrane, a s shown i n F i g . 5a. Phen an a c t i o n p o t e n t i a l o r

d e p o l a r i z a t i o n i n v a d e s t h e p r e s y n a p t i c t e r m i n a l , c a l c i u m

e n t e r s t h e t e r m i n a l , a n d s h o r t 1 y t h e r e a f t e r n e u r o t r a n s m i t t e r

i s r e l e a s e d i n t o t h e s y n a p t i c c l e f t . T h e n e u r o t r a n s m i t t e r

d i f f u s e s t o t h e p o s t s y n a p t i c membrane, where i t b i n d s w i t h a

specialized protein r e c e p t o r a n d c a u s e s a t e m p o r a r y

I I I I I

0 2 4 mscc

P i q . 4 . T h e o r e t i c a l c h a n g e s of t h e a ) mambrane

p o t e n t i a l and b ) ineabrane c o n d u c t a n c e s d u r i n g an a c t i o n

p o t e n t i a 1. ~ 3 .

and E K a r e respectively t h e sodium and

p o t a s s i u m e q u i l i b r i u m p o t e n t i a l s , and g a n d g y are t h e N a

sod iu rn and ~ o t a s s i u m c o n d u c t a n c e s . T h e e q u i l i b r i u m

p o t e n t i a l f o r a n i o n i c species is t h e p o t e n t i a l across a

s e m i p e r m e a b l e membrane c a u s e d b g on ionic c o n c e n t r a t i o n

difference {f r o r r Aidley, 1971) .

p e r m e a b i l i t y i n c r e a s e f o r p a r t i c u l a r i o n s . T h e iaagni tude

a n d sign o f t h e resulting p o s t s y n a p t i c potential (PSP)

depends on t h e p o s t s y n a p t i c membrane p o t e n t i a l before t h e

s y n a p s e i s a c t i v a t e d ; i n f a c t , a t t h e r e v e r s a l p o t e n t i a l t h e

PSP is zero. The s y n a p s e is c a l l e d e x c i t a t o r y i f t h e

r e v e r s a l p o t e n t i a l is above t h r e s h o l d and i n h i b i t o r y i f t h e

r s v e r s a l p o t e n t i a l is below t h r e s h o l d . I n aost norraal

c i r c u m s t a n c e s t h e a c t i v a t i o n o f an e x c i t a t o r y s y n a p s e moves

t h e membrane p o t e n t i a l c l o s e r t o t h r e s h o l d and t h e

a c t i v a t i o n o f an i n h i b i t o r y s y n a p s e d r i v e s t h e nnembrane

p o t e n t i a l away from t h r e s h o l d . An example o f e x c i t a t o r y and

i n h i b i t o r y p o s t s y n a p t i c p o t e n t i a l s (EPSP8s a n d IPSPf s) i n an

i n t r a c e l l u l a r r eco rd ing is shown in F i g . 5b. Some electric

s y n a p s e s h a v e been found w h i c h a r e d i r e c t , l o w - r e s i s t a n c e

c o n n e c t i o n s between neurons and f o r which there i s no

a p p r e c i a b l e t i m e d e l a y between t h e p r e s f n a p t i c a c t i v a t i o n

and the p o s t s y n a p t i c response,

Slow P o t e n t i a l s and S t o c h a s t i c V a r i a b i l i t y

T h e massive b u n d l e s of n e r v e f i b e r s i n t h e v e r t e b r a t e

b r a i n a n d the striking a l l - o r - n o n e c h a r a c t e r o f the a c t i o n

p o t e n t i a l l e d e a r l y i n v e s t i g a t o r s t o compare the n e r v o u s

system w i t h d i g i t a l c o m p u t e r s and l o g i c a l au toma ta

(McCulloch a n d F i t t s , 1943; Shannon a n d McCarty, 1 9 5 6 ) .

P i q . 5, a ) Schematic d r a w i n g s of typical chenical

synapses. The p r e s y n a p t i c t e r m i n a l is filled w i t h vesicles

c o n t a i n i n g a n e u r o t r a n s r r i t t e r . Synapses o f t e n o c c u r n o t on

the d e n i r i t e i t se l f b u t o n thorn-like p r o t u b e r a n c e s called

spines, a s shown o n t h a right. b) I n t r a c e l l u l a r recording

fron an ApZysia n e u r o n which d i s p l a y s b o t h e x c i t a t o r y and

inhibitory p o s t s y n a p t i c p o t e n t i a l s (from A i d l e y , 1 9 7 1 ) .

Bouever, a s t h e t e c h n i q u e s f o r r e c o r d i n g i n t r a c e l l u l a r

p o t e n t i a l s ware d e v e l o p e d and a p p l i e d t o n e r v e c e l l s i n a

variety of n e r v o u s s y s t e m s , the i ~ p o r t a n c e o f s u b t h r e s h o l d

g r a d e d p o t e n t i a l s f o r t h e i n t e g r a t i o n of i n f o r m a t i o n w i t h i n

t h e n e u r o n was soon a p p r e c i a t e d ( G r u n d f e s t , 1957; B u l l o c k ,

Graded membrane p o t e n t i a l s , such a s t h o s e produced by

sensory r e c a p t o r s and s y n a p s e s , a r e c a l l e d s low p o t e n t i a l s

t o d i s t i n g u i s h t h e m f rom a c t i o n p o t e n t i a l s . One o f t h e

e a r l i e s t i n v e s t i g a t o r s t o emphasize t h e r o l e of s low

p c t e n t i a l s f o r s i g n a l p r o c e s s i n g was B i shop (1 9561, who

summarizsd t h e e v i d e n c e f o r g r a d e d r e s p o n s e s and conc luded

t h a t t h e a c t i o n p o t e n t i a l s i n a s e n s o r y n e r v e transduce t h e

g r a d e d s e n s o r y s t i m u l a t i o n i n t o a c o r r e s p o n d i n g s low

p o t e n t i a l a t t h e n e x t neuron. B s i m i l a r ~ i e v was e x p r e s s e d

b y F u o r t e s (1959) i n d i s c u s s i n g r e f l e x r e s p o n s e s :

"It t u r n s o u t therefore t h a t , e v e n i n the s i m p l a s t c o n d i t i o n s ap p l y i n g t o e x c i t a t o r y monosynap t i c r e f l e x e s , a whole s u c c e s s i o n of f r e q u e n c y - i n t e n s i t y t r a n s f o r a a t i o n s i s i n v o l v e d i n t h e p r o d u c t i o n o f no rma l a c t i o n s : i n t e n s i t y t o f r e q u e n c y i n r e c e p t o r s , f r e q u e n c y t o i n t e n s i t y a t synaptic r e q i o n s , i n t e n s i t y t o f r e q u e n c y a t t h e p a c e ~ a k e r r e g i o n s r s p i k e - i n i t i a t i n q z o n e s ] o f t ao torneurones , and f i n a l l y , frequency t o i n t e n s i t y i n r a u s ~ l e s . ~

T h e e v i d e n c e f o r p r o c e s s i n g bp g r a d e d p o t e n t i a l s i s now

overuhe l ra inq a n d t h e view e x p r e s s e d b y B i s h o p and P u o r t e s ,

somet imes c a l l e d t h e vlslow p o t e n t i a l t h e o r y " , is g e n e r a l l y

Pig. 6 S c h e m a t i c i l l u s t r a t i o n o f t h e 18slow p o t e n t i a l

t h e o r y H of i n f o r m a t i o n p r o c e s s i n g i n neurons . T h e g r a d e d

p o t e n t i a l s i n t h e two n e u r o n s o n t h e l e f t a r e converted into

f irinq r a t a s w h i c h a re decoded by the p o s t s y n a p t i c n e u r o n on

t h e r i q h t . T h e membrane p o t e n t i a l of t h e postsynaptic

neuron t e n p o r a l l y and s p a t i a l l y sums t h e e x c i t a t o r y (bo t tom)

and i n h i b i t o r y [ t o p ) i n f l u e n c e s . Accord ing t o the *slow

p o t e n t i a l t h e o r y u , t h e p r imar y c o m p u t a t i o n s i n t h e nervous

system a r e per formed b y t h e g r a d e d memrane p o t e n t i a l s

w i t h i n n e u r o n s , and a c t i o n p o t e n t i a l s a r e on ly a s e c o n d a r y

i n f o r m a t i o n c a r r i e r (from Stevens, 1966 ) .

Piq, 7 . S c h e m a t i c s u m m a r y d iagram of t h e f i v e

p r i n c i p a l c e l l t y p e s and t y p i c a l s y n a p t i c c o n t a c t s found i n

v e r t e b r a t e r e t i n a s , L i q h t i s t r a n s d u c e d i n t o electrical

siqnals by t h e p h o t o r e c e p t o r cells whose receptor t e r m i n a l s

[ R T ) a r e s h o w n a t t h e t o p of the diagram. Electrical

s i g n a l s from t h e r e c e p t o r s a re conveyed by t h e b i p o l e r cells

( B ) t o t h e q a n q l i o n cells ( G ) whose l o n g a x o n s raake t h e

optic nerve . L a t e r a l interaction w i t h i n the retina is

p r o v i d e d by t h e h o r i z o n t a l cells ( H ) near t h e r ecep to r l a y e r

a n d t h e a ~ 3 c r i n e cells ( A ) n e a r t h e g a n g l i o n cell l a y e r .

T h e d i r e c t i o n o f i n f o r m a t i o n f l o w can b e i n f e r r e d from t h e

r o u n d v a s i c l e s found o n l y i n p r e s y n a p t i c t e r m i n a l s . Some

terminals, c a l l e d r e c i p r o c a l s y n a p s e s , a r e b o t h p r e s y n a p t i c

a n d p o s t s y n a p t i c (from Dowlinq, 1968) .

P i q . 8. I n t r a c e l l u l a r r e c o r d i n g s from t h e f i v e

p r i n c i p a l c e l l types i n t h e mudpuppy r e t i n a . The responses

were e l i c i t e d uith a f l a s h of l i g h t focussed on t h e l e f t

p h o t o r e c e p t o r . T h e r e c o r d i n g s a r e placed i n a sche a a t i c

w i r i n g diagram to suggest h o w the r e s p o n s e s could be

produced by s y n a p t i c i n t e r a c t i o n s . The r e c e p t o r s ( R f , t h e

h o r i z o n t a l cells ( 8 ) . and t h e b i p o l a r c e l l s ( B ) r espond u i t h

s l o v , q r a d e d p o t e n t i a l s , T h e a m a c r i n e c e l l s ( A ) respond

more t r a n s i e n t l y a n d some ganglion cells ( G L ) r e spond w i t h a

burst o f a c t i o n p o t e n t i a l s a t the onset or offset of t h e

l i q h t , o r b o t h . O t h e r g a n g l i o n c e l l s (Gi) respond with

sustained firing o r s u s t a i n e d i n h i b i t i o n (from Dowling,

1970) .

a c c e p t e d b y lotost i n v e s t i q a t o r s . S t e v e n s (1964) h a s g i v e n a

p a r t i c u l a s l y c l e a r s t a t e m e n t o f . the "s low p o t e n t i a l theory",

and F i g . 6 is an i n s t r u c t i v e i l l u s t r a t i o n fro^ h i s book

(1966) . S i g n a l p r o c e s s i n g b y g raded p o t e n t i a l s is

p a r t i c u l a r l y i m ~ o r t a n t i n the v e r t e b r a t e r e t i n a where most

n e u r o n s do n o t p roduce a c t i o n p o t e n t i a l s {Tomita, 1965;

Werbl in and Dowlinq, 1969; Kaneko, 1970) . A sche r sa t i c

d i a g r a a cf the f i v e p r i n c i p a l cell t y p e s i n the v e r t e b r a t e

r e t i n a is shovn i n F i g . 7, and i n t r a c e l l u l a r r e c o r d i n g s from

the iaudpuppy retina a r e shovn i n P i g . 8. Graded i n t e r a c t i o n

between d e n d r i t e s h a s a l s o been found i n t h e o l f a c t o r y bulb

and e l s e w h e r e i n the c e n t r a l n e r v o u s s y s t e m (Rak ic , 1975;

P e a r s o n , 1976; S c h m i t t , Dev & Smith, 1976; Shepherd , 1978) .

A t y p i c a l p y r a m i d a l c e l l in c e r e b r a l c o r t e x eay have

1,000 t o 10,000 s y n a p s e s a n d r e c e i v e 10,000 t o 100,000

p o s t s y n a p t i c p o t e n t i a l s p e r second , most of uhich a r e

s p a t i a l l y and t e m p o r a l l y summed i n the d e n d r i t i c tree,

Normal e l e c t r i c a l c o n d u c t i o n i n a d e n d r i t e is governed by a n

e q u a t i o n i d e n t i c a l t o t h a t s t u d i e d by Lord Kelv in for

e l e c t r i c a l t r a n s r c i s s i o n i n s u b m a r i n e c a b l e s . I n the s t e a d y

s t a t e t h e p o t e n t i a l f rom a b a t t e r y a t one end of a un i fo rm

c a b l e d e c r e m e n t s w i t h d i s t a n c e X a l o n g t h e c a b l e a c c o r d i n g

t o

where is t h e space c o n s t a n t which d e p e n d s o n the

Trials

I

2

2

- 50 msec

POST-STIMULUS TIME HISTOGRAM

-50 0 5'0 150 ' 250 Time in Milliseconds

Pig. 9. E x t r a c e l l u l a r r e c o r d i n g s from a single neuron

i n t h e c a t visual c o r t e x , This neuron r e s p o n d e d best t o a

s l i t of liqht o b l i q u e l y oriented i n a p a r t i c u l a r p a r t o f the

v i s u a l f i e l d . The f i r s t 12 successive r e s p o n s e s of the

neuron t o 50 m s e x p o s u r e s of the light are shown above, and

t h e average resFonse f o r 20 t r i a l s is shown below. Although

t h e pattern of f i r i n q v a r i e d from t r i a l t o trial (and some

parts of the r e s p o n s e d r o p out e n t i r e l y , s u c h a s i n t r i a l s

5 , 10, and 1 1 ) , the average over the ensemble of trials,

c a l l e d the p o s t s t i i s u l u s t h e h i s t o g r a ~ (PST), is a

r a p e a t a b l e m e a s u r e m e n t (from norrell, 1 9 7 2 ) .

electr ical p r o p e r t i e s o f t h e a e a b r a n e . F o r most n e r v e

f i b e r s t h e s p a c e c o n s t a n t is t y p i c a l l y b e t v e e n 1 0 0 P and

E l e c t r i c a l r e c o r d i n g s from s i n g l e n e u r o n s ;in t h e

v e r t e b r a t e n e r v o u s system h a v e a s u r p r i s i n g d e g r e e of

v a r i a b i l i t y . Per e x a m p l e , e x t r a c e l l u l a r r e c o r d i n g s o f a

n e u r o n i n t h e c a t v i s u a l c o r t e x a r e shown i n F i g . 9. A

n e u r o n i n v i s u a l c o r t e x w i l l r e s p o n d t o l i g h t o n l y i n a

r e s t r i c t e d r e g i o n o f t h e v i s u a l f i e l d , c a l l e d i ts receptive

f i e l d , a n d @any n e u r o n s r e s p o n d b e s t t o s l i ts o f l i g h t vith

a p r e f e r r e d o r i e n t a t i o n and d i r e c t i o n o f novement ( H u b e l a n d

Wiesel, 7 9 6 2 ) . T h e s u c c e s s i v e t r i a l s shown i n Fig. 9

i l l u s t r a t e t h e v a r i a b i l i t y of t h e r e s p o n s e u n d e r i d a n t i c a l

conditions, A l t h o u g h the d e t a i l s o f t h e r e s p o n s e are

d i f f e r e n t f r o m t r i a l t o t r i a l - the a c t i o n p o t e n t i a l s

a p p e a r t o o c c u r a t i r r e g u l a r i n t e r v a l s - the a v e r a g e

f i r i n q r a t e f ro r r p o o l e d t r i a l s , s h o v n a t t h e b o t t o m of

F i q . 9, i s a more c o n s i s t e n t measurement . S t o c h a s t i c

v a r i a b i l i t y i s f o u n d a t e v e r y l e v e l o f t h e n e r v o u s system,

f r o m sensory r e c e p t o r s t o c e r e b r a l c o r t e x .

One of t h e c h i e f s o u r c e s o f noise i n t h e n e r v o u s s y s t e m

i s variation i n t h e amount o f n e u r o t r a n s m i t t e r r e l e a s e d a t

c h e e i c a l synapses. N e u r o t r a n s a i t t e r is s t o r e d i n s m a l l

p a c k e t s c a l l e d v e s i c l e s , s h o v n i n F i g . Sa , a n d is r e l e a s e d

from t h e p r e s y a a p t i c t e r m i n a l i n d i s c r e t e u n i t s c a l l e d

q u a n t a . A t t h e f r o g n e u r o m a s c u l a r j u n c t i o n , f o r example,

e a c h s y n a p t i c a c t i v a t i o n r e l e a s e s a p p r o r i m a t e l y 300 quanta

and each quantum c o n t a i n s a p p r o x i m a t e l y 18,000 m o l e c u l e s of

a c e t y l c h o l i n e , t h e n e u r o t r a n s m i t t e r . Even i n t h e a b s e n c e o f

a c t i v a t i o n , q u a n t a a r e r e l e a s e d s p o n t a n e o u s l y , p r o d u c i n g

m i n i a t u r e s y n a p t i c p o t e n t i a l s of a p p r o x i m a t e l y 0.5 mv

a m p l i t u d e , a s shown i n F ig . 1Oa. A s t a t i s t i c a l v a r i a t i o n o f

a b o u t 17 q u a n t a can b e e x p e c t e d during n o r e a l a c t i v a t i o n of

t h e s y n a p s e , c r 6% o f t h e t o t a l . Hany s y n a p s e s r e l e a s e

f a w e r q u a n t a and c o n s e q u e n t l y h a v e even g r e a t e r v a r i a t i o n .

Another s o u r c e of n o i s e is f l u c t u a t i o n o f t h e f i r i n g

t h r e s h o l d . C a r e f u l a e a s u r e a e n t s on f r o g n e r v e s show t h a t

t h e t h r e s h o l d v a r i e s over a p p r o x i m a t e l y 1 av (Verveen and

Derksen, 1965) . Quan t a l f l u c t u a t i o n s and t h r e s h o l d f l u c t u a t i o n s o c c u r

on a r a i l l i v o l t s c a l e , f l i c r o o o l t v a r i a t i o n s a r e a l s o

o b s e r v e d owing to s h o t n o i s e from t h e o p e n i n g and c l o s i n g of

i o n i c c h a n n e l s i n t h e membrane. Fo r example, f l u c t u a t i o n s

i n l n t r a c e l l u l a r r e c o r d i n g s , a s shown in Fig . l o b , have been

used t o e s t i m a t e the k i n e t i c s of a s i n g l e a c e t y l c h o l i n e

r e c e p t o r during a c t i v a t i o n o f t h e f r o g n e u r o m u s c u l a r

j u n c t i o n : when a c e t y l c h o l i n e b i n d s t o t h e r e c e p t o r , a

c h a n n e l is opened f o r a b o u t 1 as, a b o u t 5 x 1 0 ~ u n i v a l e n t i o n s

f l o w t h r o u g h t h e meabrane ( m a i n l y po ta s s ium a n d sodium

i o n s ) , a n d a p o s t s y n a p t i c p o t e n t i a l o f a b o u t 0.3 p v is

F i q . 1 3 . Beco rd ings made f rom t h e m o t o r n e r v e synapse

of the f r o q . a ) The lower t r a c e on the l e f t p a n e l shows t h e

p o s t s ~ n a o t i c membrane p o t e n t i a l i n r e s p o n s e t o n e r v e

s t i m u l a t i o n (sca3.e: 50 mv and 2 ms). T h e u p p e r traces show

s p o n t 3 n e ~ u s a c t i v i t y i n t h e a b s e n c e of s t i m u l a t i o n (scale:

3 . 6 m v and 4 7 m s ) , T h e m i n i a t u r e s y n a p t i c p o t e n t i a l s are a

r e s u l t of t h e random r e l e a s e of a c e t y l c h o l i n e - f illed

v e s i c l z s f ror r t h e p r e s y n a p t i c t e r m i n a l . The r i g h t panel

p r e s e n t s t h e same measurelaants n a d a 2 an from t h e synapse ,

L) Lou-qain D C r e c o r d i n g s (nar row t r a c e s ) a n d high-gain BC

r e c o r d i n q s ( n o i s y t r a c e s ) o f t h e p r e s y n a p t i c mefabrane

p o t e n t i a l ( t o p ) . When a c e t y l c h o l i n e is a p p l i e d t o t h e

j u n c t i o n ( b o t t o m ) , t h e membrane p o t e n t i a l i n c r e a s e s and t h e

membrane n o i s e from t h e o p e n i n g a n d c l o s i n g of ionic

c h a n n e l s i n c r e a s e s . Spon taneous m i n i a t u r e synaptic

p o t e n t i a l s a r e a l s o shovn. c) R e c o r d i n g o f t h e c u r r e n t

t h r o u q h 3 s m a l l pa t ch of p o s t s y n a p t i c menbrane b a t h e d w i t h

s u b e r y l d i c h o l i n e , w h i c h is known t o open t h e a c e t y l c h o l i n e

r e c e p t o r f o r a l o n g e r p e r i o d t h a n a c e t y l c h o l i n e . T h e

r a p i d i t y w i t h which the c h a n n e l opens and c l o s e s i n d i c a t e s

t h a t t h e r a c e p t o r h a s on ly t u o s t a t e s ( f r o n D e F e l i c e , 1 9 7 7 ) .

Fig. 10

produced . S i n i l a r neasureatents have been made o n the soditra

and po ta s s iun ; channels whish are ac t i va ted dtasing a n action

p o t e n t i a l (Neher a n d Stevens, 1 9 7 7 ; DePelice, 1977) .

T h e LimuZus B e t i n a

One o f the most c a r e f u l l y s t u d i e d and b e s t u n d e r s t o o d

p i eces of nervous t i s s u e is the l a t e r a l eye of Limulus

polyphems. S o ~ e t i m e s called the h o r s e s h o e c r a b b u t i n f a c t

a n a r a c h n i d , LimuZus is t o d a y n o t mnch d i f f e r e n t from its

a n c e s t o r s 500 m i l l i o n y e a r s ago. T h e coapound l a t e r a l eye

of LimuZuuc contains about 600 ommatidia, each o f which i s a

f u n c t i o n a l p h c t o r e c e p t o r o n i t and r e s e m b l e s a segmented

orange, a s shcwn i n Fig. 11. There a r e a b o u t a d o z e n

seqmen t s and a l l b u t o n e a r e r e t i n u l a r cells c o n t a i n i n g a

r h o d o p s i n - l i k e v i s u a l pigment. The one odd segment is an

e c c e n t r i c c e l l whose axon g e n e r a t e s i m p u l s e s and l e a v e s the

eye t h r o u g h t h e o p t i c n e r v e . Lying j u s t b e n e a t h the

ommat id ia is a layer of f i r i e l g b r a n c h i n g a x o n s and s y n a p t i c

c o n t a c t s which p r o v i d e s l a t e r a l i n t e r a c t i o n between t h e

eccentric ce l l s ,

T h e a c t i o n p o t e n t i a l s from a s i n g l e ornmatidium c a n be

r e c o r d e d by c a r e f u l l y d i s s e c t i n g t h e o p t i c ne rve and t e a s i n g

o u t a s i n g l e fiber. The r e s p o n s e of an ornmatidium t o a

s u d d e n i n c r e a s e i n t h e i l l u m i n a t i o n is a t r a n s i e n t burst of

Fiq. 12. E l a c t r i c a l r e c o r d i n g s f r o n a s i n g l e o p t i c

n e r v e f i k e r o f t h e Limlus l a t e r a l eye. T h e r e t i n a was

i l l u i n i n a t e d by l i g h t w i t h t h e r e l a t i v e i n t e n s i t y given on

the l e f t . T h e time marks a t t h e b o t t o a of each trace

r e p r e s e n t 200 m s . T h e w h i t e band a b o v e t h e time marks is

b l a c k e n e d d u r i n g t h e p e r i o d o f i l l u . n i n a t i o n . S e v e r a l

f e a t u r e s of t h e p h o t o r e c e p t o r response a r e i l l u s t r a t e d :

f i r s t , thi? r a t s of f i r i n g i n s t e a d y state d e p e n d s

m o n o t o n i c a l l y o r t h e intensity o v e r a wide r a n g e ; s econd , a

b u r s t of f i r i n q o c c u r s a t t h e o n s e t of i l l u m i n a t i o n ; and

t h i r d , t h e d e l a y of t h e r e s p o n s e d e c r e a s e s w i t h i n c r e a s i n g

i l l u m i n a t i o n . Like o u r own e y e s , t h e LimuZus r e t i n a h a s a

wide dynamic ranqe w h i l e r e m a i n i n g s e n s i t i v e t o s m a l l

chanqes i n t h e i n t s 1 i s i t y ( f rom H a r t f i n e , 1941) .

lrght light

i I

fiber B frequency (~mpulses per s)

Fiq. 13. D e m o n s t r a t i o n o f lateral i n h i b i t i o n i n t h e

Limulus retina, The schematic drawing of the Limulus retina

on t h e l e f t s h o w s two d i s s e c t e d nerve f i b e r s . T h e

illumination on each ommatidium was independently

controlled. The g r a p h s o n t h e r i g h t show the decrease i n

t h e firing rate o f o n e f i b e r a s a f u n c t i o n o f t h e f i r i n g

r a t e i n t h e o t h e r f i b e r . I n b o t h cases t h e data a b o v e t h e

t h r e s h o l d f o r lateral i n h i b i t i o n are f i t by a s i n g l e

straight l i n e , The autual i n h i b i t i o n arises i n t h e l a t e r a l

l a y e r b e n e a t h t h e p h o t o r e c e p t o r omma t i d i a (from Rat lift,

1965).

i m p u l s e s which s u b s i d e s t o a r e g u l a r discharge , as shown in

Pig, 12, I n 1932 B a l d a a Keffes EIartfine found t h a t t h e

s t e a d y - s t a t e firing r a t e of a d i s s e c t e d ommatidiurn v a r i e s in

p r o p o r t i o n t o the l o g a r i t h m of the i n c i d e n t l i g h t i n t e n s i t y

o v e r a t l e a s t f c u r d e c a d e s . S i n c e t h e n it has been possible

t o study e v e r y s t e p i n t h e process of s e n s o r y t r a n s d u c t i o n

and neural i n t e g r a t i o n .

I n 1956 H a r t l i n e and h i s c o l l e a g u e s d i s c o v e r e d t h a t t h e

f i r i n g r a t e of an omuiat ia iua i s d i a i n i s h e d by t h e

i l l u m i n a t i o n o f n e i g h b o r i n g o a a a t i d i a , a s shown i n P i g . 13.

T h e i n h i b i t i o c be tween n e i g h b o r i n g ommat id ia d e p e n d s

l i n e a r l y on t h e i r f i r i n g r a t e s ( H a r t l i n e and R a t l i f f , 1957) .

For s t e a d y - s t a t e i l l u m i n a t i o n above t h e t h r e s h o l d for

l a t e r a l i n h i k i t i o n , t h e f i r i n g r a t e of t h e a - t h omaat id ium

r, is d s t e r m i n e d b y

b I where e,' is t h e i n p u t , K,b are t h e i n h i b i t o r y

0 c o e f f i c i e n t s , a n d r b is t h e t h r e s h o l d f o r i n h i b i t i o n . T h e

v a l i d i t y of this l i n e a r model has b e e n e s t a b l i s h e d by

e x t e n s i v a t e s t s t h a t t y p i c a l l y i n v o l v e d t h e measurement of

two o r Bore o tamat id ia , a s shovn i n F i g . 13. The s t a t i s t i c a l

d i s t r i b u t i o n of t h e i n h i b i t o r y c o e f f i c i e n t s has t h e s h a p e of

a v o l c a n i c c r a t e r a r o u n d e a c h ommatidium (Barlow, 1969) . T h e r e s p o n s e of t h e LimZus retina i l l u m i n a t e d w i t h a

s p a t i a l s t e p f u n c t i o n i s shown i n F i g . 14 . Later a1

P i g . 1 4 . T h e f i r i n g r a t e of a s i n g l e o a m a t i d i u n i n

r e s p o n s e t o a step f u n c t i o n i n i l l u s i n a t i o n . The density of

the p h o t o g r a p h i c p l a t e u s e d t o p r o j e c t t h e stimulus is shova

i n t h e uppar right corner. When a l l t h e receptors except

o n e were c o v e r e d b y a aask, t h e r e c e p t o r r e s p o n d e d directly

t o t h e l i g h t i n t e n s i t y , a s shown i n t h e t o p graph. T h e

l ower qraph shows t h e firing r a t e o f t h e same r e c e p t o r when

the mask w a s removed. The response a t t h e b o r d e r is

e n h a n c e d b y l a t e r a l i n h i b i t i o n : a b r i g h t l y - i l l u m i n a t e d

r e c e p t o r i n h i b i t s a l e s s - b r i g h t l y - i l l u m i n a t e d n e i g h b o r i n g

r e c e p t o r more s t r o n g l y t h a n it is i n h i b i t e d i n r e t u r n ( frm

R a t l i f f and H a r t l i n e , 1 9 5 9 ) .

i n h i b i t i o n enhances t h e r e s p o n s e c o n t r a s t a t b o r d e r s between

r e g i o n s with d i f f e r i n g l i g h t i n t e n s i t y . Bumans perce ive a

s i n i l a r c o n t r a s t e n h a n c e a e n t , known a s nach bands, v h i c h aay

arise f r o m l a t e r a l i n h i b i t o r y i n t e r a c t i o n i n t h e human

r e t i n a ( R a t l i f f , 1 9 6 5 ) .

T h e s t e a d y - s t a t e f i r i n g r a t e o f an e c c e n t r i c c e l l is

p r o p o r t i o n a l t o i t s d e p o l a r i z a t i o n whether t h e

d e p o l a r i z a t i o n is p r o d u c e d by p h o t o t r a n s d u c t i o n o r by

c u r r e n t i n j e c t e d i n t r a c e l l u l a r l y ( P u o r t e s , 1959b) . T h u s ,

t h e l o g a r i t h m i c r e l a t i o n s h i p b e t w e e n light i n t e n s i t y and

f i r i n g r a t e o c c u r s i n t h e f i rs t step that p r o d u c e s t h e

g e n e r a t o r p o t e n t i a l . The t r a n s i e n t b u r s t o f i a p u l s e s i n

response t o s u d d e n i n c r e a s e i n i l l u m i n a t i o n ( F i g . 12) is t h e

r e s u l t o f s e l f - i n h i b i t o r y f e e d b a c k ( S t e v e n s , 1964; P u r p l e ,

1964 ; Purple a n d D o d g e , 3965) . T h e i n h i b i t o r y p o s t s y n a p t i c

~ o t s n t i a l h a s a d e c a y time c o n s t a n t o f a b o u t 0.5 sec.

T h e l i n e a r i t y and t r a n s l a t i o n i n v a r i a n c e o f t h e LimuZus

r e t i n a h a v e b e e n e x p l o i t e d i n a series of e x p e r i m e n t s which

q e n e r a l i z e t h e H a r t l i n e - R a t l i f f model f o r a r b i t r a r y t e m p o r a l

and s p a t i a l patterns o f i l l u m i n a t i o n . Bruce K n i g h t ,

J u n - i c h i Toyoda, and F r e d e r i c k Dodge ( 1970) h a v e u s e d l i n e a r

s y s t e m s t h e o r y t o e x a m i n e each e l e m e n t o f s i g n a l p r o c e s s i n g

i n t h e LimuZus r e t i n a : (i) t h e t r a n s d u c t i o n o f l i g h t t o a

q e n e r a t o r p o t e r t i a l i n t h e e c c e n t r i c c e l l ; (ii) t h e

t r a n s d u c t i o n of the g r a d e d potential i n the e c c e n t r i c c e l l

into a c t i o n p o t e n t i a l s ; and (iii) t h e inhibitory influeace

o f neighboring o m a t i d i a .

S i n c e each t r a n s f o r m a t i o n o f t h e s i g n a l p a s t the

q e n e r a t o r p o t e n t i a l is a p p r o x i m a t e l y l i n e a r , t h e response of

the LimuZus r e t i n a s h o u l d b e p r e d i c t a b l e f rom a transfer

f u n c t i o n . K n i g h t , Toyoda, and Dodge e x p e r i m e n t a l l y u e a s u r e d

t r a n s f e r f u n c t i o n s f o r a l l three elernats of s i g n a l

p r o c e s s i n g i n t h e L h Z u s r e t i n a a n d demonstrated

s e l f - c o n s i s t e n c y between t h e atoael a n d the raeasurenaents.

T h e y illuminated a s i n g l e omraatidium i n t h e LimuZus retina

w i t h l i q h t c o n t a i n i n g a s i n u s o i d a l l y-modulated component and

measured t h e r e s p o n s e o f the g e n e r a t o r p o t e n t i a l and t h e

f i r i n q r a t e o f t h e e c c e n t r i c c e l l ( t o p and bo t tom f ~ a a e s of

P i . 15) . Because t h e sys t em is l i n e a r , t h e responses also

h a d a s i n u s o i d a l l y - m o d u l a t e d component, b u t w i t h different

a a i p l i t u a e s a n d phases . T h e e x p e r i m e n t was r e p e a t e d wi th the

liqht inoduLation replaced by modulated c u r r e n t directly

i n j e c t e d i n t o t h e e c c e n t r i c cell ( c e n t e r f r a a e of F i g . 1 5 ) .

If t h e sys tem wer? t r u l y linear, t h e n t h e p r o d u c t of t h e

measured q a i n f r o n t h e f i r s t t r a n s d u c t i o n ( l i g h t t o

g e n e r a t o r p o t e n t i a l ) with t h e a e a s u r e d g a i n f rom t h e s e c o n d

t r a n s d u c t i o n ( g e n e r a t o r p o t e n t i a l t o f i r i n g r a t e ) s h o u l d

e q u a l t h e measured gain f o r t h e overall t r a n s d u c t i o n (light

t o f i r i n g r a t e ) , a n d f u r t h e r m o r e , t h e r e s p e c t i v e p h a s e s

s h o u l d a d d . Good agreement was f o u n d a t e v e r y m o d u l a t i o n

f i q . 15. Responses of a LimuZus

modula ted i n p u t s . T h e t o p and bottom frames s h o w

r e s p e c t i v e l y t h e q s n e r a t o r p o t e n t i a l and firing r a t e i n

response t o s i n u s o i d a l l y - m o d u l a t e d i l l u m i n a t i o n . The middle

f r ame shows the f i r i n q r a t e i n response t o steady b a c k g r o u n d

i l l u m i n a t i o n v i t h s i n u s o i d a l l y-modulated current i n j e c t e d

d i r e c t l y i n t o t h e e c c e n t r i c ce l l . L e a s t - s q u a r e s f i t s were

made t o t h e d a t a t s t v e e n t h e v e r t i c a l lines. T h e t r a n s f e r

f u n c t i o n s i n F i q . 1 6 w e r e b a s e d on t h e r e su l t s of t h i s

e x p e r i n e n t a n d s i m i l a r e x p e r i m e n t s a t d i f f e r e n t modu la t ion

f r e q u e n c i e s (from K n i q h t , Toyoda, a n d Dodge, 1 9 7 0 ) .

' I , , I ) I ' I '

Modulation frequency (Hz)

F i q . 1 6 . The a m p l i t u d e s (above) and phases (below) o f

t he t r a n s f e r f u n c t i o n s f o r t v o e l e m e n t s o f signal process ing

i n t h e LimuZus r e t i n a . I n t h e l e f t p a n e l , t h e measured

t r a n s d u c t i o n o f l i g h t t o g e n e r a t o r p o t e n t i a l (open circles)

a n d c u r r e n t t o f i r l n g rate ( s o l i d circles) a r e shown w i t h

best-fit curves t h r o u g h t h e d a t a p o i n t s . I n t h e right

p a n e l , t h e o v e r a l l t r a n s d u c t i a n o f light t o firing r a t e is

shown b a s e d o n 3 i r a c t measurements (op.en circles) and

p r e d i c t e d f r o m t h e d a t a in t h e l e f t p a n e l ( s o l i d curves)

( f r o m K n i q h t , Toyoda, a n d Dodge, 197G) .

frequency t e s t e d b e t w e e n 9 . 7 a n d 10 Hz, as shown in Pig. 16.

T h e response cf the ommat id ium t o an a r b i t r a r y t e m p o r a l

p a t t e r n of i l l u a i n a t i o n can b e p r e d i c t e d froa t h e e a s n r e d

t r a n s f e r f u n c t i o n .

Knight, T o y o d a , and Dodge a l s o f o u n d t h e transfer

f u n c t i o n f o r t h e l a t e r a l i n h i b i t o r y p a t h w a y and were a b l e t o

s u c c e s s f u l l y g e n e r a l i z e the Hart l i n e - R a t l i f f e q u a t i o n t o t h e . f r e q u e n c y d o m a i n . I f l i g h t m o d u l a t i o n of i n t e n s i t y lc, a n d

frequency f p r o d u c e s a q e n e r a t o r p o t e n t i a l

where G ( f ) is the t r a n s f e r f u n c t i o n , t h e n t h e m o d u l a t e d

f i r i n q r a t e f i s d e t e r m i n e d by

where Ts(f) and TL(f) are r e s p e c t i v e l y t h e t r a n s f e r

f u n c t i o n s for s e l f - i n h i b i t i o n a n d l a t e r a l i n h i b i t i o n , a n d

K a n d ko6 a r e r e a l c o e f f i c i e n t s . The q u a n t i t i e s i n

t h i s e q u a t i o n a re g e n e r a l l y c o m p l e x a n d of the form

A eiq , w h e r e A is t h e a m p l i t u d e a n d i s t h e

p h a s e . T h e t r a n s f e r f u n c t i o n f o r 1 a t e r a 1 i n h i b i t i o n d e p e n d s

on t h e s e p a r a t i o n b e t w e e n o m m a t i d i a o n l y t h r o u g h the

a r p l i t u d e , w h i c h s c a l e s by a r e a l coefficient. T h i s a o d e l

a l l o w t h e response o f t h e LimuZus r e t i n a t o b e p r e d i c t e d f o r

a r b i t r a r y s p a t i a l a s well a s t e m p o r a l p a t t e r n s of

i l l u m i n a t i o n .

: * b a t t l e tes tedw and refined by s e v e r a l g e n e r a t i o n s of

i n v e s t i q a t o r s , r e p r e s e n t s a s tandard a g a i n s t which f u t n r e

n o d e l s mill be judged. The present study a t t e m p t s t o b u i l d

on t h e LimuZus ~ ~ o d e l by g e n e r a l i z i n g it t o the vertebrate

n e r v o u s s y s t e m . Further d e t a i l s a b o u t the LimuZus m o d e l c a n

b e f o u n d i n s e v e r a l reviews [Dodqe, 1969; D o d g e , Shapley,

a n d K n i q h t , 1970; H a r t l i n e and R a t l i f f , 1972; Knight , 1973)

and a vo lume o f c o l l e c t e d p a p e r s [ R a t l i f f , 7 9 7 4 ) .

Generalizing the LimuZus a o d e l

T h e lateral i n h i b i t o r y pathways i n t h e LhZus r e t i n a

o p e r a t e i n a way w h i c h is c o n s i s t e n t w i t h the " s l o w

p o t e n t i a l t h e o r y " o f n e u r o n a l i n t e r a c t i o n d i s c u s s e d on

page 34. Stevens ( 1 5 6 4 ) has s h o v n t h a t the "slow p o t e n t i a l

t h e o r y f f also e x p l a i n s t h e t r a n s i e n t r e s p o n s e of t h e LimuZus

r e t i n a t o temporal p a t t e r n s o f l i g h t . A sudden increase in

t h e i l l u m i n a t i o n o f a n o m m a t i d i a causes a s u d d e n r ise i n t h e

f i r i n q rate o f t h e e c c e n t r i c c e l l , followed by d e c a y t o a

new r e s t i n q level. S t e v e n s p r e s e n t e d e v i d e n c e t h a t a

s e l f - i n h i b i t o r y p a t h w a y e x i s t s w h i c h o b e y s t h e "slow

p o t e n t i a l t h e o r y " . Thus, i n a t l e a s t o n e p i ece of n e r v o u s

system, t h e n s l o w p o t e n t i a l theory" p r o v i d e s a n adequate

d e s c r i p t i o n a n d a quantitative model .

The LimuZus retina is an i d e a l p r e p a r a t i o n b e c a u s e of

its regular s t r u c t u r e and easy a c c e s s i b i l i t y . Is the

LimuZus r e t i n a special i n o t h e r r e s p e c t s a s well, and might

t h e "s low p o t e n t i a l t h e o r y " be less s n c c e s s f u l when a p p l i e d

t o o t h e r n e r v o u s s y s t e m s ? T h e l i n e a r i t y o f t h e L h Z u s

r e t i n a is a n a s p e c t t h a t may n o t a p p l y e l s e w h e r e . Below

t h r e s h o l d t h e f i r i n g r a t e o f a neu ron which is n o t

a u t o a c t i v e falls t o z e r o , a n d f o r large i n p u t s the f i r i n g

rate c a n n o t exceed s o m maximum r a t e owing t o a miniaum

r e c o v e r y time. Al though some n e u r o n s , l i k e those i n t h e

LimuZus r e t i n a , operate i n t h e a p p r o x i a a t e l y l i n e a r r e g i o n

between t h r a s h o l d and s a t u r a t i o n , many n e u r o n s i n t h e

v e r t e b r a t e n e r v o u s s y s t e m o p e r a t e i n their n o n l i n e a r region.

N o n l i n e a r i t y i n t r o d u c e s q u a l i t a t i v e l y new f e a t u r e s i n t o

n e u r o n a l i n t e r a c t i o n and g r e a t l y c o m p l i c a t e s q u a n t i t a t i v e

models.

Ano the r q u a l i t a t i v e d i f f e r e n c e between t h e ~imuZus

r e t i n a and t h e v e r t e b r a t e n e r v o u s s y s t e m is t h e d e g r e e of

v a r i a b i l i t y i n e l e c t r i c a l r e c o r d i n g s . An e c c e n t r i c c e l l i n

t h e LimuZus r e t i n a fires r e g u l a r l y i n r e s p o n s e to a flash of

l i g h t (Fiq. 121, but a c e l l i n t h e c a t v i s u a l c o r t e x

r e s p o n d s i r r e g u l a r l y ( P i g . 9) . S t o c h a s t i c models are

ccmmonly u s e d when s o u r c e s of n o i s e a r e p r e s e n t . Given t h e

deqree o f v a r i a b i l i t y found i n most parts of t h e v e r t e b r a t e

n e r v o u s system, a s t o c h a s t i c model of i n t e r a c t i n g n e u r o n s

would be aore a p p r o p r i a t e t h a n a d e t e r m i n i s t i c one. I n

s y s t e m s e n g i n e ~ x i n g the t y p e of m o d e l d e v e l o p e d i n this

s t u d y is c a l l e d a s t o c h a s t i c s t a t e - v a r i a b l e m o d e l ,

The Limulus r e t i n a h a s a p a r t i c u l a r l y s i m p l e and

r e q u l a r p a t t e r n of l a t e r a l i n h i b i t o r y i n t e r c o n n e c t i o n s ,

A l t h o u q h t h e details of s y n a p t i c i n t e r a c t i o n i n the n e n r o p i l

b e n e a t h the ommat id ia are i n fact q u i t e c o m p l i c a t e d , the

l a t t i c e - l i k e s t r u c t u r e of t h e LimuZus retina p r o d u c e s a

s t a t i s t i c a l l y a v e r a g e d i n t e r a c t i o n that is q u i t e r e g u l a r .

I n other n e r v o u s systems t h e p a t t e r n is more c o m p l i c a t e d and

d e t a i l e d c o n n e c t i o n s a r e known i n only a few special cases,

s u c h a s t h e vertebrate c e r e b e l l u n . To early ana tomi-s t s t h e

b r a n c h i n j p a t t e r n s of d e n d r i t e s a n d a x o n s in cerebral cortex

h a d a random a p p e a r a n c e , but e v i d e n c e is accumulating t h a t

c o n n e c t i o n s b e t w e e n c o r t i c a l n e u r o n s are s p e c i f i c and f a r

from r a n d o a (dountcastle, 7978) . U i t h o u t knowledge o f t h e

precise c o n n e c t i v i t y o n l y q u a l i t a t i v e p r o p e r t i e s of a model

c a n b e s t u d i e d .

A l t h o u g h t h e firing rate is the p r i m a r y v a r i a b l e i n t h e

B a r t l i n e - A a t l i f f n o d e 1 o f the LimuZus retina, the graded

membrane p o t a n t i a l i s an i m p o r t a n t i n t e g r a t i v e v a r i a b l e

w i t h i n a n e u r o n a n d has a central r o l e i n the "slow

p o t e n t i a l t h e o r y " . A m a j o r i t y o f n e u r o n s i n t h e vertebrate

r e t i n a d o not even p r o d u c e a c t i o n p o t e n t i a l s , and p a r t s of

n e u r o n s i n o t h e r a reas of t h e nervous system seen to be

s p e c i a l i z e d f o r l o c a l l y p r o c e s s i n g graded p o t e n t i a l s , T h u s , I

t h e uiembrane p o t e n t i a l is a a o r e f u n d a a e n t a l v a r i a b l e than

the f i r i n g r a t e , a l t h o u g h f o r t h o s e n e u r o n s which p roduce

a c t i o n p o t e n t i a l s t h e f i r i n g r a t e r e m a i n s an i m p o r t a n t

s e c o a d a r y v a r i a b l e ,

A s i i u p l e model o f i n t e r a c t i n g n e u r o n s is a o t i v a t e d i n

P a r t 1 which i n c o r p o r a t e s t h e n o n l i n e a r i t y , t h e e l e c t r i c a l

v a r i a b i l i t y , a n d t h e precise c o n n e c t i v i t y o b s e r v e d in

v e r t e b r a t e n e r v o u s s y s t e m s , P r o b a b i l i t y t h e o r y i s used t o

s t u d y t h e s t a t i s t i c a l p r o p e r t i e s o f t h e model in a t w o - t i e r

a n a l y s i s : t h e a v e r a g a membrane p o t e n t i a l s and a v e r a g e firing

r a t e s a r e exarrined i n Part 11, and t h e v a r i a n c e s and

c o r r e l a t i o n s between meslbrane p o t e n t i a l s a r e a n a l y z e d in

P a r t 111. I n P a r t IV t h e model is a p p l i e d t o motor l e a r n i n g

i n t h e c a r e b s l l u m ,

A11 t h e r e s u l t s i n t h e aain body of Chis s t u d y a r e

d e r i v e d f rom a n o d e l whose p r i m a r y v a r i a b l e is t h e

c o n t i n n o u s membrane p o t e n t i a l . I n t h e a p p e n d i x a

p o i n t - p r o c e s s model i s m o t i v a t e d vh ich i s a more r e a l i s t i c

model f o r inpulse-producing n e u r o n s and v h i c h takes i n t o

a c c o u n t refinements s u c h a s p r o p a g a t i o n d e l a y s i n a x o n s and

e l e c t r a t o n i c c o n d u c t i o n i n d e n d r i t e s . All t h e main r e s u l t s

o f t h e c o n t i n u o u s model a l s o ho ld f o r t h e more g e n e r a l I

p o i n t - p r o c e s s m c d e l .

I. T h e H o n l i n e a r X o d e l

I n m o s t o f the p r e s e n t s t u d y a n e u r o n is c o n s i d e r e d a

p o i n t w i t h a s e t o f p r o p e r t i e s , s u c h as an e lec t r i ca l

p o t e n t i a l a n d a n i n p u t - o u t p u t f u n c t i o n , a n d c o n n e c t i o n s

b e t w e e n neurons a r e r e p r e s e n t e d b y a s e t o f c o u p l i n g

s t r e n q t h s , a r r a y e d f o r c o n v e n i e n c e i n a s q u a r e ntatrir, A

r e a s o n a b l e q u e s t i o n t o ask is w h e t h e r s u c h a % a r t o o n g t

n e u r o n a l m o d e l o v e r s i m p l i f i e s e s s e n t i a l p r o p e r t i e s o f r e a l

n e u r o n s . T h e a n s w e r l i e s u l t i m a t e l y i n t h e v a l u e o f t h e

m c d e l f o r o r q a n i z i n g e x p e r i n e n t a l d a t a , b u t t h e q u e s t i o n c a n

b e p a r t i a l l y a n s w e r e d b y t e s t i n g w h e t h e r r e s u l t s b a s e d on

t h e m o d e l a re s s n s i t i v e t o p a r t i c u l a r s i m p l i f y i n g

a s s u r a p t i o n s . F c r e x a m p l e , the e a i n results o f t h e mode l a re

shown i n t h e a p p e n d i x t o h o l d e v e n when Bore r e a l i s t i c

d e n d r i t i c i n p u t s a r e i n c l u d e d .

N e u r o n s a n d ~ * t l i c r o c i r c u i t s t g

A c c o r d i n g t o t h e p r i n c i p l e o f " d y n a m i c p o l a r i z a t i o n "

( C a j a l , 1 9 1 1 ) , i n f o r m a t i o n w i t h i n n e u r o n s is t r a n s m i t t e d o n e

way: p a s s i v e d e n d r i t e s r e c e i v e e l e c t r i c a l s i g n a l s t h r o u g h

s y n a p s e s ; t h e i n t e g r a t e d i n p u t a t t h e s p i k e - i n i t i a t i n g z o n e

i s c o d e d i n t o i m p u l s e s ; the i m p u l s e s are t h e n s e n t o u t

t h r o u q h t h e a x o n t o i n f l u e n c e other n e u r o n s . A l t h o u g h t h i s

c l a s s i c a l p i c t u r e , a s shown i n Fig. 2 , i s a good

a p p ~ o x i a a t i o n f cr many neu rons , some r e c e n t f i n d i n g s are

i n c o n s i s t e n t w i th it: s y n a p s e s h a v e been f o u n d from d e n d r i t e

t o d e n d r i t e and from axon t o axon, and some synaptic

i n t e r a c t i o n s t a k e p l a c e t h a t a r e n o t media ted b y impulses

(Shsphe rd , 1974 ; R a k i c , 1975; S c h r n i t t , Dev , & S m i t h , 1 9 7 6 ) .

A new c c n c e ~ t t h a t h a s emergad is the ' * m i c r o c i r c u i t M

( S h e p h e r d , 1 9 7 E ) , w h i c h is a l o c a l i z e d a n a t o m i c a l region

t h a t p r o c e s s e s i n f o r m a t i o n t h r o u g h c o n t i n u o u s l y g r a d e d

aembrane p o t e n t i a l s . I n t h e a o d e l d e v e l o p e d here t h e

membrane p o t e n t i a l is the p r i m a r y p h y s i c a l v a r i a b l e so t h a t

b o t h t h e c l a s s i c a l impu l se -p roduc ing neuron and t h e l o c a l

l l m i c r o c i r c u i tsil can b e i n c l u d e d .

Because e l e c t r o t o n i c p o t e n t i a l s dec remen t e x p o n e n t i a l l y

i n p a s s i v e dendrites, a neuron v h i c h transmits s i g n a l s o v e r

a l o n g d i s t a n c e m u s t use r e g e n e r a t i v e a c t i o n potentials, o r

i m p u l s e s , t o F r c v i d e f a s t and r e l i a b l e communicat ion. These

p r o j e c t i o n neurons most n e a r l y r e s e m b l e t h e c l a s s i c a l model

shown i n Fig. 2, a n d have b e e n s t u d i e d i n g r e a t e r d e t a i l

than t h e f q l o c a l c i r c u i t neurons1* (Rak ic , 1975) which o f t e n

f crm t h e m i c r o c i r c u i t s ~ ~ . The g e n e r a l 1 y a c c e p t e d

e x p l a n a t i c n f o r how i n f o r n a t i o n i s coded i n t o i m p u l s e s and

t r a n s m i t t e d ky p r o j e c t i o n neurons w i l l b e summarized here

and used t o m o t i v a t e a q u a n t i t a t i v e n o n l i n e a r v e r s i o n .

Graded membrane p o t e n t i a l s a r e c a l l e d s low p o t e n t i a l s

t o d i s t i n g u i s h t h e m f rom t h e much more r a p i d l y v a r y i n g

a c t i o n p o t e n t i a l . Above t h e t h r e s h o l d f o r p s s d u c i n g a c t i o n

p o t e n t i a l s , t h e f i r i n g r a t e i n c r e a s e s m o n o t o n i c a l l y w i t h t h e

n e u r o n ' s merabrane p o t e n t i a l . The i l e p u l s e s t r a n s r e i t t e d by

t h e axon t o t h e s y n a p t i c c o n n e c t i o n s w i th o t h e r n e u r o n s t h e n

produce p o s t s y n a p t i c p o t e n t i a l s . Accord ing t o t h e "s low

p o t e n t i a l t h e o r y " o f n e u r o n a l i n t e r a c t i o n ( S t e v e n s , 1 9 6 4 ,

1966), t h e t e r r p o r a l l y summed s l o w p o t e n t i a l i n t h e

p o s t s y n a p t i c n e u r o n i s an a p p r o x i m a t e r e p r o d u c t i o n o f t h e

membrane p o t e n t i a l i n the p r e s y n a p t i c n e u r o n , d i m i n i s h e d i n

a m p l i t u d e b u t u n a l t e r e d i n s h a p e . I n s h o r t , t h i s app roach

assumes t h a t i n f o r m a t i o n abou t t h e membrane p o t e n t i a l i n one

neu ron is l i n e a r l y t r a n s d u c e d t o o t h e r n e u r o n s b y t h e f i r i n g

r a t e . However, below t h r e s h o l d n o i m p u l s e s a r e produced,

and f o r l a r g e irernbrane p o t e n t i a l s t h e f i r i n g r a t e s a t u r a t e s

a t some maximum. These n o n l i n e a r i t i e s a r e t a k e n i n t o

a c c o u n t i n t h e q u a n t i t a t i v e model g i v e n h e r e and deve loped

i n Bore d e t a i l i n t h e append ix .

Q u a n t i t a t i v e n o d e l

One o f t h e s i m p l e s t l i n e a r mode l s f o r n e u r o n s which do

n o t produce action p o t a n t i a l s i s g i v e n by

d T~ "L * "a = Z b K a b V b f Rain, (1)

w h e r e V, are t h e s o m a t i c membrane p o t e n t i a l s , t is

t h e aersbtane time c o n s t a n t , La a r e t h e s y n a p t i c i n p u t

c u r r e n t s , QL a r e t h e e f f e c t i v e l o a d resistanceso and L Kab a r e d i m c s i o n l e s s c o u p l i n g s t r e n g t h s . The l e f t s i d e

o f Eq. (1) p r o v i d e s t e m p o r a l i n t e g r a t i o n like t h a t of a

l e a k y c a p a c i t c r , a n d t h e r i g h t s i d e t a k e s i n t o a c c o u n t the

i n p u t s a n d i n t e r c o n n s c t i o n s ,

A c t i o n p o t e n t i a l s i n t r o d u c e a s t r o n g n o n l i n e a r i t y i n t o

n e u r o n a l i n t e r a c t i o n . T h e r e s p o n s e o f a n i d e a l i z e d

i m p u l s e - p r o d u c i n g n e u r o n t o a c o n s t a n t i n p u t c u r r e n t is

shown i n F i q . 17. I m a g i n e f o r t h e moment t h a t action

p o t e n t i a l s are p r e v e n t e d s o t h a t t h e membrane p o t e n t i a l is

a l l o w e d t o v a r y s l l ioothly a b o v e t h e t h r e s h o l d f o r d i s h a r g e ,

a s i n d i c a t e d t y t h e d a s h e d l i n e i n F i g . 17. Define t h e

effective membrane p o t e n t i a l 0 a s t h e membrane p o t e n t i h l

w h i c h would b e f r e s e n t i n a n e u r o n i f t h e a c t i o n p o t e n t i a l

were absent. The f i r i n q r a t e f3 ( $ ) o f an i d e a l i z e d

n e u r o n , w h i c h i s a f u n c t i o n o f the e f f e c t i v e nteabrane

p o t e n t i a l a s shown i n P i g . 18, i n c r e a s e s m o n o t o n i c a l l y above

t h r e s h o l d a n d , because of the a b s o l u t s r e f r a c t o r y p e r i o d ,

a p p r o a c h e s a n uFFer bound.

A c t i o n p o t e n t i a l s c a n n o t be e l i m i n a t e d f rom a n e u r o n

when t h e i n t e q r i t y of a n interacting s y s t e m o f n e u r o n s is

i n p o r t a n t . Hence, i n d i r e c t m e t h o d s mus t b e u s e d t o e s t i m a t e

t h e e f f e c t i v e membrane p o t e n t i a l a b o v e t h e t h r e s h o l d f o r

discharge, T h e r e g e n e r a t i v e p a r t o f t h e a c t i o n p o t e n t i a l

P i q . 1 7 . a) T h e membrane p o t e n t i a l of an i d e a l i z e d

neuron in response t o a c o n s t a n t i n p u t current a s a function

o f t i m e . T h e d a s h e d l i n e r ep resen t s t h e e f f e c t i v e membrane

potential w h i c h would be p r e s e n t i n the absence of a c t i o n

p o t e n t i a l s . b) T h e membrane p o t e n t i a l o f a s e c o n d i d e a l i z e d

n e u r o n w h i c h receives synaptic input f rom t h e first, as

i l l u s t r a t s d i n F i g . 1 9 (from Se j n o u s k i , 1 9 7 7 a ) .

Fig. 18. T h e f i r i n q ra te P C P ) a s a f u n c t i o n of

e f f e c t i v s mrmbrme p o t e n t i a l 4 f o r an i d e a l i z e d n e u r o n

w i t h f i r i n q t h r e s h o l d 0 ( f r o 5 Se j n o w s k i , 1977a).

lasts for o n l y a f e u m i l l i s e c o n d s a n d can b e e a s i l y

recognized a n d rersoved from the record iog , A f t e r p o t e n t i a 1 s

f o l l o w i n g a n a c t i o n p o t e n t i a l reset the aeabrane p o t e n t i a l

below t h r e s h o l d , b u t t h e e f fec t o f t h e reset decays

e x p o n e n t i a l l y ( w i t h t h e membrane time c o n s t a n t ) a f t e r each

i m p u l s e (see a p p e n d i x ) . An i n t r a c e l l u l a r r e c o r d i n g w i t h

a c t i o n potentials reaoved a n d t h e reset c o m p e n s a t e d s h o u l d

be a g o o d a p p r o x i m a t i o n t o the ef fec t ive membrane p d e n t i a l

d e f i n e d a b o v e , I n a n e u r o n which d o e s not p r o d u c e an a c t i o n

p o t e n t i a l , t h e d i f f i c u l t y a b o v e t h r e s h o l d d o e s n o t arise and

t h e e f f e c t i v e m e n b n n e p o t e n t i a l is the membrane p o t e n t i a l .

F o r the coupled p a i r of neurons r e p r e s e n t e d i n P i g . 19,

r e p e t i t i v e firing of the p r e s y n a p t i c n e u r o n p r o d u c e s a

r e p e t i t i v e p o s t s y n a p t i c p o t e n t i a l , a s shown i n Fig. 17b,

Under s t e a d y - s t a t e c o n d i t i o n s t h e a v e r a g e p o s t s y n a p t i c

p o t e n t i a l i s c o n s t a n t and, f o r an i d e a l i z e d neuron,

p r o p o r t i o n a l t o t h e f i r i n g ra te , A model for a c o l l e c t i o n

o f i d e a l i z e d i n t e r a c t i n g n e u r o n s is q i v e n b y

w h e r e is t h e neabrane tine c o n s t a n t , 7 b ( t ) i s t h e

i n p u t f i r i n g r a t e , f b ( $b(t)) i s the f i r i n g r a t e o f t h e

n o n l i n e a r i n t e r a c t i o n d e f i n e d a b o v e , and Kab a r e t h e i n p u t

c o u p l i n q s t r e n g t h s a n d 8,b a r e t h e i n t e r n a l c o u p l i n g

s t r e n q t h s , w i t h u n i t s o f p o t e n t i a l / f i r i n g r a t e . Because the

e f f ec t ive metrbrana p o t e n t i a l s $,it) a r e c o n t i n u o u s , t h i s

Fiq. 19 . Schematic i l l u s t r a t i o n of two n e u r o n s w i t h a

s y n a p t i c c o n n e c t i o n K L L from t h e first t o the second and

w i t h i n p u t s a n d r e s p e c t i v e l y . H i c r o p i p e t t e s

record t h e i a t r a c e l l u l a r membrane p o t e n t i a l s Vi and !/A

(from Se j n o v s k i , 1977a) .

n o n l i n e a r model a p p l i e s equally w e l l t o neurons vhich d o not

produce an a c t i o n potential and p a r t s of neurons which

i n t e r a c t through g r a d e d s y n a p s e s . Hence, m m i c r o c i r c n i t s M

c a n b e modeled a s well as classical neurons.

Above t h e t h r e s h o l d for l a t e r a l i n h i b i t i o n , t h e

response of t h e Limulus r e t i n a t o a s t e a d y - s t a t e p a t t e r n of

l i q h t i s g i v e n , t o a good a p p r o x i a a t i o n , by the

H a r t l i n e - R a t l i f f model (1957)

b

w h e r e TO is t h e r a t e of f i r i n g o f a n o n l a a t i d i u a . rbO is t h e i n h i b i t o r y threshold, e, represents t h e input, and

f K a b a r e t h e i n h i b i t o r y coef f icieets . Under noriaal

e x p e r i m e n t a l c o n d i t i o n s a steady b a c k g r o u n d i l l u m i n a t i o n

p r o d u c e s a s t e a d y background f i r i n g rate a g a i n s t w h i c h

v a r i a t i o n s in t h e rate a r e measured.

I f i n t h e nonlinear moael the effective m ~ b r a n e

p o t e n t i a l s a r e constant, t h e n t h e y can b e e l im ina t ed in

f avo r o f the f i r i n g r a t e s

The H a r t l i n e - R a t f i f f e q u a t i o n i s e q u i v a l e n t t o t h i s equation

when p ( $ ) , shown i n Pig. 18, is restr ic ted t o the

a p p r o x i m a te fy l i n e a r r e g i o n above t h r e s h o l d .

Al though many details of r e a l n e u r o n s a r e n o t i n c l u d e d

i n t h e c o n t i n u o u s model mo t iva t ed h e r e , t h e a a i n r e s u l t s

b a s e d o n i t a l s o h o l d i n a more general m o d e l , given i n t h e

a p p e n d i x , which takes i n t o account a x o n a l l a t e n c y and

d e n d r i t i c electrotonus.

I n summary, the h i g h l y n o n l i n e a r a c t i o n p o t e n t i a l was

e l i m i n a t e d b y f i r s t r e d e f i n i n g the membrane p o t e n t i a l above

t h e threshold for d i s c h a r g e , and s e c o n d l y b y s m o o t h i n g t h e

p o s t s y n a p t i c p o t e n t i a l s . T h e n o n l i n e a r m o d e l ( 2 ) based on

t h i s ef fect ise membrane p o t e n t i a l d i f f e r s from t h e linear

m o d e l ( I ) b y an effective n o n l i n e a r i n t e r a c t i a n , a s

r e p r e s e n t e d i n Fiq. 1 8 .

11. A v e r a g e t i e n b r a n e P o t e n t i a l s in S t e a d y S t a t e

I n e l e c t r i c a l r e c o r d i n g s f rom n e u r o n s i n v a r i o u s parts

of t h e b r a i n , i a p u l s e s o f t e n o c c u r i r r e g u l a r l y a n d aeabrane

p o t e n t i a l s h a v e a r a n d o a l g f l u c t u a t i n g c o m p o n e n t . Sorae

e x a m p l e s a r e g i v e n i n F i g s . 8, 9, a n d 10. A s d i s c u s s e d in

the i n t r o d u o t i o n , the s o u r c e s o f this r a n d o m c o a p o n e n t

i n c l u d e maabrane n o i s e f r o a t h e o p e n i n g a n d c l o s i n g o f i o n i c

channels, f l u c t u a t i o n s i n t h e t h r e s h o l d f o r p r o d u c i n g a c t i o n

p o t e n t i a l s , a n d f l u c t u a t i o n s i n t h e number o f t r a n s m i t t e r

packets r e l e a s e d d u r i n g s y n a p t i c t r a n s m i s s i o n ( V e r v e e n and

DeFelice, 1 9 7 4 ; H o l d e n , 1976 ; Neher a n d Stevens, 7977;

S t e v e n s , 1977; & F e l i c e , 1977) . Because o f t h e v a r i a b i l i t y i n e lec t r ica l r e c o r d i n g s ,

t h e e l e c t r i c a l r e s p o n s e o f the b r a i n t o a s e n s o r y s t i m u l u s

i s o f t e n m e a s u r e d o n a number o f t r i a l s u n d e r c o n d i t i o n s a s

u n i f o r m a s possible f r o m t r i a l t o t r i a l . I f the t r i a l s are

r e a s o n a b l y i n d e p e n d e n t and c o n d i t i o n s a r e r e a s o n a b l y

s t a t i o n a r y , t h e n t h e u n c o n t r o l l e d p a r t o f the v a r i a b i l i t y

cancels and t h e a v e r a g e r e s p o n s e d o e s n o t d e p e n d on w h i c h

b l o c k o f t r i a l s were c h o s e n . When s p i k e t r a i n s a r e m e a s u r e d

t h e a v e r a q e r G s p o n s e is c a l l e d t h e p o s t s t i m u l u s t i i a e

h i s t o g r a m ( P S T ) , a s shown i n P i q . 3 , a n d when g r o s s

p c t e n t i a l s f rom g r o u p s o f n e u r o n s are m e a s u r e d , t h e a v e r a g e

i s c a l l e d t h e a v e r a g e evokad r e s p o n s e ( A E R ) ,

49

I n a PST each response i s a f u n c t i o n o f time af-r t h e

stimulus and t h e a v e r a g e is computed from a c o l l e c t i o n o r

e n s e a b l e of t r i a l s . T h i s form of a v e r a g e , c a l l e d an

e n s e m b l e a v e r a ge, is used t h r o u g h o u t t h i s s t u d y . I3

c c n d i t i o n s a r e s t a t i o n a r y (the s t i m u l u s is c o n s t a n t a n d the

system i s i n e q u i l i b r i u a ) , then a time a v e r a g e can b e

computed f rom a s i n g l e trial w h i c h (unde r f u r t h e r

c o n d i t i o n s ) is e q u a l t o the ensemble a v e r a g e . F o r exarnpl'e,

i n sta t i s t i c a 1 raechanics the t i m e - a v e r a ged eaacroscopic

v a r i a b l e s , such a s t e m p e r a t u r e and pressure, can b e computed

from a Gibbs ensemble of i d e n t i c a l l y p r e p a r e d sys t ems .

L e t f (t) r e p r e s e n t a s t a t i o n a r y s i g n a l , s u c h a s a n

i n t r a c e l l u l a r r e c o r d i n g of a membrane p o t e n t i a l o r an

e x t r a c e l l u l a r r e c o r d i r i g of a spike t r a i n . T h e t i r ae -ave rage

mean o f F ( t ) o v e r t i m e T i s d e f i n e d a s 7 ' ~ ' d t fct, .

7 - 0

Now c o n s i d e r a n ensemble of r e c o r d i n g s {fa(t)} f rom N

i n d e p e n d e n t t r i a l s under un i form c o n d i t i o n s . The

ensemble-average mean i s d e f i n e d a s

' a = l T h e e n s e ~ t l e - a v e r a g e mean can b e a f u n c t i o n o f time t . as

i n t h e c a s a o f the PST. I f , houever , c o n d i t i o n s are

s t a t i o n a r y , t her : & T N ( t ) i s i n d e p e n d e n t o f tine, and N + W

i f c e r t a i n e r g o d i c c o n d i t i o n s a r e a l s o s a t i s i f i e d (Halmos,

19561, then the a p p r o p r i a t e f inits of t h e t i m e - a v e r a g e mean

a n d t h e e n s e m b l e - a v e r a g e Bean a r e equal :

S t o c h a s t i c node l

E n g i n e e r i n g models of physical s y s t e m s usually t a k e

random i n f l u e n c e s i n t o account b y a d d i n g n o i s e terms t o

d e t e r m i n i s t i c e q u a t i o n s . For e x a n p l e , the input 7]a(t) to

t h e n o n l i n e a r mcdel ( 2 ) miqh t b e modeled b y a deterministic A

P a r t 171(t) a r d a w h i t e n o i s e componen t Wa(t) :

T h e n t h e mean and covariance of the input a r e

where E i s the e x p e c t a t i o n o r ensemble average and Qnb

i s the c o v a r i a n c e of t h e w h i t e n o i s e p r o c e s s . Rore

qenerally, t h e i n p u t c a n b e c o n s i d e r e d a n arbitrary Harkov

p r o c e s s w i t h f i n i t e s econd-o rde r moments

T h e p h y s i c a l s i g n i f i c a n c e of t h e covariance is discussed i n

the n e x t part .

T h e r e s p o n s e of the nonlinear model t o an ensemble of

r a n d o n l y -vary i n q i n p u t s is a n ensemble of random1 y - r a f y i n g

membrane p o t e n t i a l s . Huch c a n b e l e a r n e d about t h e

s t o c h a s t i c mcdel b y examin ing the l o w e s t - o r d e r ~ o i r e n t s of

t h e membrane p o t e n t i a l s . From t h e e x p e c t a t i o n of Eq.

t h e mean cr a v e r a g e e f f e c t i v e membrane p o t e n t i a l s

A

s a t i s f y

where t h e a v o r a g e firing r a t e s a r e

S i n c e t h e r i g h t s i d e of Eg. (7) depends on a l l the moments

o f qO . the a v e r a g e f i r i n g r a t e Rg( 4 b ) , w r i t t e n

e x p l i c i t l y a s a f u n c t i o n o f t h e a v e r a g e meaibrane p o t e n t i a l A

$ b , d 2 p e n d s i m p l i c i t l y on a l l t h e h i g h e r - o r d e r moments a s

well, A l t e r n a t i v e L y , t h e a v e r a g e firing r a t e can b e

c o n s i d e r e d a f u n c t i o n a l o v e r a n ensemble o f raembrane

p c t ~ n t i a l s q b C + b ] , but t h i s n o t a t i o n m i g h t be c o n f u s i n g

and w i l l b e a v o i d e d . Thus , E q . (7) i s n o t a s t r i c t l y c l o s e d

set o f r e l a t i o n s f o r Tb . b u t is coup led t o e q u a t i o n s f o r

t h e h i q h e r - o r d e r moments of C/)b . Because of rrembrane p o t e n t i a l f l u c t u a t i o n s , t h e a v e r a g e

A

r a t e o f f i r i n q R b ( # b ) a s a f u n c t i o n of $b . h o l d i n g a l l

h i g h e r - o r d e r mcments fixed, is smoother t h a n p b ( $ b ) a

f u n c t i o n of +b . For example, P i g . 2 0 s h o w s

P ( # ) a s t e p f u n c t i o n a t t h r e s h o l d 0 and v i t h $ h a v i n g a G a u s s i a n d i s t r i b n t i o n . I n g e n e r a l , Eq, ( 6 ) f o r the

mean e f f e c t i v e membrane p o t e n t i a l s is c o u p l e d , through the A

t e r m s w i t h R b ( $ b ) , t o e q u a t i o n s f o r t h o higher manents .

N o n l i n e a r e q u a t i o n s s i m i l a r t o Eq. ( 6 ) h a v e been

i n v e s t i g a t e d b y Y h i t e (1961 ) , i i i l s o n and Cowan (1968) , and

G r o s s b e r g ( 1973 ) . who base their models on p o p u l a t i o n s of

n e u r o n s . The p r i m a r y v a r i a b l e i n t h e i r e q u a t i o n s i s t h e

f r a c t i o n of n e u r o n s i n an "excited s t a t e " . I n t h e p r e s e n t

c a s e t h e membrane p o t e n t i a l s of i n d i v i d u a l neurons a r e

s t u d i e d a n d t h e a v e r a g e s a re o v e r ensembles of identically

p r e p a r e d e x p e r i m e n t a l t r i a l s r a t h e r t h a n o v e r physical

p o p u l a t i o n s , T h e e x p e c t a t i o n c o r r e s p o n d s t o t h e a v e r a g e

r e s p o n s e from t h e ensemble .

I n t h e s t a t i o n a r y c a s e t h e mean e f f e c t i v e mrnbrane

p o t e n t i a l s a r e c o n s t a n t and s a t i s f y

v i t h t h e mean i n p u t s

E l i n i n a t i n q from E q . 18) i n f a v o r o f t h e a v e r a g e

f i r i n q r a t e

a n a l t e r n a t i v e form o f t h e s t e a d y - s t a t e e q u a t i o n is

d e r i v a t i v e as a f u n c t i o n of the mean ef fect ive

membrane p o t e n t i a l . The t h r e s h o l d f o r f i r i n g is 8 (from S e j n o v s k i , 1977a) .

S t e a d y - S t a t e S o l u t i o n s

Although t h e H a r t l i n e n R a t l i f f m d e l (3) of t h e ~ h Z u s

r e t i n a i s l i n e a r above t h r e s h o l d , the c u t o f f b e l o v threshold

makes t h e m o d e l n o n l i n e a r , H a d e l e r ('3974) h a s used a f i x e d

p o i n t theorom t o s t u d y t h e existence of s o l u t i o n s t o the

LimuZus model w i t h t h r e s h o l d s . H i s a p p r o a c h can be

q e n e r a l i z e d t o t h e n o n l i n e a r s t e a d y - s t a t e e q u a t i o n (8 ) f o r

the a v e r a g a membrane p o t e n t i a l s (Se jnowski , 1976a) ,

Because the s t e a d y - s t a t e e q u a t i o n (8) f o r depends

i m p l i c i t l y on t h e h i g h e r - o r d e r n o e e n t s of it c a n n o t

h e t r e a t e d i n d e p e n d e n t l y o f t h e h i g h e r - o r d e r e q u a t i o n s . The

question a s k e d here is whether t h e s t e a d y - s t a t e e q u a t i o n f o r

h a s a s o l u t i o n when a l l h i g h e r - o r d e r moments are h e l d

f i x e d , w h i c h is a n e c e s s a r y but not a sufficient c o n d i t i o n

f o r t h e r e t o exist a s t a t i o n a r y s o l u t i o n t o t h e c o u p l e d s e t

of e q u a t i o n s . T h e answer i s i n t h e affirmative f o r a l l

v a l u e s o f t h e h i g h e r - o r d e r moments.

A fixed ~ o i n t o f a f u n c t i o n is a p o i n t wh ich t h e

f u n c t i o n maps i n t o itself. T o p o l o g i c a l f i x e d - p o i n t theorems

q i v e v e r y g e n e r a l c o n d i t i o n s under w h i c h a f i x e d - p o i n t

s o l u t i o n must exist. I n o p e r a t a r n o t a t i o n t h e s t e a d y - s t a t e

e q u a t i o n (8)

can be written a s t h e fixed p o i n t

Theorem. (Erouuer) . E v e r y c o n t i n u o u s mapping of a

c l o s e d n - b a l l i n t o i t s e l f h a s a f i x e d p o i n t .

T h e map F is c o n t i n u o u s since R is c o n t i n u o u s ;

F i s bounded by

6 = l i K j / m ~ , + ! , R ( x ) ; X

and m a p s t h e ball

i n t o i t s e l f

~ : 6 - + & Hence b y t h e Brcuver f i x e d - p o i n t theorem t h e s t e a d y - s t a t e

e q u a t i o n a l w a y s has a t l e a s t one s o l u t i o n . However, the

s o l u t i o n nap n o t be s t a b l e and p e r i o d i c s o l u t i o n s , a s well

as l ess r e q u l a r behav io r , may a l s o occur . This c a s e

r o g u i r e s treatment of the coup led e q u a t i o n s f o r a l l t h e

moments o f t h e z m b r a n e p o t e n t i a l s .

A f u n c t i o n F on a metric s p a c e is a c o n t r a c t i o n n a p

i f , s t a r t i n q w i t h two a r b i t r a r y p o i n t s X I and X b , F(x , )

a n d F ( x 2 ) are a l v e y s closer together than t h e o r i g i n a l tso

p o i n t s , The c o n t r a c t i o n iaapping theorem s t a t e s t h a t a

c o n t r a c t i o n r a p a l v a y s has a u n i q u e fixed-point s o l u t i o n and

t h a t the s o l u t i o n can b e constructed by i t e r a t i o n s t a r t i n g

f rom a r b i t r a r y p o i n t .

T h e c o n t r a c t i o n mapping t h e o r e & can b e used to s h o w

t h a t if the nonlinearity is s u f f i c i e n t l y weak t h e n t h e A

s o l u t i o n cf the steady-s ta te e q o a t i a n (8) f o r $a , h o l d i n g

all h i g h e r - o r d e r a o o e n t s fixed, i s u n i q u e . Take two

a r b i t r a r y vectcrs X i and Xa i n a n N-dimensional

E u c l i d e a n space and c o n s i d e r

where

I f t h e a b o v e q u a n t i t y i n parenthesis i s l e s s t h a n one then

t h e m a p c o n t r a c t s and b y the c o n t r a c t i o n mapp ing t h e o r e m

t h e r e i s a u n i q u s s t e a d y - s t a t e s o l u t i o n . F o r t h e s igrnoid

a ( x ) shown in Pig. 20, t h e w i d t h of t h e t r a n s i t i o n r e g i o n

w f ron; t h e rrinimurr t o m a x i m u m firing r a t e is at least as

q r e a t a s the w i d t h d z f i n e d by t h e steepest t a n g e n t

X

where R~ is t h e maxiiaum f i r i n g r a t e . Hence the

c o n t r a c t i o n c o n d i t i o n is

Thus, i f n o s i n g l e c o n n e c t i o n is capable of moving any

neuron a c r o s s the t r a n s i t i o n r e g i o n (weak c o u p l i n g

c c n d i t i o n ) , then t h e r e i s a un iqua s t e a d y - s t a t e s o l u t i o n .

T h e q r a p h i c a l method used b y B h i t e (1961) a n d by Wilson

and Cowan ( 1 9 6 8 ) allows the s t e a d y - s t a t e s o l u t i o n s f o r two

i n t e r a c t i n q neurons t o b e v i s u a l i z e d . The s t e a d y - s t a t e A

e q u a t i o n ( 8 ) for can be i n v e r t e d

F c r two n e u r o n s (Piq. 27) t h e e q u a t i o n s a r e

4 A

which aro of the form Fa( ?,) and , ( ) . The

i n t e r s e c t i o n of these tuo equations is a g r a p h i c a l s o l u t i o n ,

a s i l l u s t r a t e d i n F i g . 2 2 .

P h e n the t r a n s i t i o n r e g i o n s o f R ; ( C ) are

s u f f i c i e n t l y wide ( v s a k c o u p l i n g ) , o n l y one s o l u t i o n e x i s t s ,

a s p ~ o v e n above . Nhen t h e transition r e g i o n s a r e nar row

(strong c o u p l i n g ) , m u l t i p l e s o l u t i o n s can exist (F ig . 22 )

a n d h y s t e r e s i s between t h e s e s o l u t i o n s m a y o c c u r a s t h e

i n p u t s va ry (Pig. 2 3 ) . B i f u r c a t i o n s t o new s o l u t i o n

branches c a n o c c u r even when t h e i n p u t s a r e h e l d f i x e d and

h i g h e r - o r d e r noments, such a s the variances o f which

c o n t r i b u t e t o the t r a n s i t i o n widths of Rq ( . are

a l l o w e d t o v a r y p a r a m e t r i c a l l y {Se jnowski, 1976a). St

bif u r c a t i c n p o i n t s , where new s o l u t i o n s appear o r n u l t i p l e

s c l u t i o n s merge, t h e behav io r of t h e sys t em i s probably

sensitive t o the h i g h e r - o r d e r moments of . w h i c h rere

assumed t o be c o n s t a n t i n the analysis g i v e n here.

T h e s t a b i l i t y of a s t e a d y - s t a t e s o l u t i o n can b e s t u d i e d

b y examining small p e r t u r b a t i o n s a round the s o l u t i o n . T h e

results o f t h e ~ e r t u r b a t i o n a n a l y s i s given below are u s e d in

t h e n e x t s e c t i o n t o s t u d y f e a t u r e d e t e c t i o n . A

L e t $ & ( x ) be a family o f s o l u t i o n s t o t h e s t e a d y - s t a t e A

e q u a t i o n (8) f o r i n p u t s LC,(x) smoo th ly p a r a m e t e r i z e d b y a

r e a l v a r i a b l e X . Frcm t h e Taylor series e x p a n s i o n s

t h e f i r s t - o r d e r v a r i a t i o n s of t h e s t e a d y - s t a t e ' a e q u a t i o n (8 ) satisfy

where

Fiq. 2 1 . S c h e m a t i c d i a g r a m o f the g e n e r a l two-neuron /' A

model w i t h i n p u t s ui and Ua , r e c i p r o c a l c o u p l i n g s

Pig. 22. Solutions of the two-neuron model are g i v e n A A A A

b y the intersections of TI ( ~ 2 ) (broken l i n e ) w i t h r L ( ~ I )

(solid lines), snoun here for the case of reciprocal

e x c i t a t i o n ( K l x 7 0 ) K 2 , > 0 ) without recurrent c o l l a t e r a l s

( K , , s K;zx= 0 ) . A v a r i e t y o f s t e a d y - s t a t e s o l u t i o n s is

e x h i b i t e , . i o y the four different values of the i n p u t

chosen (from Se jnouski, 1976a).

F i g . 2 3 . S a l n t i o n s o f t h e two-neuron model a re g i v e n

b y t h e i n t e r s e c t i o n s of t h 9 two c u r v e s , a s i n Fig. 22 b u t

w i t h K J L > 0 . of t h e three s o l u t i o n s shown, t h e center o n e

i s u n s t a b l e a n d the o u t e r two a r e s t a b l e . A s the input A Ihl is n r i e d ( t r a n s l a t i n g t h e v e r t i c a l curve v e r t i c a l l y )

t h e s t a o l e s o l u t i o n on one b r a n c h may d i s a p p e a r and t h e

state m a y j u m p ( a r row) t o t h e o t h e r s o l u t i o n branch. T h e

c l o s e d c i r c u i t d e s c r i b e s a hysteresis curve.

B e c a u s e o f t h e f a c t o r R; o n l y those n e u r o n s v h ~ s e

f i r i n g r a t e s a r e v e r y s e n s i t i v e t o changes i n t h e i r ~ i a b r a n e

p o t e n t i a l s , such as t h o s e n e u r o n s n e a r t h r e s h o l d , contribute /

s i g n i f i c a n t l y t c K a b , a s shown i n P i g , 20, T h e s o l u t i o n

for t h e f i r s t - o r d e r v a r i a t i o n , i f i t exists, i s g i v e n b y

where

T h i s resolvent e x i s t s i f a n d o n l y i f n o n e o f t h e eigenvalues

An o f K L ~ is e q u a l t o o n e . The Neumann series f o r t h e

r e s o l v e n t i s

which c o n v e r g e s i f a n d o n l y if 1 < 1 f o r a l l t h e

e i g e n v a l u e s . Each term i n t h e Neumann ser ies r e p r e s e n t s a

chain of n e u r o n s , t h e n - t h term c o r r a s p o n d i n g t o a pathway

w i t h n l i n k s . T h e p a t h w a y s w h i c h c o n t r i b u t e most

s i g n i f i c a n t l y t o Rab a r e the ones t h r o u g h neurons whose

average membrane p o t e n t i a l s a r e n e a r t h r e s h o l d .

The s t a b i l i t y of t h e s t e a d y - s t a t e s o l u t i o n is

d e t e r m i n e d by t h e l i n e a r i z e d t i m e - d e p e n d e n t e q u a t i o n

u h i c h i s a s y m p t o t i c a l l y s t a b l e if a n d o n l y if Re An < I f o r a l l t h ~ e i q e n v a l u e s . B i f u r c a t i o n s , t o new s o l u t i o n

\

b r a n c h e s ( B e r g e r , 1 9 7 7 ) can o c c u r o n l y when Re Art = f .

T h e second-order v a r i a t i o n s f r o m steady state

s a t i s f y

where

t h e solution of wh ich i s

F e a t u r e D e t e c t i o n

V i s i o n is the b e s t - s t u d i e d v e r t e b r a t e sensory s y stein.

T h e t r a n s f o r m a t i o n o f the v i s u a l field from t h e r e t i n a t o

cerebral cortex has been e x t e n s i v e l y s t u d i e d w i t h

e x t r a c e l l u l a r r e c o r d i n g s f r o m single n e u r o n s a n d w i t h

i n c r e a s i n g l y s c p h i s t i c a t e d a n a t o m i c a l t e c h n i q u e s (Cooke and

L i p k i n , 1 9 7 2 ; H u b e l a n d Riesel, 1977) .

I t was n o t a t f i r s t clear t h a t s i g n i f i c a n t p r o g r e s s

c o u l d b e made i n u n d e r s t a n d i n g the visual systen by

r e c o r d i n g f rom one n e u r o n a t a time, T h e k e y t o s u c c e s s

was, f i rs t , t h e use of s i m p l e v i s u a l stimuli w h i c h mimic

t h o s e o c c u r i n g u n d e r n a t u r a l c o n d i t i o n s , a n d s e c o n d , t h e

search f o r t h e most e f f e c t i v e s t i m u l u s f o r each c e l l . T h e

" t r igqer f e a t u r e " o f a n s u r o n i s d e f i n e d a s t h e s t i m u l u s

which p r o d u c e s t h e h i g h e s t average f i r i n g r a t e (Barlow and

Lev ick , 1965) , nany s u r p r i s i n g and r e m a r k a b l e r e s u l t s have

been f o u n d b y a p p l y i n g this a p p r o a c h t o e a c h s t a g e of visual

processing,

I n the vertebrate retina there are already s e v e r a l

l a y e r s of visual p r o c e s s i n g b e t v e s n t h e p h o t o r e c e p t o r

mosa ic , u h i c h t r a n a d u c e s p h o t o n s i n t o e l e c t r i c a l s i g n a l s ,

a n d t h e q a n q l i o r c e l l s , whose axons compose the o p t i c n e r v e .

A g a n g l i o n cell r e s p o n d s t o l i g h t o v e r a r e s t r i c t e d r e g i o n

o f t h e v i s u a l f i e l d c a l l e d i ts r e c e p t i v e f i e l d (Adr i an ,

7 9 2 8 ) . I n c a t s and monkeys t h e r e c e p t i v e f i e l d s of g a n g l i o n

ce l l s a r e c i r c u l a r a n d t h e c e l l s r e s p o n d best t o spots of

l i q h t . I n t h ~ r e t i n a s o f o t h e r v e r t e b r a t e s , s u c h a s f r o g s

and r a b b i t s , some " t r i g g e r f e a t u r e s w a r e nore coraplerc, s u c h

a s edqes o r NovGment i n a p a r t i c u l a r d i r e c t i o n .

A more r a d i c a l t r a n s f o r m a t i o n of t h e visual field

o c c u r s i n v i s u a l c o r t e x ( H u b e l and H i e s e l , 7 9 6 2 ) . Neurons

there are o n l y weakly i n f l u e n c e d by d i f f u s e i l l u m i n a t i o n and

s p o t s of l i q h t . flowever, a v i g o r o u s r e s p o n s e c a n u s u a l l y be

e l i c i t e d by u s i ~ g s u c h s t i m u l i a s e d g e s o r s l i t s of l i g h t in

3 p a r t i c u l a r o r i e n t a t i o n , and p e r h a p s moving i n a p a r t i c u l a r

d i r e c t i o n . I a p o r t a n t p a r a m e t e r s w h i c h d i s t i n g u i s h t h e

" t r i g g e ~ f e a t ures4I i n v i s u a l c o r t e x i n c l u d e p o s i t i o n , s i z e ,

o r i e n t a t i o n , d i r e c t i o n o f lilovement , o c c u l a r i t y , b i n o c u l a r

d i s p a r i t y , and c o l o r contrast. Noreover, a t l e a s t t w o of

these parame tars, o r i e n t a t i o n a n d o c c u l a r i t y , v a r y i n a

r e g u l a r p a t t e r n o v e r t h e c o r t i c a l s u r f a c e (Hubel and SJiesel,

1977) .

T h e I 1 t r i s g e r f e a t u r e s " become more c o m p l i c a t e d a t each

s u c c e s s i v e stage of v i s u a l p r o c e s s i n g , b u t i n a n o t h e r

respect t h e r e s F o n s e kecomes l e s s s p e c i f i c : the s i z e of t h e

r e c e p t i v e field i n c r e a s e s s o t h a t t h e p o s i t i o n o f t h e

" t r i g g e r f e a t u r e " becomes less i ~ p o t t a n t . I n the i n f e r i o r

t e m p o r a l c o r t e x , a n a r e a of t h e c e r e b r a l c o r t e x which is

i n p o r t a n t f o r v i s u a l l e a r n i n g , some n e u r o n s have e x t r e m e l y

s p e c i f i c " t r i q g e r f e a t u r e s M t o g e t h e r wi th r e c e p t i v e f i e l d s

that c o v e r much of the v i s u a l f i e l d {Gross , Rocha-Hiranda, &

Bender , 1 9 7 2 ) . T h e s e f i n d i n g s h a v e i n t r i g u i n g i m p l i c a t i o n s

for v i s u a l p e r c e p t i o n ( B a r l o v , 1972) , b u t t h e s o l e c o n c e r n

here is u i t h t h e o r i g i n o f t h e obse rved s p e c i f i c i t y and

i n v a r i a n c e i n t h e r e s p o n s e o f s i n g l e n e u r o n s .

Althouqh t h e c o n c e p t 0 f . a t t t r i g g e r f e a t u r e u has been

u s e f u l i n o r g a n i z i n g e x p e r i m e n t a l d a t a , some o f its

u s e f u l n e s s B a y b e due more t o the o r g a n i z i n g a b i l i t y o f t h e

e x p e r i m e n t e r t h a n to t h e d e s i g n o f t h e n e r v o u s sys tem. I n

t h e v i s u a l sys t em, f o r example, an i n f i n i t e number of

s p a t i a l and t e rnpora l p a t t e r n s o f l i g h t c o u l d b e used a s

s t i m u l i , b u t i r p r a c t i c e t h e r e s p o n s e o f a neuron i s t e s t e d

u i t h o n l y a s a a l l number o f s t i m u l i chosen b y t r a d i t i o n , A t

mos t , a n e x p e r i n e n t e r c a n c l a i m t h a t a neu ron responded b e s t

t o a " t r i g g e r f f a t u r e t l from a restricted class o f possible

s t i m u l i . T h u s , t h e response Bag b e aaxilrmu~ o n o n l y a

s u b s p a c e o f a l l p o s s i b l e v a r i a t i o n s a r o u n d t h e % r i g g e r

f e a t u r e n , and even i f t h e maximum were a t r u e l o c a l maximum,

i t m i q h t n o t b e a g l o b a l raaxiaum, w h i c h r e q u i r e s that all

p c s s i b l e s t i m u l i b e tested.

I n p r a c t i c e the " t r i g g e r f e a t u r e " of a n e u r o n is f o u n d

b y v a r y i n q a f e w p a r a m e t e r s of t h e s t i m u l u s t o optimize t h e

n e u r o n ' s f i r i n g r a t e . I f the r e s p o n s e of a neuron i s a t a

local raaximum t h e n s m a l l v a r i a t i o n s a r o u n d t h e o p t i m a l

s t i m u l u s should produce no v a r i a t i o n i n t h e a v e r a g e f i r i n g

r a t e t o f i r s t o rde r a n d on ly negative v a r i a t i o n s t o s e c o n d

o r d e r . These v a r i a t i o n a l c o n d i t i o n s w i l l b e a p p l i e d t o the

n o n l i n e a r n o d e l .

D e f i n e the l o c a l " t r i g g e r f e a t u r e r t a s the s e n s o r y input A .

kb ( 0 ) f o r which t h e r esponse of a n e u r o n i s maximoia

w h e r e t h e r e a l v a r i a b l e p a r a m e t e r i z e s a o n e - d i m e n s i o n a l

s u b s e t o f stimuli i n t h e s p a c e of a l l p o s s i b l e s t i m u l i . The

a v e r a g e men b r a n e p o t e n t i a l , a n d h e n c e t h e a v e r a g e f i r i n g

r a t e , i s a t a n e x t r e m u m i f and o n l y i f the f i r s t - o r d e r A /

v a r i a t i o n s v a n i s h $ a ( ~ ) 5. 0 . Prom E q . ( 1 4 ) t h i s extremum

c o n d i t i o n implies t h a t

where 6 is t h e subset of i n p u t s i n the recept ive field

o f t h e a-th n e u r o n .

The t y p e of e x t r e a u m depends of t h e s i g n o f t h e

second-order v a r i a t i o n , F roa E q ( 2 0 ) t h e r e is a maxiisuz~ if

a n d o n l y i f

A / A I/ w h e r e h b and k b are allowed t o v a r y a r b i t r a r i l y in

the receptive f i e l d o f t h e a - t h neuron, b E (Ba . The

f i r s t term on t h e r i q h t s i d e v a n i s h e s by v i r t u e of t h e

c o n d i t i o n on f i r s t - o r d e r v a r i a t i o n s ( 2 2 ) . The second term 4 /

o n the r i g h t d e p e n d s only on KC and o n l y i n d i r e c t l y

t h r o u q h a t l e a s t one n e u r o n o u t s i d e t h e r e c e p t i v e f i e l d

Prom Eq. ( 1 4 ) t h e s e c o n d - o r d e r c o n d i t i o n c a n be

w r i t t e n a s a g u a d ~ a t i c f o r m

where

L o c a l maxina a r e n o t ~ o s s i b l e i n l i n e a r m o d e l s since

I n terlas o f t h e e i g e n v e c t o r s o( ( b ) a n d t h e e i g e n v a l u e s

d b of Fka , t h e s e c o n d - o r d e r c o n d i t i o n c a n b e w r i t t e n

A ( b ) a t % ( z L h % x b R 0

A / f o r a l l UA i n t h e r e c e p t i v e f i e l d , f rom w h i c h it f o l l o w s

t h a t

Hence, the spectrum o f FtL must b e n o n p o s i t i r e i n t h e

r e c e p t i v e field f o r there t o be a l o c a l maximum, T h e

e i g e n v e c t o r s w i t h t h o l a r g e s t n e g a t i v e e i g e n v a l u e s a r e t h o s e

d i r e c t i o n s a l o n g which t h e r e s p o n s e o f t h e neuron is nost

s e n s i t i v e t o v a r i a t i o n s from the "trigger f e a t a r e s i . F o r

example , t h e o r i e n t a t i o n o f an e d g e p a r a ~ e t e r i z e s such a

d i r e c t i o n f o r most n e u r o n s i n v i s u a l c o r t e x , S i m i l a r l y , the

e i g e n v e c t o r s w i t h e i q e n v a i u e s n e a r z e r o g i v e t h e d i r e c t i o n s

v h i c h l e a v e t h e r e s p o n s e of t h e n e u r o n i n v a r i a n t .

T h e e x i s t i n g e x p e r i m e n t a l t e c h n i q u e s f o r tracing

f u n c t i o n a l c o u p l i n q s b e t ween n e u r o n s a r e d i f f i c u l t t o u s e

a n d are o f t e n ambiguous (Gerstein, 1 9 7 0 ) . Even i n the

b e s t - s t u d i e d p a r t s o f t h e b r a i n , such a s t h e c e r e b e l l u m

where t h s b a s i c w i r i n g diagram i s a l r e a d y knovn ('rSccles,

I t o , and ~ z e n t g ~ o t h a i , 1 9 6 7 ) . p r o g r e s s i n c o n s t r u c t i n g

r e a l i s t i c models is s l o g (Pelli o n i s z , ~ l i n ' a s , and P e r k e l ,

1977) . S imple models have b e e n p u t fo rward where less

s t r u c t u r a l i n f o r m a t i o n i s known, such a s t h e R u b e 1 and

Wiesel h i e r a r c h i c a l model ( 1 9 6 2 ) of c e l l types i n v i s u a l

c o r t e x . Although s u c h o v e r s i m p l i f i e d models a r e u n l i k e l y t o

~ u r v i v e , t h e y a t l e a s t s u n m a r i z e t h e d a t a and o f t e n

s t i m u l a t c l f u r t h e r a d v a n c e s , such a s the r e c e n t evidence for

p a r a l l e l v i s u a l processinq i n r e s p o n s e t o t h e h i e r a r c h i c a l

model (Dou, 1976) . The v a r i a t i o n a l a n a l y s i s o f " t r i g g e r

f e a t u r e s w g i v e n above m i g h t b e u s e f u l i n a n a l y z i n g these

h y p o t h e t i c a l m o d e l s .

These is a n a l t e r n a t i v e t o m o d e l i n g incomplete

structural d a t a when t h e f u n c t i o n o f a b r a i n area is known

o r p a r t i a l l y u n d e r s t o o d : i f a c o i a p u t a t i o n a l l y s u c c e s s f u l

m o d e l c a n be f o u n d f o r t h a t f u n c t i o n , t h e n c o n n e c t i o n s

b e t w e e n n e u r o n s m i g h t be p r e d i c t e d Srom t h e m o d e l . S u c h a

t e s t a b l e ~ o d e l i s examined i n t h e next s e c t i o n .

S t e r e o p s i s

T h e p e r c e p t i o n of d e p t h is of g r e a t b i o l o g i c a l

i m p o r t a n c e f o r many animals, s o it is n o t t o o s u r p r i s i n g

t h a t s e v e r a l i n d e p e n d e n t m e c h a n i s m s c a n be f o u n d i n t h e

human v i s u a l s y s t e m f c r c o m p u t i n g d e p t h ( G u l i c k a n d L a u s o n ,

1 9 7 6 ) . Art is t s h a v e d i s c o v e r e d a a n y m o n o c u l a r c u e s t o

d i s t a n c e , s u c h a s p e r s p e c t i v e , s h a d o w i n g , a n d v a r i a t i o n s i n

t e x t u r e , A n o t h e r m o n o c u l a r d e p t h c u e , d i s c o v e r e d by

L e o n a r d o , i s movement p e r s p e c t i v e : a s a n o b s e r v e r a o v e s his

h e a d , n e a r b y o b j e c t s p a s s more r a p i d l y a c r o s s the retina

t h a n m o r e d i s t a n t o b j e c t s ,

I n b i n o c u l a r d e ~ t h p e r c a p t i o n , o r s t e r e o p s i s , d e p t h is

c o m p u t e d f r o m t h e p a r a l l a x b e t w e e n t h e t w o e y e s . S t e r e o p s i s

i s i n d e p e n d e n t o f m o n o c u l a r c u e s , a s shown b y J u l e s z (7960)

who c o n s t r u c t e d t h e first r a n d o m - d o t s t e r e o g r a m , I n a

r a n d o m - d o t s t e r e o g r a m e a c h e y e sess a random f i e l d of d o t s ,

b u t t h e dots shcm t o t h e right e y e a re h o r i z o n t a l l y shifted

w i t h r e s ~ e c t t o t h e d o t s shown t o t h e l e f t eye: the amount

of s h i f t , o r b i n w n l a r d i s p a r i t y , is i n t e r p r e t e d b y t h e

v i s u a l s y s t e i a a s d e p t h v a r i a t i o n i n a s i n g l e "cyclopeanN

f i e l d of d o t s . When fused, t h e central d o t s i n t h e

random-dot s t e reogram shown i n Pig. 24 appear t o hover above

t h e p l a n e o f t h e paper even t h o u g h each eye o n l y sees a

s l i q h t l y d i f f e r f n t p a t t e r n of r andoa do t s ,

Since 1 9 5 8 a m a c h i n e has been commercial ly a v a i l a b l e ,

t h e Wild-Bay theon B8 s te reomat automated p l o t t e r , v h i c h can

draw a c o n t o u r nap from two ove r l app ing a e r i a l photographs.

T h e machine c o r r e l a t e s scan p a t h s from t h e two i laaqes and

a t t e m p t s t o a s s i q n a d e p t h t o each point. Unfo r tuna t e ly ,

s e r i o u s robl lens occu r owing t o l o c a l minima, and even

s p e c i a l t e chn iques d o n o t c o a p l e t e l y eliminate false

assignments 1Eor i , Kidode, & Asada, 1973) . V i e u i n g t h e random-dot stereoqram i n Fig. 24, most

peop le , after a d e l a y , expe r i ence a s h a r p t r a n s i t i o n f rom

t h e d i s o r d e r e d dot p a t t e r n t o a n o rdered image. Caref a1

e x p e r i m e n t s shcv t h a t f u s i o n of the image e x h i b i t s

h y s t e r e s i s : a l t h o u g h t h e binocular d i s p a r i t y must be less

t h an abou t 6' i n h u m a n s f o r f u s i o n t o occur , a f t e r f u s i o n

t h e d i s p a r i t y can b e increased to n e a r l y 2' before t h e

p e r c e i v e d image is l o s t ( F e n d e r and J u l e s z , 1 9 6 7 ) .

Ambiquous randorr-do t stereograms h a v e been c o n s t r u c t e d vh i ch

Fig. 24. When t h i s random-dot s t e r e o g r a m is

stereoscopically fused ( b y crossing o r everting the eyes or

b y using a prism o v e r one e y e ) , a square is seen hovering

over t h e random background (from J u l e s z , 1 9 7 1 ) .

can be p e r c e i v e d a s two d i f f e r e n t s u r f a c e s ; o n c e l o c k e d o n t o

o n e imaqe, most p e o p l e h a v e d i f f i c u l t y shifting their

p e r c e p t i o n t o t h e a l t e r n a t i v e image ( J u l e s z a n d J o h n s o n ,

1968) . J u l e s z (1 974) has e a p h a s i z e d t h a t t h e p r o p e r t i e s of

s t e r e o p s i s - such a s a s h a r p t r a n s i t i o n t o a n o r d e r e d

s t a t e , hysteresis, and m u l t i p l e s t a b l e s t a t e s - are

c h a r a c t e r i s t i c o f c o o p e r a t i v e phenomena. The term

" c o o p e r a t i v e 8 ' refers t o the way i n which local o p e r a t i o n s

n o n l i n e a r l y c o o p e r a t e i n f o r m i n g a g l o b a l o r d e r ; e x a m p l e s

c a n b e f o u n d i n many p h y s i c a l and b i o l o g i c a l s y s t e m s (Haken,

1 9 7 4 ) . Julesz (197 1, 1976) has p r o p o s e d a m e t a p h o r i c a l

raodel o f s t e r e o ~ s i s c o ~ p o s e d of s p r i n g s a n d magnets which

q u a l i t a t i w l y r e p r o d u c e s h i s s t e r e o p s i s d a t a a n d which

e x h i b i t s a l l t h e g e n e r a l c h a r a c t e r i s t i c s o f a cooperative

phenonenon.

A n o t h e r more b i o l o q i c a l l y - o r i e n t e d mode l h a s emerged

t h r o u g h the c o m t i n e d e f f o r t s o f several u o r k a r s [ J u l e s z ,

1 9 6 0 , 1 9 6 3 ; S ~ e r l i n q , 1970; Dev, 1975; N e l s o n , 1975; H a r r

ar,d P o q g i o , 1 9 7 6 ) . G i v e n b i n o c u l a r i m a g e s a s i n p u t , t h e

mode l attempts t o f i n d the b e s t g l o b a l t h r e e - d i m e n s i o n a l

r e c o n s t r u c t i o n b y p e r f o r m i n q many l o c a l c o m p u t a t i o n s i n

p a r a l l e l . T h e i n p u t t o the model i s a p a i r o f randota-dot

f i e l d s v h e r a e a c h p o i n t i n a d i s c r e t e field ( X , Y ) is

e i ther a one (vhite) o r a z e r o ( b l a c k ) , a s shown i n Fig. 24.

Lett y e Right y e

Dots in,right Dots in,'kft eye - rle-

%= Right eye %= Left w

F i q . 2 5 . Cep th l o c a l i z a t i o n b y the i d e n t i f i c a t i o n of

c o r t e s p o n d i n q d o t s in t h e r i g h t a n d l e f t eyes. Each eye

v i e w s f o u r d o t s f r o m which s i x t e e n c o r r e s p o n d e n c e s are

p o s s i b l e . T h e r e a r e s i x f a l s e d e p t h p l a n e s above and below

t h e true d e p t h p l a n e ( s o l i d squares). The d e p t h p l a n e can

be moved u p o r down b y increasing o r d e c r e a s i n g the r e l a t i v e

shift o f the d o t p o s i t i o n s , called the d i s p a r i t y , b e t w e e n

t h e two r e t i n a s ( f rom J u l e s z , 1971) .

T h e f i r s t step o f the model i s t h e computation of a

t h r e e - d i n s n s i a n a l disparity matrix, Dd ( X, V ) composed

of two-dimensional l a y e r s p a r a m e t e r i z e d b y d, Each layer

is the p o i n t w i s e p r o d u c t o f the left v i s u a l field and t h e

riqht v i s u a l field after a horizontal shift of d dot

T h e g e o m e t r i c r e l a t i o n s h i p between h o r i z o n t a l s h i f t a n d

d i s p a r i t y i s i l l u s t r a t e d i n F ig . 25.

H i t h i n t h e three-dimensional disparity matrix, which '

r e p r e s e n t s an i n t e r n a l r e p r e s e n t a t i o n of space i n the f i e l d

of v i e w , a two-d imens iona l surface a u s t be found

r e p r e s e n t i n g the boundaries o f objects i n t h a t space , The

model proposed b y Dev (1975) a n d Nelson (1975) f o r

pe r fo rming t h i s computa t ion c o n s i s t s o f stacked

two-dimens iona l layers, s i a i l a r t o t h e d i s p a r i t y m a t r i x ,

w i t h e x c i t a t o r y c o n n e c t i o n s b e t lieen nearby u n i t s within the

same lays,r, a rd i n h i b i t o r y i n t e r c o n n e c t i o n s between u n i t s in

nearby layers, a s schematically shown i n F i g . 26. Hasr and

Poqqio (1 9 7 6 ) i m p l e i x e n t e d t h i s c o m p u t a t i o n a l scheme with a n

i t e r a t i ve a l g c r i t h a

where is a s i g m o i d f u n c t i o n , D&(x,Y) is t h e input (n)

disparity matrix d e f i n e d above, C d ( x , v ) is t h e s t a t e of

Fig . 2 6 . a ) O n a - d i m e n s i o n a l images i n t h e r i g h t retina

TiX and l e f t r e t i n a f -x a r e p l o t t e d a s a two-d imens iona l

q raph . L i n e s of c o n s t a n t d i s p a r i t y r u n d i a g o n a l l y ( d a s h e d

l i n e s ) . b ) T h e e x p a n d e d - s c a l e i n s e t shows t h e l o c a l

c o m p u t a t i o n scheme a t a s i n g l e p o i n t on t h e g r a p h : n e a r b y

p o i n t s of t h e same d i s p a r i t y (dashed l i n e ) h a v e e x c i t a t o r y

(+) i n t e r c o n n e c t i o n s , while nea rby p o i n t s w i t h different

d i s p 3 r i t y ( s o l i d l i n e s ) h a v e i n h i b i t o r y ( - ) i n t e r c o n n e c t i o n s

( f rom Harr a n d Pogqio , 1 9 7 6 ) .

t h e u n i t a t p o s i t i o n ( % , * I ) and d i s p a r i t y & a t i t e r a t i o n

% is tho i n h i b i t i o n c o n s t a n t , a n d S' and 0' a r e

t h e s u b s e t s of (d: x: Y ' ) n e a r ( d,x, Y ) r e p r e s e n t e d

r e s p e c t i v e l y by the d a s h e d [ d = d 8 ) a n d t h i c k ( d f d ' )

n e i q h b o r h o o d s a p p e a r i n g i n the inset of Big. 2 6 ,

B e c a u s e o f t h e e x c i t a t o r y c o u p l i n g within each layer,

n e i q h b o r i n g u n i t s w i l l t a c o o p e r a t e t l w i t h e a c h other.

However, tecause of i n h i b i t o r y M c o m p e t i t i o n l ~ b e t w e e n l a y e r s ,

the d o m i n a n t l a y e r w i l l suppress n e i g h b o r i n g l a y e r s . T h u s ,

t h e d i s p a r i t y a l q o r i t h m a t t e m p t s t o c o n v e r t t h e raw

d i s p a r i t y i n p u t i n t o a c o n s i s t e n t d e p t h a s s i g n m e n t - defined a s t h e m a x i m a l l y e x c i t e d l a y e r - a t each p o i n t ,

J u l e s z (1960) was t h e f i r s t t o u s e d i s p a r i t y l a y e r s i n

a mode l of s t e r e o p s i s , b u t h e s u g g e s t e d t a k i n g p o i n t v i s e

d i f f e r e n c e s r a t h e r t h a n p o i n t v i s e p r o d u c t s i n t h e

c o a p u t a t i o n of t h e d i s p a r i t y natrix . The t h r e s h o l d o f t h e

s i q m o i d i n t h e d i s p a r i t y a l g o r i t h m (25) s u p p r e s s e s weak

b a c k q r o u n d 'lnciseIt, a t e c h n i q u e u h i c h Julesz ( 1963)

e x p l o i t e d i n a n e a r l y model. Dev (1975) a n d Nelson (1975)

c o n t r i b u t e d t h e e x c i t a t o r y and i n h i b i t o r y c o m p u t a t i o n a l

scheme. Harr a n d P o q g i o (1976) were t h e first t o i n t r o d u c e

t h e n o n l i n e a r i t y r e q u i r e d f o r successful c o n v e r g e n c e , a s

d e m o n s t r a t e d i n F i q . 2 7 .

Marc a n d P o g q i o d i d n o t g i v e a p h y s i o l o g i c a l

i n t e r p r e t a t i o n f o r t h e i r a l q o r i t h m , a l t h o u g h i m p l i c i t i n

Fiq. 27. S t a r t i n q w i t h t h e random-dot stereogram shown

at t h e t o p , t h e d i s p a r i t y a l g o r i t h m of Karr'and Poggio

c o n v e r q e s t o t h e image p e r c e i v e d by humans: a rectangle

s t a n d i n q i n front cf the page. The size of each dot is

p r o p o r t i o n a l to t h e d i s p a r i t y or d e p t h a s s i g n e d to each

p o i n t ( f rom Marr a n d Poggio, 1 9 7 7 ) .

t h e i r work a n d i n that of t h e i r predecessors is t h e

a s s l a m p t i o n that " u n i t s v c o u l d r e p r e s e n t n e u r o n s . Several

i m p o r t a n t q u e s t i o n s r e m a i n o p e n : Are t h e terms i n t h e

d i s p a r i t y a l g o r i t h m r e l a t e d t o d i r e c t l y m e a s u r a b l e p h y s i c a l

q u a n t i t i e s o r some p o p u l a t i o n average? Bow is the

d i s c r e t e - t i m e c o m p u t a t i o n r e l a t e d t o t h e c o n t i a n o u s - t i m e

c a m p u t a t i o n i n the n e r v o u s system?

T h e r e i s a c l o s e r e l a t i o n s h i p b e t w e e n t h e d i s p a r i t y

a l q o r i t h m ( 2 5 ) and t h e p h y s i o l o g i c a l l y - m o t i v a t e d model being

s t n d i e d h e r e , L e t a s i d e n t i f y s o l u t i o n s o f t h e algorithm

w i t h a v e r a q e firing r a t e s

Then a f i x e d - p o i n t s o l u t i o n of the d i s p a r i t y a l g o r i t h a can

b e w r i t t a n

T h i s e q u a t i o n i s i d e n t i c a l t o t h e s t e a d y - s t a t e e q u a t i o n (11)

f o r t h e a v e r a g e f i r i n g r a t e s , uith t h e s u b s c r i p t &

indexing ( d , x , y ) , Dd ( x , Y ) i d e n t i f i d w i t h t h e i n p u t s A

t h e s i g m i d 5 i d e n t i f i e d w i t h t h e a v e r a g e 4

f i r i n g rate R,( #,) a s a f u n c t i o n o f t h e a v e r a g e metabrane

p o t e n t i a l ( F i q . 20), a n d t h e e x c i t a t o r y a n d i n h i b i t o r y d! X'Y'

q e o m e t r y i d e n t i f i e d w i t h the c o u p l i n q s t r e n g t h s K d x Y . If t h e ' f u n i t s " i n t h e d i s p a r i t y a l g o r i t h m (25) a r e

i d e n t i f i e d v i t h n e u r o n s a n d t h e s t a t e of a un i t is

i d e n t i f i e d with a n e u r o n ' s a v e r a g e f i r i n g r a t e , then t h e

model of s t e r e o p s i s p r e d i c t s t h a t (i) d i s p a r i t y - s e n s i t i v e

n e u r o n s s h o u l d be o r g a n i z e d i n l a y e r s o r f l c o l n ~ n s v o f

c o n s t a n t d i s p a r i t y , (ii) n e i g h b o r i n g n e u r o n s w i t h i n t h e same

d i s p a r i t y l a y e r s h o u l d have e x c i t a t o r y i n t e r c o n n e c t i o n s , and

(iii) n e i q h b o r i n q neurons i n d i f f e r e n t disparity l a y e r s

s h o u l d h a v a i n h i b i t o r y i n t e r c o n n e c t i o n s .

T h r e e consEquences f o l l o w frora i d e n t i f y i n g t h e neuro n a l

network rxodel w i t h t h e n o d e l o f s t e r o p s i s : f i r s t , t h e

m a t h e m a t i c a l a n a l y s i s o f t h e st e a d y - s t a t e e q u a t i o n given

h e r e a p p l i e s e q u a l l y well t o f i x e d - p o i n t s o l u t i o n s o f t h e

d i s p a r i t y a l q o r i t h a ; s econd , % h e n o n p h y s i o l o g i c a l

d i s c r e t e - t i n e a l q o r i t h n c a n be n a t u r a l 1 y generalized t o a

p h y s i o l o g i c a l c o n t i n u o u s - t i m e model; a n d t h i r d , t e s t a b l e

p r e d i c t i o n s c a n be made a b o u t the c o m p u t a t i o n o f s t e r e o p s i s

i n t h e vertebrate n e r v o u s sys t em, Each of t h e s e

c o n s e g u e n c z s will b e d i s c u s s e d i n turn.

The p r o p e r t i e s of t h e s t e a d y - s t a t e e q u a t i o n are

precisely t hcse t h a t a r e c h a r a c t e r i s t i c of c o o p e r a t i v e

phenomena. A proof is g i v e n i n t h e above section o n

s t e a d y - s t a t e solutions t h a t t h e e q u a t i o n a l w a y s has a t l e a s t

one s o l u t i o n and a s u f f i c i e n t c ' o n d i t i o n is given f a r t h e r e

t a e x i s t a un ique s o l u t i o n . A s i m p l e model is a n a l y z e d

which d e m o n s t r a t e s n u l t i p l e s t a b l e s t a t e s a n d h y s t e r e s i s

between t h e m .

A l t h o u g h t h e d i s c r e t e t i m e disparity a l g o r i t h m worts

well o n s t a t i c r andom-dot s t e r e o g r a m , t h e a l g ~ s i t h m is less

s u c c e s s f u l on d y n a a i c random-do t s t e r e o g r a m s , which huaans

p e rcei ve a s iaoving surfaces ( J u l e s z , p e r s o n a l

c o m m u n i c a t i o n ) . T h e p r o b l e m p r o b a b l y a r i ses because t h e

a l g o r i t h a d c e s n o t t a k e i n t o a c c o u n t t e m p o r a l i n t e g r a t i o n .

A p h y s i o l o g i c a l l y n a t u r a l c o n t i n u o u s - t i m e g e n e r a l i z a t i o n o f

t h e d i s c r e t e - t i m e a l g o r i t h m i s s u g g e s t e d b y t h e p r e s e n t

model: t h e s t e a d y - s t a t e e q u a t i o n (1 I ) , which is i d e n t i c a l

w i t h f i x e d - p o i n t s o l u t i o n s o f t h e d i s p a r i t y a l g o r i t h m , i s a

s p e c i a l c a s e of t h e t i n e - d e p e n d e n t e q u a t i o n (6 ) f o r t h e

a v e r a g e membrane p o t e n t i a l s

T h i s t i m e - d e p e n d e n t mode l h a s the same i n p u t s a n d c o u p l i n g

s t r e n q t h s a s the s t e a d y - s t a t e m o d e l , b u t t h e a v e r a g e

nembrane p o t e n t i a l r a t h e r t h a n t h e a v e r a g e f i r i n g r a t e is

t h e p r i m a r y v a r i a b l e a n d temporal i n t e g r a t i o n is p r o v i d e d by

t h e term w i t h t h e membrane time c o n s t a n t . The a v e r a g e

f i r i n q r a t e is related t o the a v e r a g e membrane p o t e n t i a l b y h A = ( ) , a s s h o w n i n Pig. 20. A l t h o u q h these a r e

e q u i v a l e n t v a r i a b l e s , the p h y s i o l o g i c a l l y n a t u r a l

c o n t i n u o u s - t i m e g e n e r a l i z a t i o n of t h e d i s p a r i t y a l g o r i t h m

f a v o r s t h e a v e r a q e membrane p o t e n t i a l a s t h e p r i m a r y

v a r i a b l e .

I n f o r m a t i o n f r o m t h e two eyes c o n v e r g e s f o r t h e f i r s t

tirue i n v i s u a l c o r t e x area 17, and s i n g l e n e u r o n s sensitive

t o b i n o c u l a r d i s p a r i t y have been f o u n d i n area 17 of t h e cat

{ B i s h o p , 1 9 7 0 ) a n d a r ea s 17 a n d 18 i n the monkey ( H n b e l and

Wiesel, 1 9 7 3 ; Poqqia a n d P i s c h e r , 1977). T h e n o d e l of

s t e r e o p s i s w i t h t h e p h y s i o l o g i c a1 i n t e r p r e t a t i o n given h e r e

is e x p e r i m e n t a l l y t e s t a b l e . T h e f u n c t i o n a l c o u p l i n g s

b e t v e e n d i s p a r i t y - s e n s i t i v e n e u r o n s c o u l d b e m e a s u r e d by

r e c o r d i n g s i m u l t a n e o u s l y f r o m p a i r s of n e i g h b o r i n g n e u r o n s

a n d c r o s s - c o x r e l a t i n g t h e i r s p i k e t r a i n s ( G e r s t e i n , ' 1970) . D i s p a r i t y - s e n s i t i v e n e u r o n s h a v e t h u s f a r b e e n s t u d i e d

w i t h s p o t s a n d slits o f l i g h t ; i n v i e u of Juleszgs work and

t h e e v i d e n c e f o r c o o p e r a t i v i t y , more c o m p l e x spatial

p a t t e r n s , i n c l u d i n g r a n d o m - d o t s t e r e o g r a m s , m i g h t be

v a l u a b l e s t i m u l i t o u s e while r e c o r d i n g from

d i s p a r i t y - s e n s i t i v e a r e a s o f v i s u a l c o r t e x . Random-dot

s t e r e o p s i s may n o t o c c u r i n a n a n s s t h e t i z e d monkey, b u t

monkeys can be trained t o perceive d e p t h i n a r a n d o m - d o t

s t e r e o q r a m (Elough, 1970) a n d r e c o r d i n g s can b e made from

awake monkey t r a i n e d t o f i x a t e on t h e s t e r e o g r a m ( P o g g i o a n d

F i s c h e r , 1 9 7 7 ) .

111. C o r r e l a t i o n s Be tween Hembrane P o t e n t i a l s

Bost s e n s o r y n e u r o n s i n t h e v e r t e b r a t e nervous s y s t e m

~ a i n t a i n c o n t i n u a l i m p u l s e f i r i n g i n t h e a b s e n c e of sensory

s t i m u l a t i o n , a f e a t u r e which is u s u a l l y c a l l e d s p o n t a n e o u s

a c t i v i t y ( G r a n i t , 1955) . I n t h e v i s u a l s p s t e t a , f o r e x a a p l e ,

r e t i n a l q a n q l i o n ce l l s i n c o m p l e t e d a r k n e s s h a v e s p o n t a n e o u s

a c t i v i t y w h i s h ray increase, decrease, o r r e m a i n u n c h a n g e d

when t h e r e t i n a i s e x p o s e d t o a s t e a d y b a c k g r o u n d l e v e l o f

i l l u m i n a t i o n ( K u f f l e r , P i t z H u q h , C Barlow, 1957) . Numerous

i n v e s t i g a t o r s have s t u d i e d s p o n t a n e o u s a c t i v i t y and have

a t t e m p t e d t o d e t e r m i n e i t s o r i g i n ( L e v i c k , 1973) . Some

i n v e s t i g a t o r s r e q a r d s p o n t a n e o u s a c t i v i t y a s i n h e r e n t n o i s e

i n t h e n e z v o u s s y s t e m ; o t h e r s v i e w it a s a c a r r i e r w h i c h

a l l o w s f o r i n h i b i t o r y a s we11 a s e x c i t a t o r y m o d u l a t i o n .

R e g a r d l e s s o f s p e c u l a t i o n , s p o n t a n e o u s a c t i v i t y i s c l e a r l y

a n i m p o r t a n t Thenoinenon and may p r o v i d e a c l u e t o t h e

o p e r a t i o n of t h e n e r v o u s system.

I n f o r m a t i o r p r o c e s s i n g b y a v e r a g e f i r i n g r a t e s was

d i s c u s s e d i n the p r e v i o u s p a r t . F u r t h e r i n f o r m a t i o n c o u l d

b e c o d e d i n t h e t e m p o r a l p a t t e r n o f the i m p u l s e s , s u c h a s i n

s p i k e - t r a i n c o r r e l a t i o n s ( P e r k e l a n d B u l l o c k , 1968;

G e r s t a i n , 1 9 7 9 ) . However, a n i m p u l s e - p r o d u c i n g n e u r o n is

n o t a n i d ~ ? a l d e v i c e f o r p r o c e s s i n g t e m p o r a l l y - c o d e d

i n f o r m a t i o n b e c a u s e t h i s t y p e of n e u r o n is m a x i m a l l y

s e n s i t i v e t o the timing of a n i n c o m i n g i m p u l s e o n l y vhon its

membrane ~ o t e n t i a l i s j u s t b e l o w t h r e s h o l d . One might

therefore e x p e c t t h a t a n e u r o n v h i c h p r o c e s s e s c o r r e l a t i o n s

v o o l d o p e r a t s nea r t h r e s h o l d m o s t o f the t i n e and, as a

c o n s e q u e n c e , m a i n t a i n a m o d e r a t e r a t e o f f i r i n g . Thus,

s p o n t a n e o u s a c t i v i t y i n a n e u r o n may reflect a state of

s e n s i t i v i t y t o t e m p o r a l l y - c o d e d i n f o r m a t i o n and a r e a d i n e s s

t o r e s p o n d t o i n p u t c o r r e l a t i o n s .

C o r r e l a t i o n s

As a n e x p e r i m e n t a l t e c h n i q u e , a u t o c o r r e l a t i o n c a n be

u s e d t o d e t e c t s e r i a l d e p e n d e n c e i n a s i n g l e s p i k e t r a i n ,

s u c h a s a t e n d e n c y f o r s p i k e s t o occur a t p r e f e r r e d

i n t e r v a l s , a n d c r o s s - c o r r e l a t i o n b e t w e e n a p a i r o f s p i k e

t r a i n s c a n be u s e d t o d e t e c t m u t u a l d e p e n d e n c e , s u c h a s

i n t e r a c t i o n o r a common influence (Moore, P e r k e l , E S e g u n d o ,

1966 ; Gerstein, 1970) . T h e c o r r e l a t i o n is c a l l e d a

s e c o n d - o r d s r p r o p e r t y because i t d e p e n d s o n two e v e n t s , i n

c o n t r a s t t o t h e nean which is a f i r s t - o r d e r property.

L e t f (t) and 9 it) r e p r e s e n t t u o s i m u l t a n e o u s l y

m e a s u r e d s t a t i o n a r y s i g n a l s , s u c h a s i n t r a c e l l u l a r

r e c o r d i n q s f r o m a p a i r o f n e u r o n s or e x t r a c e l l u l a r

recordings of a p a i r o f s p i k e t r a i n s , T h e t i m e - a v e r a g e

c o r r e l a t i o n b e t w e e n f( t \ and 3 ct) o v e r tiee is

d e f i n e d a s T

T h e c o r r e l a t i c n d e p e n d s on a d e l a y p a r a m e t e r LC which is

the shift b e t w e e n t h e two s i g n a l s i n t h e p r o d u c t . If

f = 9 t h e n C T (k) is called t h e a u t o c o r r e l a t i o n a n d

C T ( Q ) i s c a l l e d t h e v a r i a n c e .

Now c o n s i d e r an ensemble o f s i m u l t a n e o u s l y m e a s u r e d

p a i r s o f s i g n a l s , Cf,(t)} a n d { g , ~ ~ l ) , from N i n d e p e n d e n t t r i a l s u n d e r u n i f o r m c o n d i t i o n s , The

e n s e m b l e - a v e r a g e correlation i s d e f i n e d a s

I n q e n e r a l t h e ensemble-average c o r r e l a t i o n d e p e n d s on b o t h

times a n d S , h u t i f t h e s i g n a l s a r e s t a t i o n a r y then

6, t , S ) d e p e n d s o n l y on t h e d i f f e r e n c e L L = s - ~ . Under c n r t a i n e r q o d i c c o n d i t i o n s (Halmos, 1956) t h e

a p p r o p r i a t e l i ~ i t s o f t h e two t y p e s of c o r r e l a t i o n a re

equal:

T h e c o r r e l a t i o n d e p e n d s o n t h e means vhich a r e

f i r s t - o r d e r properties. By s u b t r a c t i n g o f f t h e d e p e n d e n c e

cf t h e means, a n o r n a l i z e d c o r r e l a t i o n can b e d e f i n e d ,

called t h e cova r i ance , which d s p e n d s o n l y on s e c o n d - o r d e r

propert ies:

A A

where $ (t) and 5 ( s ) a r e the e n s e a b l e - a v e r a g e aeans

d e f i n e d i n the p r e v i o u s p a r t . Onder s t a t i o n a r y c o n d i t i o n s

t h e c o v a r i a n c e d i f f e r s from t h e c o r r e l a t i o n o n l y b y an

a d d i t i v e c o n s t a n t , b u t f o r t i w e v a r y i n g signals t h e

c o r r e l a t i o n may be s i g n i f i c a n t 1 y i n f l u e n c e d by the

t i m e - v a r yinq means,

T h e c o v a r i a n c e i s a measu re of t h e j o i n t r e l a t i o n s h i p

between two e v e n t s r e l a t i v e t o t h e i r chance o r a c c i d e n t a l

r a t e o f c o i n c i d e n c e . A p o s i t i v e c o v a r i a n c e Beans t h a t t h e

e v e n t s o c c u r t o g e t h e r more o f t e n t h a t by c h a n c e , and a

n e g a t i v e c o v a r i a n c e means that t h e e v e n t s occur together

l ess o f t e n than by chance. If t h e two e v e n t s are

i n d e p e n d e x t , so t h a t they o c c u r t o g e t h e r e n t i r e l y b y c h a n c e ,

t h e n t h e i r c o v a r i a n c e i s z e r o . Although c o v a r i a n c e i s a

measure o f t h e t e m p o r a l p a t t e r n i n a s i g n a l , i t r e p r e s e n t s

o n l y a s m a l l f r a c t i o n of a l l the i n f o r m a t i o n t h a t c o u l d be

c o n t a i n e d i n the d e t a i l e d t i m i n g o f each i ~ a p u l s e o r i n t h e

fine s t r u c t u r e of t h e membrane p o t e n t i a l v a r i a t i o n s ,

However, the presence of c o v a r i a n c e i n a s i g n a l i n d i c a t e s

that t h e s y s t e m a t l e a s t has a v a i l a b l e t o it

t e m p o r a l l y - c o d e d i n f o r m a t i o n beyond the t e m p o r a l l y - v a r y i n g

nean v a l u e o f t h e s i g n a l . A s summarized b e l o w , there is i n

f a c t some i n d i r e c t e v i d e n c e f o r t h e s e n s o r y c o d i n g and t h e

u s e of t e ~ p o r a l l y - c o d e d i n f o r n a t i o n i n t h e a u d i t o r y ,

s o m a t o s e n s o r y , and v i s u a l s y s t e m s ,

I n t h e a u d i t o r y s y s t e m , p h a s e i n f o r m a t i o n i n t h e

phase- locked a c t i o n p o t e n t i a l s f rom t h e two e a r s is used for

s p a t i a l l o c a l i z a t i o n a t f r e q u e n c i e s below 9 K H z [Jeffress,

1975; K o n i s h i , 1977) . Neurons h a v e been found in t h e

a u d i t o r y s y s t e m o f t h e owl with a s p a t i a l r e s o l u t i o n of 5O,

which c o r r e s p o n d s t o a t i n e d e l a y between t h e ovlls e a r s of

less t h a n 10 mic roseconds (Knudsen, Kon i sh i , G P e t t i g r e u ,

1977).

I n t h e s o m a t o s e n s o r y s y s t e m i n f o r m a t i o n c o d e d as

i n t e r s p i k e i n t e r v a l s c a n a p p a r e n t l y b e used f o r p e r c e i v i n g

the f r e q u e n c y cf f l u t t e r - v i b r a t i o n ( B o u n t c a s t l e , 1967) , and

some c e n t r a l n e u r o n s f i r e v i th preferred i n t e r s p i k e

intervals e v e n u n d e r normal s t i m u l a t i o n of t h e touch

r e c e p t o r s [Amassian and G i b l i n , 1974) . I n the v i s u a l system, random-dot s t e r e o g r a m s , w h i c h

have a random t f x t u r e when viewed monocu la r ly , a r e p e r c e i v e d

i n d e p t h i f t h e d o t s a r e b i n o c u l a r l y c o r r e l a t e d , e i t h e r

s p a t i a l l y ( Ju le sz , 1971) o r t e m p o r a l l y (Ross, 1 9 7 4 ) . Color

c a n b e p e r c e i v e d i n a t e m p o r a l l y v a r y i n g s p o t of w h i t e

l i g h t ; t h e a p p a r e n t c o l o r depends o n t h e t e m p o r a l p a t t e r n of

t h e i n t e n s i t y m o d u l a t i o n ( P e s t i n g e r , A l l y n , and ' i ih i te ,

1971) .

S i r a u l t a n e d u s r e c o r d i n g s of s p i k e t r a i n s h a v e been

o b t a i n e d from p a i r s o f neurons i n t h e r e t i n a (Rodiek , 1967) ,

t h e l a t e r a l g a n i c u l a t e nucleus ( S t e v e n s a n d G e r s t e i n , 1 9 7 6 ) ,

the c e r e b e l l u m { B e l l and Kawasaki, 1972), the a u d i t o r y

cortex [ D i c k s a n and Gerstein, 1972) , a n d e l s e v h e r e . In nany

cases t h e spike trains were si q n i f i c a n t l y c o r r e l a t e d . Prom

t h e present p e r s p e c t i v e , t h e quest ion o f whether t h e

c o r r e l a t i o n s were p r o d u c e d by direct i n t e r a c t i o n o r a common

input i s s e c o n d a r y t o t h e q u e s t i o n o f whether l a r g e - s c a l e

c o r r e l a t i o n s e x i s t and a re r e l a t e d t o sensory process ing .

When the mesabrane p o t e n t i a l o f a n e u r o n is b e l o w t h e

t h r e s h o l d f o r p r o d u c i n g i i s p u l s e s , c o r r e l a t i o n s i n its

membrane p o t e n t i a l c a n n o t c o n t r i b u t e t o c o r r e l a t i o n s i n its

spike t r a i n ; hence, c o r r e l a t i o n s b e t w e e n membrane p o t e n t i a l s

s h o u l d b e a t least a s p r o r a i n e n t a s c o r r e l a t i o n s m e a s u r e d

b e t w e e n spike t r a i n s .

Although t h e e n s e r c b l e a v e r a g e o f the f i r i n g r a t e ,

called t h e PST, i s a w e l l - e s t a b l i s h e d e x p e r i m e n t a l

t e c h n i q u e , ensemble-av~rage c o r r e l a t i o n has not y e t been

e x p l o i t e d . T i m e - a v e r a q e c o r r e l a t i o n i s r e s t r i c t e d t o

s t a t i o n a r y c o n d i t i o n s , v h i c h d o e s n o t a l l o w the s t i m u l u s t o

vary w i t h t i m e . More i n t e r e s t i n g r e s u l t s s h o u l d be f o u n d i n

n o n s t a t i o n a r y responses t o s t i m u l i f o r w h i c h the

ensemble -average c o r r e l a t i o n i s t h e n a t u r a l t e c h n i q u e . One

aim of this part is t o examine how t h e nervous system might

u t i l i z e c o r r e l a t i o n s in p r o c e s s i n g s e n s o r y information.

C o v a r i a n c e E q u a t i o n

T h e c o v a r i a n c e between t h e e f f e c t i v e membrane

p o t e n t i a l s is def ined a s

and b y v i r t u e o f the n o n l i n e a r model i n E q . (2) t h e

c o v a r i a n c e s a t i s f i e s

In the monkey's c e r e b r a l c o r t e x there a r e a p p r o x i a a t e l y

I O ' O n e u r o n s a n d e a c h neuron t y p i c a l l y receives

c o n n e c t i o n s f rom 10' o t h e r s . T h e l a r g e number o f neurons

and t h e i r h i g h degree of connectivity will prove h e l p f u l i n

a n a l y z i n q these n o n l i n e a r e q u a t i o n s .

An u n e x p e c t e d s i m p l i f i c a t i o n o c c u r s i f a physically

reasonable a s s u m p t i o n is made c o n c e r n i n g t h e p r o b a b i l i t y

d i s t r i b u t i o n af t h e membrane p o t e n t i a l s (Se jnowsk i , 1976b) . By the central l i m i t t h e o r e a t h e sum of a l a r g e number of

i n d e p e n d e n t random i n p u t s h a s , unde r q u i t e general

c o n d i t i o n s , a G a u s s i a n d i s t r i b u t i o n . T h e membrane

p o t e n t i a l , which d e p e n d s on an e x t r e m e l y l a r g e number of

s y n a p t i c e v e n t s a l o n g many i n p u t s , m i g h t be a p p r o x i m a t e l y

G a u s s i a n . W i t h t h i s a s s u m p t i o n , t h e n o n l i n e a r terms i n

Eq. ( 28 ) s i q n i f i c a n t l y s i m p l i f y b y t h e following:

Theorem, (Bussganq, 1952) . I f X and Y are

jointly Gaussian random variables, and p is any function

such t h a t p ( Y ) i s a w e l l - d e f i n e d r andon v a r i a b l e w i t h

f i n i t e s e c o n d - o r d e r a o ~ e n t s , then

where

If the function p ( y ) is c o n s i d e r e d a n o n l i n e a r

t r a n s f o r m a t i o n of a n i n p n t , t h e n Bussganggs theorem

s t a t e s t h a t the o u t p u t c o v a r i a n c e COV ( X , p ( y ) ) with an

a r b i t r a r y Gaussian r a n d o m v a r i a b l e Y is p r o p o r t i o n a l t o

t h e i n p u t c o v a r i a n c e C ov ( X, y ) . Bussgang's theoren is

useful f o r es t i x a t i n g t h e c h a r a c t e r i s t i c s of a nonlinear

system ( G e l b , 1974; Haddad, 1975) and h a s been a p p l i e d t o

the n o n l i n e a r i t y i n the g o l d f i s h r e t i n a ( S p e k r e i jse, 1969;

S p e k r e i j s e a n d Oos t ing , 1970) .

A s s u s i n g t h a t #a (t ) are Gaussian and a p p l y i n g t h i s

theorem to E q . ( 2 8 ) , we f i n d t h a t t h e dif ferences A

( - ) h a v e the same j o i n t d i s t r i b u t i o n as

w h e r e are Gaussian p r o c e s s e s hav ing zero mean and I

t h e s a a e c o v a r i a n c e a s 7 b l ? ) , and Aab = K&, - J a b ,

where dnb i; the K r o n e c k e r d e l t a a n d

with ~ / b ( $ a ) a s d e f i n e d i n the above theorem. quat ti on

(291, w h i c h d e t e r n i n e s t h e c o v a r i a n c e of $:(t) and rill

be c a l l e d t h e c o v a r i a n c e eqna t i o n , r e s e n b l e s the l i n e a r

mode l f o r graded e l e c t r i c a l c o u p l i n g i n Eg. ( I ) , with t h e /

i n t e r a c t i o n n a t r i x Kab p l a y i n g t h e r o l e of t h e linear

c o u p l i n g c o e f f i c i e n t s . T h e a e l t i p l i c a t i v e w e i g h t s Rb ( $ b )

a p p e a r i n g i n t h e s e e f f e c t i v e c o u p l i n g s t r e n g t h s d e p e n d o n

the membrane p o t e n t i a l s , a s shown i n Pig. 20. Those n e u r o n s

w i t h a v e r a g e melrhrane p o t e n t i a l s n e a r threshold a n d t h o s e

c o n n e c t i o n s b e t w e e n such c r i t i c a l neurons c o n t r i b u t e most

e f f e c t i v e l y to the covariance e q u a t i o n .

Despite the linear f o r m o f t h e c o v a r i a n c e e q u a t i o n , t h e

coupled e q u a t i o n s f o r the means and c o v a r i a n c e s are, of

c o u r s e , n o n l i n e a r . Because only the mean a n d c o v a r i a n c e of

a G a u s s i a n process a r e i n d e p e n d e n t , t h e e q u a t i o n s f o r

h i g h e r - o r d e r rnorrents c o n t r i b u t e no new i n f o r m a t i o n . If the

membrane potentials a r e i n d e e d G a u s s i a n , t h e n o n l y a small

p a r t o f a l l t h e d e t a i l e d t i m i n g i n f o r m a t i o n i n i n p u t s p i k e

t r a i n s i s a v a i l a b l e f o r p r o c e s s i n g b y the membrane

p o t e n t i a l s . Hence , meabrane p o t e n t i a l c o v a r i a n c e c o u l d be

an i m p o r t a n t form o f temporal c o d i n g i n t h e c e n t r a l n e r v o u s

s y s t e m .

The a s s u m p t i o n t h a t t h e effective membrane potentials

a r e Gaussian c a n be t e s t e d e x p e r i m e n t a l l y a n d enters a t t h e

s a n e l e v d as t h e a s s u n p t i o n s w h i c h l e d t o t h e n o n l i n e a r

model. Membrane p o t e n t i a l s a r e l o s t l i k e l y t o b e Gaussian

i n a h i g h l y i n t e r c o n n e c t e d area s u c h a s cerebral c o r t e x .

D e p a r t u r e s from a G a u s s i a n d i s t r i b u t i o n may o c c u r i n a

n e u r o n which is d o m i n a t e d by a few i n p u t s . However, e v e n i n

a n e u r o n with only o n e i n p u t the nembrane p o t e n t i a l in igh t

still b e a p p r o x i m a t e l y G a u s s i a n i f t h e i n p u t s p i k e t r a i n

were s u f f i c i e n t l y randon?,. A more g e n e r a l m o d e l is g i v e n in

t h e a p p e n d i x which t a k e s i n t o a c c o u n t t h e r a n d o m n e s s

o b s e r v e d i n s p i k e t r a i n s and from which i t f o l l o w s as a

t h e o r e m t h a t t h e membrane p o t e n t i a l s a r e G a u s s i a n ,

T h e c o v a r i a n c e e q u a t i o n i s i d e n t i c a l w i t h t h e

l i n e a r i z e d t i t n e - d e p e n d e n t a o d e l i n E q , ( 1 7 ) , a s i t ntust be

t c r e m a i n consistent i n the l i m i t o f s m a l l d e p a r t u r e s f r o &

s t e a d y - s t a t e . However, f o r G a u s s i a n membrane p s t e n t i a l s t h e

c o v a r i a n c e e q u a t i o n is e x a c t l y l i n e a r and, b e c a u s e t h e

p r o b a b i l i t y d i s t r i b u t i o n enters B u s s g a n g t s t h e o r e m i n a

well-beha ved r a n n e r , small departures f r o m n o r m a l i t y

i n t r o d u c e c o r r e s p o n d i n g l y small d o p a r t u r e s from strict

l i n e a r i t y .

Since a l l t h e i n f o r m a t i o n a b o u t t h e c o v a r i a n c e of

, a n d only c o v a r i a n c e i n f o r m a t i o n , is c o n t a i n e d i n

$ k ( t ) , t h i s v a r i a b l e will b e c a l l e d t h e c o v a r i a n c e even

t h o u g h , s tr ict1 y s p e a k i n q , the c o v a r i a n c e i s a s e c o n d - o r d e r

Piq. 2 8 . Sumnary diagram of t h e v a r i a b l e s which e n t @ r

i n the analysis of neuronal interaction. a) The membrane

potential V,(t) i n c l u d e s action potentials a s w l l as

graded membrane p o t e n t i a l s . b) $(t) is the effective

membrane p o t e n t i a l which would be p r e s e n t i f the a c t i o n

p o t e n t i a l were a b s e n t . c ) $a(t) i s t h e mean ef fec t ive

nembcane p o t e n t i a l . d) $;(t) is equivalent to the

d i f f a r e n c e plo(+)- $a(t) and i s q u a d r a t i c a l l y r e l a t e d to t h e

cova r i ance of $a(t) (from S e j n o u s k i , 1977a).

p r o p e r t y . S i n i l a r l y , a p r i z i e w i l l d e n o t e t h e covariance i n

o t h e r v a r i a b l e s .

T h e n o n l i n e a r model o f n e u r o n a l i n t e r a c t i o n , summarized

i n Fig. 28, beccaes more l i n e a r a t each s u c c e s s i v e stage o f

p r o b a b i l i s t i c a n a l y s i s : a) T h e m e ~ b r a n e p o t e n t i a l , i n c l u d i n g

t h e h i q h l y n o n l i n e a r a c t i o n p o t e n t i a l , i s t h e p r i m a r y

v a r i a b l e . b) A s m o o t h e r effective membrane p o t e n t i a l is

introduced which l e a d s t o a n o n l i n e a r a o d e l ( 2 ) . c) A t t h e

n e x t l e v e l t h e e q u a t i o n (5) f o r t h e mean e f f e c t i v e mnerabrane

p o t e n t i a l i s s i g n i f i c a n t l y less n o n l i n e a r . d) I n t h e l a s t

s t a g e of analysis t h e c o v a r i a n c e e q u a t i o n (29) h a s a l i n e a r

f c rm. The l a s t two l e v e l s are c o u p l e d b y t h e a v e r a g e f i r i n g

r a t e s ( 7 ) .

S t a t i o n a r y Case

T h e e q u a t i o n s f o r t h e a e a n s a n d c o v a r i a n c e s o f t h e

membrane p o t e n t i a l s a r e i m p l i c i t l y coup led t h r o u g h t h e A A

v a r i a n c e s , w h i c h a f f e c t R,( 3,) i n ~ q . ( 6 ) and R:( 9,) in

Eq. ( 2 9 ) . I f t h e coupled e q u a t i o n s have a s t a t i o n a r y

s o l u t i o n , t h e n t h e i n t e r a c t i o n n a t r i x is c o n s t a n t and t h e

c o v a r i a n c e e q u a t i o n c a n b e s o l v e d a n a l y t i c a l l y .

In terms of t h a fundamen ta l homogeneous s o l u t i o n s of

E q . (29 ) , w h i c h s a t i s f y

t h e s o l u t i o n of t h e s t a t i o n a r y covariance e q u a t i o n is

t h e impulse response.

T h e soluticn of t h e c o v a r i a n c e equation represents a

a o l t i d i m e n s i o n a l l i n e a r filter (Wiener, 1 9 4 9 ) . and the

c o v a r i a n c e e q u a t i o n is the s o - c a l l e d s t a te-variable f ora

(Kalman G Bucy, 196 1 ) . I d e n t i c a l e q u a t i o n s a r e used in

coamunica t ion theory t o extract s i g n a l s f r o n noise a n d i n

systems t h e o r y t o model and c o n t r o l p h y s i c a l s y s t e R s (Bocy ' 6

0 Joseph, 1 9 6 8 ; ~ s t r o m , 1 9 7 0 ) .

T h e s o l u t i c n of the c o v a r i a n c e e q u a t i o n i s p a r t i c u l a r l y

s i m p l e in a b a s i s where the i n t e r a c t i o n n a t r i x K' i s in

J o r d a n f o r m ( E i r k h o f f and Rota, 1 9 6 9 ) . It is always

p o s s i b l e t o f i n d a s i r r i l a r i t y t r a n s f o r m

s u c h that K' c n l y h a s entries on t h e d i a g o n a l o r just

above i t . Purthermore, 2' c a n be written a s a d i r e c t s u n

of blocks w i t h an e i g e n v a l u e A X o f K' on t h e d i a g o n a l of

each b l o c k :

S i n c e several b l o c k s aay have the same e i g e n v a l u e , let

t h e k - t h block of hl, have d i ~ e n s i o n E Q ~ ~ . Define

a s t h e m - t h co lumn o f t h e (n , k ) - b l o c k of t h e s i m i l a r i t y

t r a n s f o r m 3 w h i c h t ransforms t h e i n t e r a c t i o n matrix t o * sl!

Jordan for.. S i m i l a r l y , l e t b e t h e n-th row of

. B y v i r t u e of t h e i r d e f i n i t i o n , t h e s e two sets of

v e c t o r s a re b i o r t h o n o r n a l

w h e r e the l e f t side is t h e inner p r o d u c t of the t s o vectors.

Each set o f v e c t o r s forras t h e b a s i s f o r the l i n e a r vector

space, t h c u q h not n e c e s s a r i l y a n o r t h o n o r m a l one.

The two s e t s o f v e c t o r s {y:) a d [w*:'] s a t i s f y

a n d are s c m e t i m e s c a l l e d generalized e i g e n v e c t o r s

( S c h m e i d l e r , 1965) . T h e c o n v e n t i o n a l eigenvectors o f K' ?la,

are the y m w i t h B = 1, a n d t h e c o n v e n t i o n a l e i g e n v e c t o r s x n8

of t h e a d j o i n t o f KLa a r e the Y / m, with m = m n ~ .

Generalized e i g e n v e c t o r s a r i se because a n o n s y r a m s t r i c matrix

c a n n o t always b e diagonalized. However, t h e g e n e r i c

n o n s y m r n e t r i c m a t r i x i s d i a g o n a l i z a b l e a n d o n l y s p e c i a l cases

r e q u i r e o f f - d i a g o n a l e l e m e n t s , An e x a m p l e i s given later i n

t h i s section u h i c h i l l u s t r a t e s t h e difference.

fn o p e r a t o r no t a t i o n t h e t r a n s i t i o n a a t r i x is

and t a k e s t h a form

and

The t r a n s i t i o n n a t r i x r e p r e s e n t s t h e response of the f i l t e r

t o an impulse i n p u t , I f t h e s o l u t i o n is s t a b l e ( o(,,< F

f o r all t h e e i q e n v a l u e s ) , then e a c h e i g e n v a l u e of /('

contributes a term w h i c h , i n the gene ra l case, is

e x p o n e n t i a l l y damped and s i n u s o i d a l l y iaodulated. The

enve lope of t h e k- th term f o r t h e e i g e n v a l u e hk. owing t o t R - I

the f a c t o r , has a p e a k occu r ing a t (k-1) t /( I - - N ~ )

after t h e i r a p u l s e . Bttached t o each h h is a sequence of

nested s u k s p a c e s s p a n n e d by

which are i n v a r i a n t unde r t h e o p e r a t o r s En, i n E q . [34)

for the t r a n s i t i o n ma t r ix :

T h e s o l u t i o n o f t h e c o v a r i a n c e equation t o a n i m p u l s e

input c a n b e u n d e r s t o o d q e o m e t r i c a l l y : t h e i n p u t is

p r o j e c t e d o n t o t h e subspaces ank and the o u t p u t from each

s u b s p a c e unfolds i n time; each terra of Jab(*) i n Eg. ( 3 4 )

represents a s p i r a l (first o u t g o i n g , t h e n i n g s f n g ) in t h e rl e

p l a n e spanned b y the r e a l and i m a g i n a r y parts of m. ; far r\ l

example, i f t h e input were Y/% A c t ) then the o u t p u t would

Y-?- / \ V f i l f o l l o w t h e s e q u e n c e Y m , I 1-1 ... A deeper insight i n t o t h e a n a l y t i c structure of t he

c o v a r i a n c e e q u a t i o n c a n b e g a i n e d b y examining t h e L a p l a c e

t r a n s f o r m o f t h e t r a n s i t i o n matrix i n Eq . ( 3 1 )

which h a s t h a fcra of a r e s o l v e n t . I n terms o f L a p l a c e

t r a n s f o r m s , the solution o f t h e c o v a r i a n c e equation is

XMLI 2 Z L L . ~ L B ~ , $ L $ ~ . ( 3 8 ) b C

~ h u s , t h e ~ a p l a c e - t r a n s f o r m e d o u t p u t C $61 is just a

l i n e a r a l g e b r a i c t r a n s f o r m a t i o n of t h e Laplace-transf ormed I

i n p o t [ 7 with a n i n t e r a c t i o n p a r t [ T ~ ~ ] and an

i n p u t b r a n c h i n g p a r t Bbc . An e x p l i c i t form of t h e i n t e r a c t i o n p a r t is given by

t h e L a p l a c e transf orrn o f Eq . ( 3 4 ) :

Each p o l e o f t h e r e s o l v e n t h a s a s s o c i a t e d w i t h i t an

i n v a r i a n t s n b s p a c e as defined above. An i n p o t r e c t o r

w i t h i n 3'k is t r ans fo rmed by E nii i n t o an output r e c t o r

w i t h i n the sane s u h s p a c e . As t h e o r d e r o f t h e pole

i n c r e a s e s , which is only p o s s i b l e when t h e c e a r e

o f f - d i a q o n a l e l ements i n t h e J o r d a n fora of /(' , t h e

dimension o f the i n v a r i a n t s u b s p a c e a t t a c h e d t o the p o l e

d e c r e a s e s i n a nes ted f a s h i o n

ank+ , c ank . The Laplace t r a n s f o r a o f t h e transition n a t r i x i n

Eq. (37) is an a n a l y t i c f u n c t i o n of small p e r t u r b a t i o n s t o

A evsn t h o u q h t h e e i g e n v e c t o r s and eigenfunctions which

appear i n Eq. ( 3 9 ) a r e g e n e r a l l y n o t a n a l y t i c (Kato, 1 9 6 6 ) .

T h e s t a k i l i t y o f the c o v a r i a n c e e q u a t i o n t o s m a l l

perturbations o f the c o u p l i n g s t r e n g t h s between neurons is

e x a a i n e d i n t h e next p a r t .

The damped harmonic o s c i l l a t o r is a s i m p l e example

which i l l u s t r a t e s soBe of t h e p r o p e r t i e s of t h e g e n e r a l

case. T h e second-order e q u a t i o n

aL x + L O X + W ? X = 0 , dt

v h e r e 0.3, i s the f requency o f t h e undamped o s c i l l a t o r and

D is t h e d a m ~ i n g c o e f f i c i e n t , can b e written a s t h e

e q u i v a l e n t f irst -order e q u a t i o n

which h a s t h e same form as t h e t w o - d i ~ e n s i o n a l covariance

e q u a t i o n . The c h a r a c t e r i s t i c e q u a t i o n f o r t h i s problem is

A' + 2 D A + u,b = 0 ,

which has t h e s c l u t i o n s

D > W, o v e r damped c r i t i c a l l y damped

D < W, underdampea

T h e underdamped s o l u t i o n i s

which c a n b e coapa red t o E q . ( 3 4 ) w i t h ) and

mi = p d t Only i n the c r i t i c a l l y damped case can the

i n t e r a c t i o n m a t r i x fail t o d i a g o n a l i z e , b u t t h i s

c i rcurns ta r?ce o n l y o c c u r s i n two and t h r e e d i m e n s i o n s . Fo r

t h e r e t o e x i s t two e q u a l complex e i q e n v a l u e s (which would

a l l o w an unde r d a m p e d o f f - d i a g o n a l J o r d a n for^) requires a t

l e a s t f o u r d i r r e n s i o n s hecause complex e i g e n v a l u e s a l w a y s

come i n c o n j u g a t e p a i r s .

The "quality f a c t o r t f o r Q o f a damped o s c i l l a t i o n is a

measure of t h e s h a r p n e s s o f i t s re sonance . For t h e

o s c i l l a t o r r e p r e s e n t e d b y Eq. ( 3 7 ) t h e Q is d e f i n e d as

If t h e o s c i l l a t o r i s e x c i t e d by a sudden displacement , then

t h e number of cycles d u r i n g decay of t h e e n v e l o p e t o I/e of

i t s o r i g i n a l a i a p l i t u d e is a p p r o x i m a t e l y Q . L e t us

q e n e r a l i z e t h e Q t o t h e damped o s c i l l a t i o n s r e p r e s e n t e d in

Eq. ( 3 4 ) by a s s i g n i n g t o each e i g e n v a l u e

Overdamped modes a r i s e from e i g e n v a l o e s which l i e on t h e

real axis (Q, = 0 ) .

The damping t i m e c o n s t a n t f o r a mode i s

r , = ~ / ( l - ~ n ) . (4 1)

For there t o occur underdamped o s c i l l a t i o n s l a s t i n g l o n g e r

than N s e c o n d s w i t h a t y p i c a l menbrane time c o n s t a n t

T 13 o s e c r e q u i r e s t h a t ( 1 - 0 I / N . H e nce , o n l y t h o s e o i g e n v a l u e s whose r e a l p a r t s are n e a r l y e q u a l to

one c o n t r i b u t e t o c o r r e l a t i o n s on a l o n g t i m e scale.

A l t hcuqh no e x p e r i m e n t a l l y de t e rmined i ~ t e r a c t i o n

m a t r i x i s a v a i l a b l e i n t h e v e r t e b r a t e n e r v o u s system, t h a t

some m o d e s i n t h e b r a i n a r e underdamped m i g h t b e i n f e r r e d

from t h e u i d e s ~ r e a d rhy thms o b s e r v e d i n EEG r e c o r d i n g s o v e r

human c e r e b r a l c o r t e x ( N o r r e l l , 1967) a n d from r e c o r d i n g s of

i n t r a c e l l u l a r membrane p o t e n t i a l s i n c o r t i c a l n e u r o n s ( E l u l ,

1 9 6 8 ; Offenbach, 1 9 7 5 ) . With a t y p i c a l membrane t i m e

c o n s t a n t of - 1 0 met, t h e a l p h a rhythm at Ij, -- IO/sec

c o r r e s p o n d s t o

PR I- ( 2 ~ 2 / ~ ) ' ~ -- 0 . 6 3

O t h e r EEG r h y t h n s o c c u r between 5 / s e c and 40/sec .

Neurona l F i l t e r s and l lenory

In t h e s t a t i o n a r y case t r e a t e d above , t h e membrane

p o t e n t i a l c o v a r i a n c e s a r e p a r t i c u l a r l y s e n s i t i v e t o i n p u t

c o v a r i a n c e s with t h e s p a t i a l and t e ~ p o r a l p a t t e r n s

a s s o c i a t e d w i t h e a c h e i g e n v a l u e : these w i l l b e c a l l e d the

c o v a r i a n c e modes of t h e n e u r o n a l f i l t e r i n a n a l o g y vith the

norlaal a o d e s of mechan ica l s y s t e n s , However, t h e no rma l

modes of an undamped mechan ica l system a r e d e r i v e d f r o m a

symmet r i c m a t r i x and a r e a lmays o r t h o g o n a l , b u t t h e

i n t e r a c t i o n matrix i s g e n e r a l l y n o t symmetric and its

e i g e n ~ e c t o r s a r e g e n e r a l l y n o t o r t h o g o n a l . A s a

consequence , c o u p l e d c o v a r i a n c e nodes c a n a p p e a r w h i c h

e n e r q e and d e c a y i n t e m p o r a l s e q u e n c e .

Because t h e i n t e r a c t i o n m a t r i x i n the c o v a r i a n c e

e q u a t i o n d e p e n d s on R ; ( $ b ) , t h e c o v a r i a n c e .odes o f a

n e u r o n a l f i l t e r a r e a d j u s t a b l e and depend o n t h e average

f i r i n q r a t e s , a s s h o u r i n Fiq. 29. T h u s , by c o n t r o l l i n g t h e

background firing r a t e s in a " l o w e r f t p r o c e s s i n g c s n t e r , a

" h i q h e r " ~ r o c e s s i n q c e n t e r could c o n t r o l t h e f i l t e r i n g of

incominq i n f o r r a t i o n . Such a n a d j u s t a b l e f i l t e r nay

u n d e r l i e t h e a b i l i t y of an organism t o s e l e c t i v e l y d i r e c t

i t s a t t e n t i o n and t c extract s p 2 c i a l f e a t u r e s from s e n s o r y

Fig. 2 9 . B l o c k d i aq ram r e p r e s e n t i n g a n e u r o n a l filter:

t h e input coua r i ances h e t v e e n spike trains 9: , linearly filtered, drive t h e o u t p u t c o v s r i a n c e s b e t w e e n

membrane putectials qk (t) . The i m p u l s e - r e s p o n s e

Tab (t-t') of t h e filter d e p e n d s on t h e b a c k g r o u n d f i r i n g

r a t e s R Q ,

i n f o r a a t i o n .

Some i n p u t s t o ar! a r e a may c o n t r i b u t e little t o the

background f i r i n g r a t e s of p o s t s y n a p t i c n e u r o n s , b a t c o u l d

have a s i g n i f i c a n t effect on m e ~ b r a n e p o t e n t i a l c o v a r i a n c e s ,

An e x p e r i n e n t c o n c e r n e d s o l e l y y i t h a v e r a g e f i r i n g r a t e s

miqht n o t d e t e c t s u c h an i n p u t e v e n i f it were a major

s o u r c e o f i n f o r m a t i c n t o t h e a r e a . C a s e s o f major

a n a t o m i c a l pathways w i t h o u t c o r r e s p o n d i n g i n f l u e n c e a s

judqed b y a v e r a g e f i r i n g r a t e s a r e n o t uncoamon. For

example, mos t a s c e n d i n g p r o j e c t i o n s f rom t h a l a m i c n u c l e i t o

c e r e b r a l c c r t e x a re accompanied by r e c i p r o c a l

c o r t i c o t h a l a m i c t r a c t s ( C a r p e n t e r , 1976), b u t t h e

e l e c t r o p h y s i o l o q i c a l i n f l u e n c e of t h e s e d e s c e n d i n g

p r o j e c t i o n s t o t h e t ha l amus is veak compared t o t h e

i n f l u e n c e o f o t h e r i n p u t s , a s nieasured by t h e f i r i n g r a t e s

o f t h a l a m i c nEurons . A s a s p e c i f i c example, i n t h e v i s u a l

s y s t e m o f t h e c a t t h e r e s p o n s e . o f ce l ls i n t h e t h a l a m i c

d o r s a l l a t e r a l g e n i c u l a t e n u c l e u s t o s p o t s of l i g h t is n o t

a f f e c t e d by c o o l i n q v i s u a l c o r t e x ( K a l i l and Chase , 1 9 7 0 ) .

However, f o r none n e u r o n s t h e c o o l i n g had a s i g n i f i c a n t

r e v e r s i b l e effect o n t h e p a t t e r n of i m p u l s e s i n r e s p o n s e t o

rroving s l i t s o f l i g h t , a s shown i n Pig. 30. I f t h e p r imary

pu rpose o f d e s c e n d i n q i n f l u e n c e t o t h a l a m i c r e l a y n u c l e i is

t o p r o v i d e s u c h t i m i n q i n f o r m a t i o n , t h e n c o r t i c a l f e e d b a c k

c o u l d p l a y a n i n p o r t a n t r o l e i n c o v a r i a n c e p r o c e s s i n g .

Pig. 30, Average f i r i n g r a t e s of two neurons in t h e

c a t lateral g e n i c u l a t e nucleus (LGN) i n response t o a moving

s l i t o f l i q h t . The LGN receives i n p u t s from both the retina

and visual c o r t e x , The p a r t of c o r t e x t h a t p r o j e c t s t o the

LGN was coo led (thereby abolishing t h e o u t p u t froa t h e

cooled area) in order t o s t u d y its effect on the response of

the L G N t o l i q h t . A t t h e s t a r t o f each his togram t h e slit

beqan moving from a p o s i t i o n 10 ' t o t h e r i g h t of t h e

r e c e p t i v e - f i e l d center. After p a s s i n g t o a position 10' to

t h e l e f t of center t h e slit reversed d i r e c t i o n . T h e top

p a i r of histoqrams shcws t h e r e s p o n s e b e f o r e and after t h e

c o r t e x was cocled, a v e r a g e d over 23 trials each. heir

similarity i n d i c a t e s t h a t t h e effect of c o o l i n g was

r e v e r s i b l e . a ) The n e u r o n was inhibited when t h e s l i t was

i n t h e r e c e p t i v e field. T h e a v e r a g e r e s p o n s e o f t h e n e u r o n

while the cortex was c o o l e d , shown a t the bottom, was

significantly different f r o a t h e normal response: t h e

a v e r a g e backqround activity uas r e d u c e d , the response t o t h e

s l i t moving from left t o r i g h t was increased, and a new

b u r s t a p F e a r e d when t h e slit moved i n t h e o p p o s i t e

d i r e c t i o n . b) T h e neuron g a v e a burst of firing when t h e

slit was i n t h e r e c e p t i v e field. S e c o n d a r y b u r s t s ( a r rows)

a p p e a r e d when the c o r t e x was cooled ( f r o m ~ a l i l a n d Chase ,

1 9 7 0 ) .

P O S T C O O L 1101 ,, ,-

P R E AND P O S T

C O M B I N E D ( * a )

? R E C O O L c a o l

P O S T C O O L I201

C O O L 1401

Fig. 30

Reaa rkab ly l i t t l e is knova a b o u t t h e p h y s i c a l basis of

~ e i a o r y . dt is g e n e r a l l y b e l i e v e d t h a t s h o r t - t e r n meaorg

depends on t r a n s i e n t e l e c t r i c a l a c t i v i t y , and t h a t long-term

memory i s t h e r e s u l t of r e l a t i v e l y permanent s t r u c t u r a l

chanqes . E a r l y n e u r o a n a t o ~ c a l e v i d e n c e f o r f e e d b a c k Issps

between n e u r o n s ( L o r e n t e d e ~ 6 , 1938) l e d t o t h e i d e a t h a t

r e v e r b e r a t i o n and r e v e r b e r a t o r y c i r c u i t s were a p o s s i b l e

p h y s i c a l b a s i s f o r s h o r t - t e r m memory (Hebb, 1949 ; Kimble ,

3965) . Al though c i r c u l a t o r y c h a i n s of n e u r o n s do e x i s t ,

some s y n a p s e s b e t y e e n neu rons a r e know t o be i n h i b i t o r y ; i n

a d d i t i o n , a single a c t i o n p o t e n t i a l is u s u a l l y i n s u f f i c i e n t

b y i t s e l f t o c a u s e a n a c t i o n p o t e n t i a l t o o c c u r i n a n o t h e r

neu ron . Hence, t h e c o m p u t e r - l i k e c i r c u l a t i o n of i t a p u l s e s in

c l o s e d n e u r a l c i r c u i t s is u n l i k e l y .

T h e analysis of c o r r e l a t i o n p r o c e s s i n g given i n t h i s

p a r t shows how r e p e t i t i v e p a t t e r n s c a n c i r c u l a t e a s a

" s t a t i s t i c a l r e v e r b e r a t i o n i f e v e n though i n d i v i d u a l impulses

need n o t . T h e i n f o r m a t i o n i s d i s t r i b u t e d i n a l a r g e

c o l l e c t i o n of n e u r o n s and is represented a s t e m p o r a l

s e q u e n c e s of c o v a r i a n c e modes. If some of the b r a i n t s

c o v a r i a n c e modes had a s u f f i c i e n t l y l o n g c o h e r e n c e time ( a

s u f f i c i e n t l y h i g h Q ) , t h e n membrane p o t e n t i a l c o v a r i a n c e

c o u l d serve a s t h e p h y s i c a l basis f o r s h o r t - t e r m memory. A

n e u r o n a l f i l t e r h a s t h e a d d i t i o n a l a d v a n t a g e t h a t d i f f e r e n t

c o v a r i a n c e modes c a n h e s e l e c t e d by a d j u s t i n g t h e background

f i r i n g r a t e s of t h e n e u r o n s . I n t h i s a t h e filter

c h a r a c t e r i s t i c s of a n a r e a c a n be s e l e c t i v e l y a l t e r e d in

o r d e r t o s e l e c t i v e l y s t o r e a p a r t i c u l a r t y p e of i n f o r a a t i o n .

T h i s i s t h a same mechanism p roposed f o r s e l e c t i v e attention,

b u t i n the c a s e o f s e l e c t i v e s t o r a g e a f v l o v e r n p r o c e s s i n g

c e n t e r c o n t r o l s the background f i r i n g r a t e s i n a " h i g h e r "

p r o c e s s i n q c e n t e r ,

The c e r e b r a l c o r t e x r e c e i v e s i n p u t s from t h a l a m i c

n u c l e i , a s s o c i a t i o n f i b e r s f rom o t h e r c o r t i c a l a r e a s ,

cornmisural f i t e r s f r o m the c o r p u s co l lo som, and a v a r i e t y o f

o t h e r a f f e r e n t s f rom t h e b r a i n stem, b a s a l g a n g l i a , and

elsewhere ( C a r p e n t e r , 1976) . T h e background f i r i n g r a t e s i n

a c o r t i c a l a r e a cou ld be c o n t r o l l e d b y s e v e r a l of t h e s e

i n p u t s , one p a r t a r i s i n g f rom g e n e r a l a r o u s a l and a n o t h e r

p a r t f rom a s p e c i f i c s e n s o r y m o d a l i t y , w h i l e o t h e r i n p u t s

may be p r i m a r i l y c o n c e r n e d w i t h p r o c e s s i n g c o v a r i a n c e .

I n p u t s from a r e a s which p r o v i d e t empora l ly -coded i n f o r m a t i o n

should synapse near t h e s p i k e - i n i t i a t i n g r e g i o n o f a n e u r o n

t o r e d u c e the d e l a y and decrement from e l e c t r o t o n i c

c o n d u c t i o n i n d e n d r i t e s , I n p u t s from a r e a s w h i c h c o n t r o l

t h e backqround f i r i n g r a t e s c o u l d a s well , o r w i t h

a d v a n t a q e , s y n a F s e on t h e d i s t a l p a r t s of d e n d r i t e s .

O u r u n d e r s t a n d i n g o f t h e p h y s i c a l b a s i s f o r long-term

memory i s n o t much b e t t e r t h a n t h a t of A r i s t o t l e , who

compared mamory w i t h t h e i m p r e s s i o n s l e f t i n wax. T h e

h o l o q r a n i s s t o r e d d i s t r i b u t i v e l y a n d c a n b e r e c a l l e d

a s s o c i a t i v e l y ( J u l e s z and Pennington , 1965; Longuet -Higgins ,

1968; Gabor, 1968) . I n a t t e m p t i n g to make t h i s metaphor

Bore precise, s o w worke r s have s o u g h t t o i d e n t i f y p a r t s of

t h e h o l o g r a p h i c p r o c e s s w i t h p h y s i c a l components of neu rons ,

s u c h a s t h e c c ~ p u t a t i o n of s i n u s o i d a l i n t e r f e r e n c e f r i n g e s

a t synapses (8es t l ake , 1970; P r i b r a r n , Nouar, C Baron, 1974) .

Another g r o u p o f workers, r e c o g n i z i n g t h a t t h e m a t h e m a t i c a l

b a s i s f o r the storage and r e c a l l o f h o l o g r a p h i c i n f o r m a t i o n

i s c o r r e l a t i o n s t o r a g e and l i n e a r f i l t e r i n g

(Lonque t -B igg ins , H i l l s h a w , & Buneman, 1 9 7 0 ; P f a f f e l h u b e r ,

1975), have s t u d i e d t h e p r o p e r t i e s of s i m p l i f i e d l i n e a r

f i l t e r s ( S t e i n b u c h , 1961 ; Anderson, 1972; i ? i g s t r ~ m , 1973;

Kohonen 1977) . An o n t i r e l y d i f f e r e n t s t a r t i n g p o i n t - t h e s t o c h a s t i c

model inq of n o n l i n e a r l y i n t e r a c t i n g n e u r o n s - h a s l e d t o

e q u a t i o n s f o r a l i n e a r f i l t e r a t t h e l e v e l of c o v a r i a n c e

b e t w e e n aerttbrane p o t e n t i a l s , T h e s t o c h a s t i c model i n c l u d e s

a d d i t i o n a l . features, s u c h a s t h e s e l e c t i v e s t o r a g e and

r e t r i e v a l of i n f o r m a t i o n , which a r i s e f rom t h e e s s e n t i a l

advanced t e c h n o l o g i e s of each a g e have p r o v i d e d a e t a p h o r s

f o r m e m o r y - from h y d r a u l i c s i n t h e 17th c e n t u r y and

telephone s u i t c h b o a r d s i n t h e 7 9 t h c e n t u r y , t o d i g i t a l

computers i n t h e 2 0 t h c e n t u r y . The most r e c e n t metaphor i s

ho log raphy : a s i n l o n g - t e r m memory, i n f o r m a t i o n i n a

n o n l i n e a r i t y of t h e model .

The q a p between the m a t h e m a t i c a l s t u d y o f memory i n

s i m p l e m o d e l s a r d t h e e x p e r i m e n t a l s t u d y of t h e p h y s i c a l

b a s i s f o r memory c o u l d b e b r i d q e d by r e f i n i n g e x i s t i n g

t e c h n i q u e s f o r i n t r a c e l l u l a r r e c o r d i n g from c o r t i c a l neurons

and b y m e a s u r i n g e n s e a b l e - a v e r a g e reembrane p o t e n t i a l

c o r r e l a t i o n s d u r i n q c o n t r o l l e d s e n s o r y s t i m u l a t i o n

( i n i t i a l l y by m e a s u r i n g a u t o c o r r e l a t i o n in s i n g l e neurons

a n d l a t e r b y ~ e a s u r i n g the c r o s s - c o r r e l a t i o n b e t w e e n

n e i q h b o r i n q p a i r s of n e u r o n s ) . A t e s t a b l e t h e o r e t i c a l

p r e d i c t i o n f o r t h e p h y s i c a l basis o f l o n g - t e r m memory is

made i n t h e next p a r t ,

IV. Motor L e a r n i n g i n t h e C e r e b e l l u r a

A l t h o u g h t h e mechanisms w h i c h u n d e r l i e l e a r n i n g a re

unknown, e x p e r i r a e n t a l e v i d e n c e i n d i c a t e s t h a t s t o r e d

i n f o r m a t i o n i n the n e r v o u s systein i s d i s t r i b u t e d i n basge

c o i l e c t i o n s o f n e u r o n s r a t h e r t h a n l o c a l i z e d a s i n t h e case

o f d i g i t a l c o r p u t e r s ( R o s e n z u e i g and B e n n e t t , 1 9 7 6 ) . T h e

p r o b a b i l i s t i c a n a l y s i s o f i n t e r a c t i n g n e u r o n s i n P a r t s I1

a n d 111 s u q g e s t s t h a t d i s t r i b u t e d i n f o r m a t i o n c o u l d be

p r o c e s s e d and t e m p o r a r i l y s t o r e d a s c o r r e l a t i o n s between

membrane p o t e n t i a l s . The main p u r p o s e of P a r t I V i s t o

examine how these c o r r e l a t i o n s c o u l d be p e r m a n e n t l y s t o r e d

a n d subsequently r e t r i e v e d (Se j n o v s k i , 7977a) .

C e r e b e l l a r C o r t e x

T h e r s g i o n o f t h e b r a i n whose l o c a l c i r c u i t s are b e s t

known i s t h e c e r e b e l l u m , a g r a p e f r u i t - s i z e d s t r u c t u r e which

lies a t t h e back of t h e s k u l l , a s shown i n P ig . 3 1 ( ~ l i n ' i s ,

1 9 7 5 ) . A n i m a l s d e p r i v e d o f a c e r e b e l l u m s u f f e r severe

d i s t u r b a n c e s of c o o r d i n a t i o n and e q u i l i b r i u m . Because t h e

c f r e b e l l u a r e c e i v e s i n p u t s f r o m b o t h the m o t o r system and

the s e n s o r y n o n i t o r systenr, t h e r e is l i t t l e d o u b t t h a t i t is

i n t i m a t e l y i n v o l v e d i n motor c o o r d i n a t i o n , b u t t h e p r e c i s e

f u n c t i o n o f t t e cerebelluia i n o r g a n i z i n g movement is s t i l l

Cerebellum

PONS

N

F i g . 31. S i d e v i e v s o f the human b r a i n with t h e f r o n t

of t h e h e a l to the left. a) T h e cerebellum l i e s a t the back

o f t h e b r a i n b e n e a t h t h e convoluted cerebrum, b ) A midline

c r o s s - s e c t i o n o f t h e b r a i n s tem and c e r e b e l l u m is shown

below (froin L l i n a s , 1 9 7 5 ) .

unknown. The b a s i c e l e m e n t s o f t h e c e r e b e l l u m a re t h e same

i n a L 1 v ~ r t e b r a t e s , i n d i c a t i n g t h a t its e v o l u t i o n i s h i g h l y

c o n s e r v a t i v e . The c e r e b e f l u n has e n l a r g e d b e t w e e n t h r e e f o l d

a n d f o u r f o l d i n the past m i l l i o n y e a r s o f human e v o l u t i o n

- w h a t e v e r t h e c e r e b e l l u a c o n t r i b u t e s t o m o t o r

c o o r d i n a t i o n i s o b v i o u s l y o f g r e a t a d a p t i v e v a l u e .

S u r p r i s i n q l y , i n s o m e e lec t r i c f i s h u h i c h can s e n s e v e r y

weak e l e c t r i c f i e l d s , t h e c e r e b e l l u m is e n o r m o u s , f i l l i n g

m o s t o f t h e s k u l l .

L i k e t h e c e r e b r u m , t h e c e r e b e l l u m i s e l a b o r a t e l y f o l d e d

a n d h a s a t h i n c o r t i c a l l a y e r o f g r e y mat ter . U n l i k e t h e

c e r e b r a l cortex, h o w e v e r , c e r e b e l l a r c o r t e x c o n t a i n s a

r e a a r k a b l y regular, r e p e t i t i v e s t r u c t u r e . The f u n d a m e n t a l s

of c e r e b e l l a r s t r u c t u r e w h i c h were established b y ~ a m 6 n y

C a j a l i n 1888 h a v e b e e n r e p e a t e d l y c o n f i r m e d i n a v i d e r a n g e

o f v e r t e b r a t e s (~lings, 1969) . T h e cerebellar c o r t e x has

s e v e n b a s i c e l e m e n t s : t w o c a r r y i n p u t s i n t o t h e cerebellum,

t h e m o s s y fibers a n d t h e c l i r s b i n g f i b e r s , a n d only o n e

e l e m e n t s e r v a s a s o u t p u t , t h e P u r k i n j e c e l l a x o n s ; t h e f o u r

r e m a i n i n g e l e m e n t s a r e n e u r o n s i n t r i n s i c t o t h e c e r e b e l l a r

c o r t e x - t h ~ g r a n u l e cells, t h e G o l g i ce l l s , t h e s t e l l a t e

c e l l s , a n d t h e basket cel ls . E a c h of t h e s e e l e m e n t s h a s a

d i s t i n c t q a o m e t r i c a l r e l a t i o n s h i p i n t h e c o r t e x , a s shown i n

Fiq. 3 2 .

T h e F u r k i n je c e l l i s one o f the raost d i s t i n c t i v e and

Fig. 3 2 .

Molecular layer

Purkinje iayer

Granular iayer

White matter

Climbing fiber

Deep cerebellar

nude1

T h r e e - d i m e n s i o n a l s c h e a a t i c diagram of

basic e l e m e n t s in the cerebel lar cortex. The m o s s y fi

the

bers

and c l i m b i n g fibers a r e the two na in i n p u t s and t h e Purk in je

c e l l axons are the only outputs. The axons of the g r a n u l e

c e l l s r i se t o t h e surface of the cortex uhere t h e y bifurcate

t o form l o n g p a r a l l e l fibers. E a c h Purkinje c e l l d e n d r i t i c

tree receives aanp e x c i t a t o r y synapses f ron passing p a r a l l e l

fibers a n d m u l t i p l e e x c i t a t o r y s y n a p s e s from a single

climbing f i b e r . The s t e l l a t e c e l l s , the G o l g i ce l l s , and

t h e basket cells receive e x c i t a t o r y input frols the p a r a l l e l

fibers a n d h a v e e x c l u s i v e l y i n h i b i t o r y e f f e c t s on t h e other

n e u r o n s in t h e c o r t e x (from B u l l o c k , 1977) .

d r a m a t i c n e u r o n s i n t h e n e r v o u s sys t em. E a c h Purk in je cell

has a l a r q e d e n d r i t i c t r e e w i t h i n t r i c a t e l y bifurcating

branches that resembles an e s p a l i e r e d pear tree, a s shown in

Pig, 1. The f l a t d e n d r i t i c t r e e s are densely stacked,

without c o n t a c t , p e r p e n d i c u l a r to t h e s u r f a c e o f the c o r t e x .

P u r k i n j e c3ll.s r e c e i v e tuo types of e x c i t a t o r y i n p u t : i n t h e

h u m a n c e r e b e e l u a each Purkinje cel l receives a p p r o x i m a t e l y

330 s y n a F s e s f rorr a single c l i m b i n g f ibe r and approximately

1C0,000 s y n a p s e s from passing p a r a l l e l fibers. G r a n d e c e l l

axons, called p a r a l l e l f ibers , aake synaptic c o n t a c t s a s

they r u n p e r p e n d i c u l a r l y t h r o u g h t h e p l a n e of t h e d e n d r i t i c

trees and parallel t o t h e c o r t i c a l s u r f a c e , I n c o n t r a s t , a

s i n q l e climbing f i b e r e n t w i n e s e a c h dendritic t r e e l i k e a

climbing v i n e and nakes n u l t i p l e s y n a p t i c contacts on e v e r y

major b r a n c h . A s c h e m a t i c view of a p a t c h o f P u r k i n j e c e l l

d e n d r i t e is shown i n F i g . 3 3 .

The g r a n u l e cell b o d i e s and dendrites form a dense

layer i m m e d i a t e l y b e n e a t h t h e Pnrkin je cells. T h e rnossy

fibers enter t h e q r a n u l e c e l l layer f r o n below and fo rm

n e s t s of s y n a p t i c c o n t a c t s w i t h t h e granule c e l l dendrites.

The r e m a i n i n q i n t e r n s u r o n s - t h e b a s k e t ce l ls , t h e

s t e l l a t e cells, and the G o l g i cells - a re a l l i n h i b i t o r y

i n t h e i r a c t i o n and m e d i a t e be tween t h e p a r a l l e l f i b e r s and

other c e r e b e l l a r n e u r o n s .

T h e c e r e b e l l a r c o ~ t e x r e c e i v e s i n p u t s , through the

F i g . 3 3 . Schematic i l l u s t r a t i o n of a c e r e b e l l a r

P u r k i n j e c a l l d e n d r i t e Pd (with a d e n d r i t i c branchlet), a

c l i m b i n g f i b e r CF ( e n t w i n i n g t h e d e n d r i t i c t r u n k ) , and a

p a r a l l e l f i b e r p f ( p a s s i n g t h r o u g h t h e d e n d r i t i c t r e e ) ,

b a s e d on P a l a y a n d Chan-Pa lay ( 1 9 7 4 ) . C l i m b i n g f i b e r

v a r i c o s i t i e s make n u m e r o u s s y n a p t i c contacts v i t h s p i n e s on

t h e d e n d e i t i c trunk ( f r o m S e j n o u s k i , 19773) .

mossy f i b e r s and c l i a b i n g fibers, f rom a b e w i l d e r i n g variety

of s o u r c e s : t h e r e a r a s e v e r a l nerve bundles i n the spinal

c o r d d e s t i n e d f o r t h e c e r e b e l l u m which c a r r y inf o r n a t i o n

a k o u t the c o n d i t i o n of muscles , t e n d o n s , j o i n t s , and spinal

i n t e r n e u r c n s ; isotor neu rons i n cerebral c o r t e x t h a t s end

motor commands down t h e s p i n a l c o r d have branches w h i c h

r e a c h t h e c e r e b e l l u m th rough r e l a y n u c l e i ; t h e v e s t i b u l a r

o r g a n s , which sense head v e l o c i t y a n d c o n t r i b u t e t o o u r

b a l a n c e , h a v e d i r e c t p r o j e c t i o n s t o t h e c e r e b e l l u r a . Inputs

t o the c e r e b e l l u m g e n e r a l l y send b r a n c h e s to t h e c e r e b e l l a r

n u c l e i w h i c h l i e b e n e a t h t h e cerebellar c o r t e x , a s shown i n

Fig. 32. These i n t e g r a t i o n c e n t e r s a l s o r e c e i v e t h e o u t p u t

f r o m t h e P u r k i n j e ce l l s , w h i c h is e n t i r e l y i n h i b i t o r y .

Thus, the c e r e b e l l a r cortex works i n p a r a l l e l w i t h the

c e r e b e l l a r n u c l e i .

T h e regularity of t h e c e r e b e l l a r structure h a s nade it

a n i d e a l m a t e r i a l f o r e l e c t r o p h y s i o l o g i c a l r e c o r d i n g

(Eccles, I t o , and s z e n t / a g o t h a i , 1967) . The e l e c t r i c a l

r e s p o n s e of each c e l l t y p e i n t h e c e r e b e l l u m h a s been

c a r e f u l l y s t u d i e d and a l l the synapses b e t v e e n t h e seven

b a s i c e l e a e n t s h a v e been c h a r a c t e r i z e d . P u r k i n je ce l l s

g e n e r a l l y have a h igh s p o n t a n e o u s f i r i n g r a t e of

20-100/second, b u t c l i m b i n g f i b e r s o n average d i s c h a r g e a t

o n l y l /second. Even though t h e i r a v e r a g e firing r a t e s are

low, n a i q h b o r i n g c l i i x b i n g f i bers some t imes f i r e

F i q . 3 4 . a) C r o s s - c o r r e l a t i o n b e t u e e n two nearby

c l i m b i n g f i b e r s i n t h e cerebellar c o r t e x a n d b ) their

a u t o c o r r e l a t i o n s , T h e c o r r e l a t i o n is g i v e n b y t h e s o l i d and

d o t t e d c o l u m n s a d d e d t o g e t h e r ; t h e s o l i d columns alone

r e p r e s e n t c r o s s - i n t e r v a l h i s t o g r a m s b a s e d o n l y o n nearest

p a i r s of impulses. Two Purkinje c e l l s were recorded

s i m u l t a n ~ o u s l y on two e x t r a c e l l u l a r m i c r o e l e c t r o d e s and t h e

d i s c h a r g e s o f t h e i r c l i m b i n g f i b e r s v e r e identified b y the

* c h a r a c t e r i s t i c c o m p l e x - s p i k e p a t t e r n . T h e a u t o c o r r e l a t i o n s

i n d i c a t e t h a t most c o m p l e x s p i k e s a re s e p a r a t e d b y a t least

100 ms. T h e c r o s s - c o r r e l a t i o n i n d i c a t e s s t e n d e n c y f o r the

t w o c l i m o i n q fibers t o d i s c h a r g e e i t h e r i n s y n c h r o n y or a t a

130 a s i n t e r v a l ( f r o m B e l l a n d K a w a s a k i , 1 9 7 2 ) .

s g n c h r o n o u s l y , Simultaneous r e c o r d i n g s f rom p a i r s of

c l i n b i n g fibers show s i g n i f i c a n t c o r r e h t i s a s l a s t i n g f o r

s e v e r a l hundred m i l l i s e c o n d s , a s shown i n F i g . 3L1 ( B e l l and

Kavasaki, 1 9 7 2 ) . T h u s , c l i m b i n g f i b e r s may b e p r i m a r i l y

conce rned w i t h t empora l ly -coded i n £ o r m a t i on.

Among i t s many f u n c t i o n s , t h e c e r e b e l l u m is i n v o l v e d i n

" f i n e - tuningi* t h e v e s t i b u l o - o c u l a r r e f l e x , a compensa to ry

e y e raovereent i nduced b y head r o t a t i o n ( I t o , 1975; Robinson ,

1976, M i l e s , 1977) . Because s y n a p t i c d e l a y s vould be

q r e a t e r t h a n the r e s p o n s e t i n e of t h e ref lex , no c l o s e d - l o o p

feedback f rom t h e r e t i n a t o t h e v e s t i b u l a r n u c l e i is

p o s s i b l e . HowGVer, t h e c e r e b e l l u m d o e s r e c e i v e v i s u a l

feedback t h r o u g h t h e climbing fiber s y s t e r a , and t h e a c c u r a c y

o f t h e open- loop vestibule-ocular r e f l e x i s improved b y

v i s u a l experience o v e r a tinte sca le o f days. Lesion o f the

vestibule-cerebellum e n t i r e l y a b o l i s h e s t h i s behavioral

p l a s t i c i t y . Thus, t h e c e r e b e l l u a is r e q u i r e d f o r

v e s t i b u l o - o c u l a r l e a r n i n g ; however, t h e e v i d e n c e d o e s n o t

yet p r o v e t h a t t h e cerebe l lum is indeed t h e s i t e o f t h i s

p l a s t i c i t y .

The ona-to-one match between c l i m b i n g f i b e r s and

P u r k i n j e cells and t h e c o m p l e t e c o m p a r t n e n t a l i z a t i o n o f each

P u r k i n j e c e l l d e n d r i t i c tree is un ique i n t h e n e r v o u s

sys tem. Br ind ley ( 1 9 6 4 ) and s z e n t G g o t h a i (1 96 8) have

p o i n t e d o u t t h a t a c l i m b i n g fiber i s i d e a l l y p o s i t i o n e d t o

i n f l u e n c e a l l the s y n a p s e s from p a r a l l e l f i b e r s t o a

P u r k i n j e cell, a n d h a v e s u g g e s t e d t h a t t h e p u r p o s e o f t h e

climbing f i b e r c o u l d b e t o c o n t r o l t h e p l a s t i c i t y of t h e

p a r a l l e l fiber s y n a p s e s . F o l l o w i n g t h e i r s u g g e s t i o n , Plarr

(1969) a n d B l b u z ( 1 9 7 1 ) p r o p o s e d d e t a i l e d t h e o r i e s f o r how

t h e c e r e b e l l u m c o u l d c o o r d i n a t e m o t o r acts and l e a r n t o

p e r f o m m o t o r s k i l l s . T h e aim here i s more l i m i t e d i n

s c o p e : r a t h e r t h a n p r o p o s e a n o t h e r a l l - e n c o m p a s i n g t h e o r y of

t h e c e r e t e l l u m , a t t e n t i o n i s f o c u s s e d o n t h e P u r k i n j e cell

d e n d r i t i c tree , and t h e o r i g i n a l s u g g e s t i o n a a d e by B r i n d l e y

a n d S z e n t a q o t h a i i s r e f o r m u l a t e d w i t h i n t h e f r a m e w o r k of

c o v a r i a n c e p r o c e s s i n g .

S t o r i n g C o v a r i a n c e

H i q h l y s k i l l e d movements, s u c h a s t h o s e r e q u i r e d t o

p l a y a a u s i c a l i n s t r u m e n t , aEe s h a p e d b y . r e p e t i t i v e

p r a c t i c e , s u g g e s t i n g t h a t each r e p e t i t i o n of a skilled

rrovement s l i g h t l y m o d i f i e s t h e m o t o r p rogram. I n t h i s

s e c t i o n motor l e a r n i n g is t r e a t e d a s a g r a d u a l p e r t u r b a t i o n

o f the coupling s t r e n g t h s b e t w e e n n e u r o n s i n t h e c e r e b e l l u m .

A l e a r n i n q a l g o r i t h m i s d e r i v e d which s e l e c t i v e 1 y m o d i f i e s

t h e i m p u l s e f i r i n q p a t t e r n s of t h e o u t p u t P u r k i n je ce l l s .

L e t $I,(+) b e t h 2 e f f e c t i v e membrane potentials of a l l

t h e n e u r o n s i n t h e c e r e b e l l u i n , i n c l u d i n g t h e i n t r i n s i c

fiber and c l i m b i n g f i b e r input Siring s a t e s with input

conplinq strengths Bkb and Ca r e s p e c t i v e l y . Since each

Purkin je ce l l r e c e i v e s a p p r o x i n a t e l y 100,000 synaptic

connections f rorr p a r a l l e l fibers, we c a n r e a s o n a b l y assame

t h a t t h e m e m b r a n e p o t e n t i a l s of P u r k i n j e cells a r e Gauss i an .

T h e n , following the c o n v e n t i o n s i n Part II, t h e c o v a r i a n c e s

s a t i s f y

I a n d the p a r t o f t h e c o r a r i a n c e s va(t) a r i s i n g from

climbing fiber inputs a l o n e s a t i s f y

H o w should s y n a p t i c s t r e n g t h s K ab b e a l t e r e d so

t h a t t h e o r i q i n a l membrane p o t e n t i a l c o v a r i a n c e s $: I

a r e a u g ~ e n t e d by a s l ra l l amount E ya(+) after l e a r n i n g

h a s o c c u r r e d ? I f the s y n a p t i c s t r e n g t h s were altered by a

small amount C Q b , then Eqs. ( 4 2 ) a n d ( 4 3 ) r e q u i r e , t o

f i r s t o r d e r i n E , t h a t

If t h e p l a s t i c s y n a p s e s a r e t h e o n e s be tween p a r a l l e l

fibers a n d P u r k i n j e ce l l s , t h e n t h e o n l y c o n t r i b u t i o n s t o

the right side o f E q . ( 4 4 ) a r e the o u t p u t c o v a r i a n c e s f r o a

t h e q r a n u l e c e l l s . If t h e membrane p o t e n t i a l s o f t h e

granule c e l l s have G a u s s i a n d i s t r i b u t i o n s , t h e n i n terms of

t h e f i r i n g r a t e s of t h e granule ce l l s

t h e c o v a r i a n c e s i n t h e p a r a l l e l f i b e r s c a n b e w r i t t e n

T h e c o n d i t i o n o n K2b in Eg. ( 44 ) becomes

D

where e i (t) a n d $': (t ) a re the input c o v a r i a n c e s t o

Purkinje c e l l s a r i s i n g respectively f r o m the climbing f i b e r s

a n d t h e p a r a l l e l f i b e r s , Thus, t h e c l i m b i n g - f i b e r i n p u t

c o v a r i a n c e s can be a s s o c i a t i v e l y s t o r e d by m a t c h i n g them t o

a l i n e a r c o m b i n a t i o n o f t h e p a r a l l e l - f i b e r input

c o v a r i a n c e ~ ~ However, t h e a l t e r a t i o n s i n t h e c o u p l i n g

s t r e n g t h s a r e f i x e d m o d i f i c a t i o n s , b u t the r i g h t and left

s i d e s of E q . ( 46 ) a r e a r b i t r a r y t e m p o r a l p r o c e s s e s . T h u s ,

o n l y an a p p r o x i m a t e Y_& can h e f o u n d .

I n t h e s t a t i o n a r y c a s e , a p a r t i c u l a r l y simple o p t i a a l

choice o f & a h i s t h a t w h i c h m i n i m i z e s t h e mean s q u a r e L -

e r r o r b e t w e e n t h e right a n d l e f t s i d e s of E q . (46)

f i x e d t ime

The minimum, which o c c u r s when t h e v a r i a t i o n o f

w i t h respect t o Q b v a n i s h e s ,

is i n d e p e n d e n t of tiiae. I n t h e n o n s t a t i o n a r y case, a ,

s imilar o p t i m a l s o l u t i o n can be f o u n d based on t h e

i n t e g r a t e d mean square e r r o r ( S e j n o u s k i , 1977a) .

T h e p l a s t i c i t y r e q u i r e d by Eq. ( 4 8 ) d e p e n d s on a p r i o r i

knovledqa o f the c o v a r i a n c e s and c a n n o t b e p r a c t i c a l l y

implemented w i t h n e u r o n s . S t o c h a s t i c a p p r o x i m a t i o n is a

constructive method f o r e s t i m a t i n g t h e o p t i m a l s o l u t i o n when

o n l y raw d a t a a re given (Tsypkin, 1973). The a d a p t i v e

l e a r n i n g a l g o r i thrn f o r the p r e s e n t p r o b l e m is

where \j i s a p o s i t i v e c o n s t a n t ( o r a n e g a t i v e c o n s t a n t

i f t h e a s s o c i a t i v e s t o r a g e i s c o m p l e m e n t a r y ) c o r r e s p o n d i n g

t o t h e s t r e n g t h of the p l a s t i c i t y . The r e s u l t f r o a Eqs,

( 4 7 ) and ( 4 9 ) i s

T h e present p r o b l e m d i f f e r s f r o m most a p p l i c a t i o n s of this

method i n t h a t t h e n o n l i n e a r background a s well a s t h e

l i n e a r system b e i n g o p t i n t i z e d depend o n the c o u p l i n g

s t r e n g t h s , T h i ~ d i f f i c u l t y w i l l n o t be dealt w i t h here; t h e

p r e s e n t t r e a t m e n t i s v a l i d o n l y f o r small. changes t o the

s y n a p t i c s t r e n g t h s , i n which case t h e s e c o n d term o n t h e

r i q h t side of E q . ( 5 0 ) is much s m a l l e r t h a n t h e f i r s t terra

a n d can b e n e g l e c t e d . Thus, the a p p r o x i m a t e l e a r n i n g

a l q o r i t h r n is

N o t i c e t h a t t h e i n t e g r a t e d f o r m o f Eq. (51) is p r o p o r t i o n a l

t o t h e f i r s t f a c t o r o f t h e o p t i m a l s o l u t i o n i n

E q . (48) ( P f a f f e l h u b e r , 1 9 7 5 ) , #hen o n l y raw d a t a a re g iven

the l e a r n i n q a l g o r i t h m i s

T h e c o v a r i a n c e on t h e right s i d e is t h e t e m p o r a l c o r r e l a t i o n

between (t) a n d rb (t) r e l a t i v e t o t h e p r o d u c t o f

their m e a n s - t h a t p a r t o f t h e c o r r e l a t i o n o w i n g t o

c h a n c e . The l e a r n i n g a l g o r i t h m r e q u i r e s t h a t a s i n g l e

c l i m b i n g f i b e r i n f l u e n c e a l l t h e a o d i f i a b l e s y n a p s e s on a n

e n t i r e P u r k i n j e c e l l . This r e q u i r e m e n t c o u l d b e s a t i s f i e d

s i n c e a c l i m b i n q f i b e r d o e s m a k e m u l t i p l e s y n a p t i c c o n t a c t s

o n all t h e m a j o r d e n d r i t i c branches o f a P u r k i n j e c e l l a n d

i s t h e r e f o r e i n a p o s i t i o n t o i n f l u e n c e a l l the p a r a l l e l

f i b e r s y n a p s e s .

Becanse t h e c o v a r i a n c e i n t h e l e a r n i n g a l g o r i t h m can b e

e i ther p o s i t i v e o r n e g a t i v e , t h e p l a s t i c s y n a p s e s h o u l d be

c a p a b l e of b o t h l o n g - t e r ~ f a c i l i t a t i o n a n d d e p r e s s i o n , I n

c o m p a r i s o r r , n a r r 9 s t h e o r y (1969) o n l y p r e d i c t s f a c i l i t a t i o n

when t h e r e i s a w c o n j u n c t i o n of p r e s y n a p t i c a n d c l i m b i n g

f i b e r ( o r p o s t - s y n a p t i c ) a c t i v i t y u ; s u c h a p l a s t i c s y n a p s e

e v e n t u a l l y r e a c h e s maximum s t r e n g t h t h r o u g h c h a n c e

c c i n c i d e n c e s , o r else l o s e s i n f o r m a t i o n b y n o n s p e c i f i c

d e c a y . Other t h e o r i e s f o r c e r e b e l l a r m o t o r l e a r n i n g ( A l b u s ,

3971; G i l b e r t , 1975) p r e d i c t o n l y w e a k e n i n g o f t h e p l a s t i c

124

s y n a p s e s ; s u c h a synapse e v e n t u a l l y r e a c h e s its ~ l i n i w u r n

s t r e n q t h t h r o u g h chance c o i n c i d e n c e s . T h e average strength

of t h e synapse p r e d i c t e d bere r e ~ a i n s c o n s t a n t when t h e

i n p u t s are uncorre la ted and c a n b e f l e x i b l y a d j u s t e d

a n y u h e s e v i t h i n i t s dynamic r a n g e when t h e inputs a r e

a p p r o p r i a t e l y c c r r e l a t e d (So j n o u s k i , 1977b) .

S y n a p t i c Plasticity

The forra o f t h e learning a l g o r i t h m i n E q . (52) s u g g e s t s

a mechanism f o r n o t o r l e a r n i n g i n t h e c e r e b e l l u m based o n

c a i n c i d e n c e s a n d a n t i c o i n c i d e n c e s b e t w e e n i m p u l s e s i n t h e

c l i m b i n g f i b e r a n d p a r a l l e l . f i b e r inputs. Suppose t h a t t h e

s y n a p t i c s t r e n g t h o f t h e p l a s t i c synapse i n c r e a s e s b y a

s x r a l l amount o( w h e n e v e r a c o i n c i d e n c e o c c u r s between a

d i s c h a r g e of t h e c l i m b i n g f i b e r a n d a p r e s y n a p t i c a c t i v a t i o n

o f t h e p l a s t i c synapse b y t h e p a r a l l e l f i b e r vithin a ti=

i n t e r v a l -71% /a . N e g l e c t i n g m u l t i p l e c o i n c i d e n c e s , t h e

accidental o r c h a n c e c o i n c i d e n c e rate f o r i n d e p e n d e n t l y

o c c u r i n q e v e n t s is q c R,PTC.L . w h e r e R c a n d R p are

r e s p e c t i v e l y the averaqe f i r i n g r a t e o f t h e c l i m b i n g fiber

a n d a v e r a q e a c t i v a t i o n r a t e of the p l a s t i c s y n a p s e by t h e

p a r a l l e l f i b e r . ( T h e s u b s c r i p t s t"cw a n d " p H w i l l a l w a y s

refer r e s p e c t i v e l y to t h e climbing fiber and the p a r a l l e l

fiber, f

C h a n c e s t r e n g t h e n i n g must be b a l a n c e d b y a mechanism

which v e a k e n s t h e s y n a p s e . For e x a m p l e , S t e n t (1973) has

postulated that n e u r o t r a n s w i t t e r r e c e p t o r s a r e rereoved f ro le

a p l a s t i c s y n a p s e when t h e p o s t s y n a p t i c aembrane r e g e a t e d l y

d i s c h a r g e s i n t h e a b s e n c e o f a p r e s y n a p t i c a c t i v a t i o n .

S t e n t f u r t h e r s u g g e s t e d t h a t t h e mechanism m i g h t a c c o u n t f o r

d e v e l o p m e n t a l p l a s t i c i t y i n t h e cat v i s u a l c o r t e x .

Assuma t h a t the p l a s t i c s y n a p s e d i m i n i s h e s i n strength

b y P whenever t h e c l i m b i n q fiber d i s c h a r g e s w i t h o u t an

a c t i v a t i o n of t h e p l a s t i c synapse by t h e p a r a l l e l fiber

w i t h i n T P / & . If m u l t i p l e c o i n c i d e n c e s within this t i m

i n t e r v a l are n e g l i g a b l e , t h e n the c o n d i t i o n f o r balance

b e t w e e n s t ~ e n g t h e n i n g and weakening ' o f t h e s y n a p t i c s t r e n g t h

when the inputs a r e u n c o r r e l a t e d is

#ith a p p r o p r i a t e l y c o r r e l a t e d i n p u t s the s y n a p t i c s t r e n g t h

c a n be a d j u s t e d a n y w h e r e b e t w e e n its ~axi raum a n d minimum.

If t h e balance c o n d i t i o n i s s a t i s f i e d , t h e n t h e p l a s t i c

s y n a p s e is m o d i f i e d whenever t h e c l i m b i n g f i b e r d i s c h a r g e s ;

w h e t h e r t h e s y n a p s e is s t r e n g t h e n e d o r weakened depends on

w h e t h e r o f n o t i t is a c t i v a t e d i n c o i n c i d e n c e with t h e

c l i m b i n q f i b e r . C o n s i d e r t h e c a s e when x p is

s u f f i c i e n t l y seal1 s o t h a t R p c p << 1. . Then t h e r a t i o o f

w e a k e n i n q t o s t r e n g t h e n i n q is, by E q . ( 5 3 ) , p r o p o r t i o n a l t o

t h e f i r i n q r a t e of t h e p l a s t i c s y n a p s e :

s u b s t a n c e which was p r e s e n t i n t h e s y n a p t i c d e f t in

p x o p o r t i o n t o t h e a v e r a g e a c t i v a t i o n r a t e of t h e synapse

( s u c h a s a d e g r a d a t i o n p r o d u c t o f t h e n e u r o t r a n s a i t t e r )

c o u l d s e r v e as a g e n t i n s a t i s f y i n g t h i s c o n d i t i o n f o r

balance,

An a l t e r n a t i v e way t o b a l a n c e t h e s t r e n g t h e n i n g a n d

w e a k e n i n g i s t o h a v e t h e s y n a p t i c s t r e n g t h diminish by

13 whenever the plastic s y n a p s e is a c t i v a t e d w i t h o u t a \

d i s c h a r g e of t h e c l i n b i n q f i b e r w i t h i n z C / 2 . T h e

c o r r e s p o n d i n g b a l a n c e c o n d i t i o n f o r u n c o r r e l a t e d i n p u t s i s

p ( , - R d c ) = 4 R c - C o c . (54)

T h e number o f c o i n c i d e n c e s a n d a n t i c o i n c i d e n c e s is

s u b j e c t t o r andon ; f l u c t u a t i o n . Fo r s t a t i o n a r y i n d e p e n d e n t

i n p u t s o v e r a time T , t h e s y n a p t i c s t r e n g t h , w i t h i n its

r a n q e , will u n d e r g o a r andom w a l k w i t h s t a n d a r d d e v i a t i o n I

E ( T ) = [ ( a A + * p ) Q C R P t s ~ j X ( 5 5

around i t s mean s t r e n g t h . Random f l u c t u a t i o n can be

m i n i m i z e d b y l i m i t i n g t h e t i m e d u r i n g w h i c h t h e p l a s t i c

synapse i s s e n s i t i v e t o m o d i f i c a t i o n . A s l o w a n d d i f f u s e

c h e m i c a l s y s t e m - s u c h a s t h e r e c e n t l y d i s c o v e r e d

n c r a d r e n e r g i c i n n e r v a t i o n o f t h e c e r e b e l l e r c o r k e x by

p r o j e c t i o n s from t h e l o c u s c o e r u l e u s (Olson and Fuxe, 1971;

Hoffer, S i q g i n s , O l i v e r , a n d Bloom, 1973) - c o u l d serve t o

r e q u l a t e p l a s t i c i t y and m i n i m i z e r andom f l u c t u a t i o n

(Kety, 1 9 7 0 ) ,

T h e m a g n i t u d e of random f l u c t u a t i o n s h o u l d b e compared

w i t h t h e a a x i n u m possible n o n r a n d o n p l a s t i c change of t h e

s y n a p t i c s t r e n g t h . I f e v e r y c l i m b i n g - f i b e r i m p a l s e we re t o

o c c u r i n c o i n c i d e n c e w i t h an a c t i v a t i o n o f t h e p l a s t i c

synapse b y the p a r a l l e l fiber, t h e n t h e maximum

s t r e n g t h e n i n q of t h e s y n a p t i c s t r e n g t h o v e r a time would

T h u s , t h e r a t i o of r a n d o m f l u c t u a t i o n t o maximum c h a n g e over

where t h e n a t u r a l t i n e scale is set by

L e t u s a p p l y these g e n e r a l s t a t i s t i c a l c o n s t r a i n t s

s p e c i f i c a l l y t o t h e cerebellum. Under t y p i c a l c o n d i t i o n s

t h e firing r a t e of a ~ a r a l l e l f i b e r i s R p -- 5O/sec a n d t h e

f i r i n g r a t e o f a c l i m b i n g f i b e r i s Rc I l/sec. If we

a s s u m e that t h e c o i n c i d e n c e "window" f o r s t r e n y t h e n i n g is

rx - 2 msec [ c o m p a r a b l e t o t h e time c o u r s e of an a c t i o n

p o t e n t i a l ) , t h e n t h e a c c i d e n t a l c o i n c i d e n c e rate is

R, R p E N 0 .l/sec; t h a t is, a b o u t o n c e i n e v e r y 10 sec.

If 2. i< 20 msec ( s o t h a t R p zp << 1 ) , t h e n t h e b a l a n c e P

c o n d i t i o n (53) becomes P/g = 0.1 ; t h a t is, t h e s i z e o f a

decre iaen t f rom a s i n g l e a n t i c o i n c i d e n c e must be

a p p s s x f n a t e l y c a e - t e n t h t h e s i z e o f a n i n c r e m e n t from a

s i n q l e c o i n c i d e n c e . Under t h e same s t e a d y - s t a t e conditions

t h e n a t u r a l time s c a l e is T* = 0,1 sec , v h i c h is t h e minimum

time r e q u i r e d f o r t h e nonrandon c h a n g e t o e q u a l t h e random

f l u c t u a t i o n . T h e t i m e scale f o r learning is t y p i c a l l y

b e t v e e n 10 s e c and 1,000 sec, d u r i n g which t h e randorn

f l u c t u a t i o n i s ketveen 1OaL and 1 % of t h e maximum nonrandom

c h a n g e , I f a plastic s y n a p s e v e r e a l w a y s s e n s i t i v e t o

t n o d i f i c a t i o n , t h e n t b e r a n d o a f l u c t u a t i o n would be a

c u m u l a t i v e , l o n g - t e r m noise problem.

I f one o f the above b a l a n c e c o n d i t i o n s i s s a t i s f i e d ,

t h e n t h e a v e r a g e s t r e n g t h of t h e p l a s t i c s y n a p s e c a n n o t be

a l t e r e d b y s i m p l y i n c r e a s i n g t h e f i r i n g r a t e o f t h e c l i m b i n g

f i b e r o r t h e p a r a l l e l f i b e r : t h e s t r e n g t h o f t h e s y n a p s e can

b e s y s t e m a t i c a L l y a l t e r e d o n l y . t h r o u g h c o r r e l a t e d

d i s c h a r g e s , a n d t h e s y n a p s e i s t h e n e i t h e r s t r e n g t h e n e d o r

ueakened d e p e n d i n g o n t h e s i g n o f t h e c o v a r i a n c e . A d i r e c t

t e s t o f the p r e d i c t e d s y n a p t i c p l a s t i c i t y i n v e r t e b r a t e s is

a t p r e s e n t a d i f f i c u l t e x p e r i m e n t ; however, t h e g e n e r a l

s t a t i s t i c a l c o n s t r a i n t s c o n s i d e r e d above a p p l y e q u a l l y well

t o ' i n v e r t e b r a t e s , and s i m i l a r p l a s t i c i t y may o c c u r i n a more

f a v o r a b l e p r e p a r a t i o n .

An example of t h e t y p e of p l a s t i c i t y p r e d i c t e d i n t h e

c e r e b e l l u m h a s k e e n found i n ApZysia (Kandel and Tanc, 7965;

\

S h i m a h a r a a n d T a n c , 1 9 7 6 ; ~ a n d e h , 19761. The s t rength of an

i d e n t i f i e d Eynapse h a s been shown t o facilitate and t o

remain e l e v a t e d f o r many n i n u t e s a f t e r an i d e n t i f i e d

c o n d i t i o n i n g n e u r o n h a s been a c t i v a t e d , Howevet, t h e

d e m o n s t r a t e d he t e r o s y n a p t i c f a c i l i t a t i o n is n o t c o n t i n g e n t

upon p a i r i n g of the s y n a p t i c a c t i v a t i o n vith t h e

c o n d i t i o n i n q s t i m u l u s : a c t i v a t i o n o f t h e c o n d i t i o n i n g n e u r o n

a l o n e p r o d u c e d the same f a c i l i t a t i o n . I n t h i s r e s p e c t t h e

f a c i l i t a t i o n d e m o n s t r a t e d i n A p Z y s i a differs fron t h a t

p r e d i c t e d i n t h e c e r e b e l l n m , which d e p e n d s . o n v h e t h e r o r n o t

the c l i n b i n g f i b e r d i s c h a r g e i s p a i r e d v i t h a p a r a l l e l f i b e r

a c t i v a t i o c o f the p l a s t i c s y s n a p s e .

H o t o r l e a r r i i n g i n the c e r e b e l l u m r a s t r e a t e d as a

g r a d u a l a d a p t a t i o n o f t h e c o r t i c a l f i l t e r i n g p r o p e r t i e s : 3

i n p u t c o v a r i a n c e asuong t h e c l i ~ b i n g f i b e r s was stored i n

a s s o c i a t i o n v i t h t h e i n p u t c o v a r i a n c e among the mossy

f i b e r s , a p r o c e s s w h i c h w i l l be c a l l e d c o v r i a n c e s t o r a g e . P A s i m i l a r process may o c c u r e l s e w h e r e i n t h e n e r v o u s system

and may u n d e r l i e o t h e r forms o f l e a r n i n g a n d memroy.

F i b e r s s i m i l a r i n a p p e a r a n c e t o c l i m b i n g f i b e r s h a v e

been f o u n d i n c e r e t r a l c o r t e x by Cajal ( 1 9 1 1 ) , and more

recently ~ z e n t s g o t h a i (1969) h a s v r i t t e n that " i n t h e G o l g i

picture it i s q u i t e cornmn t o see a number o f fine t e r m i n a l

axons r u n n i n g f c r c o n s i d e r a b l e distances c l o s e 1 y a s s o c i a t e d

i n t o b u n d l e s w h i c h i n aany c a s e s c a n b e s e e n t o c o n t a i n a

d e n d r i t i c shaft i n t h e i r a x e s . * * Thus, t h e l e a r n i n g

a l q o r i t h m (52 ) c o u l d a l s o b e used t o s t s s e c o v a r i a n c e i n

c e r e b r a l c o r t e x .

There a r e , however, major d i f f e r e n c e s b e t ween t h e

c e r e b e l l a r c o r t e x a n d the c e r e b r a l cortex which vou ld have

t o b e taken i n t o a c c o u n t b y a d e t a i l e d model . I n t h e

c e r e b e l l u m , t h e membrane -po teo t i a l c o v a r i a n c e s o f g r a n u l e

cells depend m a i n l y on t h e a o s s y f i b e r s and v e r y l i t t l e on

t h e c l i m b i n g f i b e r s . As a consequence , t h e l e a r n i n g

a l g o r i t h m s i n p l y a s s o c i a t e s t h e c o v a r i a n c e s a l o n g t h e s e two

i n p u t s . I n a more h i g h l y i n t e r c o n n e c t e d a r e a , s u c h as

c e r e b r a l c o r t e x , the c l i z b i n g f i b e r i n p u t c o u l d i n f l u e n c e

neurons which t h e m s e l v e s a p p e a r on t h e r i g h t s i d e of

E q . (46) . Al though t h e presumed p l a s t i c s y n a p s e s be tween p a r a l l e l

f i b e r s a n d P u r k i n j e c e l l s i n t h e c e r e b e l l u m a r e a l l

e x c i t a t o r y , p l a s t i c s y n a p s e s elsewhere might b e i n h i b i t o r y .

F a r i n h i b i t o r y s y n a p s e s t h e d i r e c t i o n o f t h e plastic change

p r e d i c t e d b y t h e l s a r n i n q algorithm is r e v e r s e d : p o s i t i v e

c o v a r i a n c e would weaken t h e s y n a p s a and n e g a t i v e c o v a r i a n c e

would s t r e n g t k e n it.

A n o t h e r d i f f e r e n c e b e t ween c e r e b e l l a r c o r t e x and

c e r e b r a l cortex i s reflected i n t h e i r p r o c e s s i n g tiae

scales. The c e r e b e l l u m p a r t i c i p a t e s i n b a l l i s t i c m a v e a e n t s

s u c h a s saccadic e y e movements. Long r e v e r b e r a t i o n s would

n c t b e h e l p f u l a n d might even i n t e r f e r e w i t h t h e precise

t irninq r e q u i r e d f o r f i n e c o n t r o l . Parts o f the cerebral

c o r t e x are c o n c e r n e d w i t h c o o r d i n a t i o n on a l o n g e r time

s c a l e a n d c a n be e x p e c t e d t o h a v e a c o r r e s p o n d i n g l y longer

c o h e r e n c e time for c o r r e l a t i o n s b e t w e e n membrane p o t e n t i a l s .

The long- term s t o r a g e o f membrane p o t e n t i a l c o v a r i a n c e

was t r e a t e d l i n e a r l y , b a t the model i s essentially

n o n l i n e a r ; o n e c o n s e q u e n c e is t h a t u n d e r some conditions

small changes i n t h e c o u p l i n g s t r e n g t h s m i g h t result i n a

larqe change in t h ~ b a c k g r o u n d f i r i n g rates (for e x a m p l e , if

t h e s t e a d y - s t a t e e q u a t i o n s t u d i e d i n P a r t 11 were t o

b i f u r c a t e t o a new s o l u t i o n b r a n c h ) .

The s t a b i l i t y o f the s t a t i o n a r y c o v a r i a n c e e q u a t i o n t o

s m a l l chanqes i n t h e c o u p l i n g strengths c a n b e t r e a t e d

a n a l y t i c a l l y . The s o l u t i o n g i v e n by E q . (32) d e p e n d s o n t h e

t r a n s i t i o n matrix whose L a p l a c e t r a n s £ orm i s

/ where A E = K f - L and t h e i n t e r a c t i o n m a t r i x is

assumed t o depend analytically on E . T h e t r a n s i t i o n

matrix can b e r e c o v e r e d by t h e i n v e r s e L a p l a c e t r a n s f o r m

where the c o n t o u r is p a r a l l e l t o t h e imag ina ry axis b

a n d l i e s t o the r i g h t o f a l l s i n g n l a r i t i e s . F o r a

s u f f i c i e n t l y s i r a l l t h e i n t e g r a n d o f E q . (57) is

a n a l y t i c everywhere on t h e c o n t o u r (Xato, 1966) .

Hence, a r o u n d e v e r y s o l u t i o n t h e r e is a n o p e n ne ighborhood

c o n t a i n i n g o t h e r s o l u t i o n s ,

I f the r e s o l v e n t i s n o n s i n g u l a r i n t h e r i g h t h a l f of

t h e complex s - p l a n e , then t h e s o l u t i o n is a s y o p t o t i c a l l y

s t a b l e . Bovever , i f t h e r e s o l v e n t is n e a r l y s i n g u l a r o n the

i n a g i n a r y a x i s , t h e n t h e s o l u t i o n i s nearly u n s t a b l e and

small but fin it^ c h a n g e s i n t h e c o u p l i n g s t r e n g t h s could

lead t o a ve ry l a r g e change i n t h e b e h a v i o r o f the sys tea .

T h i s c o n d i t i o n c o r r e s p o n d s t o the necessary c o n d i t i . c n given

i n Part I1 f o r t h e n o n l i n e a r s t eady-s ta te equation [8) t o be

near a b i f u r c a t i o n p c i n t . Hence, i n h e r e n t i n the n o n l i n e a r

model i n a d d i t i o n t o t h e s l o w l y - a d a p t i n g c o n t i n u o u s c h a n g e

t r e a t e d i n t h i s p a r t is the p o s s i b i l i t y o f d i s c o n t i n u o u s

change ,

D i s c u s s i o n

No model o f i n t e r a c t i n g neurons h a s y e t succeeded i n

r e l a t i n g t h e brain s t r u c t u r e o f a v e r t e b r a t e t o i ts

b e h a v i o r . Because t h e p r e s e n t work v a s i n s p i r e d primarily

b y c u r r e n t e x p e r i m e n t a l p rob lems and was based on c o n c e p t s

d e r i v e d f rom c u r r e n t e x p e r i m e n t a l p r o c e d u r e s , there is at

l e a s t the hope that t h e o r y a n d p r a c t i c e w i l l b e n e f i t from

one a n o t h e r a s t h e y h a v e i n o t h e r a r e a s o f s c i e n c e .

Part 11, which was mainly c o n c e r n e d w i t h a v e r a g e f i r i n g

rates, m a k e s t h e c l o s e s t c o n t a c t w i t h current e x p e r i m e n t a l

work on s i n g l e - n e u r o n r e c o r d i n g i n v a r t e b r a t e s . The model

of s t e r e o p s i s examined i n t h i s part c a n b e t e s t e d w i t h

a v a i l a b l e e x p ~ r i n e n t a l t e c h n i q u e s . A s e m p h a s i s shifts

t oward r e l a t i n g complex s e q u e n c e s o f s e n s o r y a n d moto r

p a t t e r n s t o neu ron f i r i n g p a t t e r n s , t e a p o r a l and s p a t i a l

c o r r e l a t i o n s s h o u l d become a o r e common e x p e r i m e n t a l

measurements . T h e a n a l y s i s o f c o r r e l a t i o n s be tween membrane

p o t e n t i a l s i n s i m p l e models, such as t h a t i n Part III, may

b e h e l p f u l i n i n t e r p r e t i n g e x p e r i m e n t a l measurements. T h e

n e x t step t o w a r d d e s c r i b i n q b i o l o g i c a l r e a l i t y is t h e

d e t a i l e d niodel inq o f s p e c i f i c b r a i n a r e a s , s u c h a s t h e

a t t e m p t i n P a r t I V t o s t u d y motor learning i n t h e

c e r e b e l l u m .

The c e n t r a l r e s u l t of this research program, of

interest i n i t s e l f , a r o s e i n the analysis of colaarianee

p r o c e s s i n g . A1 though t h e model i s strongly n o n l i n e a r , the

c o v a r i a n c e e q u a t i o n h a s an e x a c t l y l i n e a r f o r m when ~lerabraxle

p c t e n t i a l s a r e Gaussian. I f some p a r t o f the n e r v o u s systeel

t a k e s a d v a n t a q e of s t a t i s t i c a l l i n e a r i t y , then t h e membrane

p o t e n t i a l s of n e u r o n s i n t h a t a r e a should have a Gaussian

distribn t i o n . T h i s p o s s i b i l i t y c a n b e tested with

i n t r a c e l l u l a r r e c o r d i n g s .

L i n s a r i t y has f a r - r e a c h i n g consequences which are

nowhere more clearly seen t h a n i n a l i n e a r t h e o r y l i k e

quantuni mechanics. For e x a ~ p l e , b e c a u s e t h e S c h r o e d i n g e r

equation for a quanturr system i s l i n e a r , i t s s o l n t i o n s can

be c l a s i f i e d k y its symmetry group (Higne r , 1959) .

Similarly, t h e s o l u t i o n s o f t h e l i n e a r c o v a r i a n c e e q u a t i o n

can be c l a s s i f i e d b y the s y n n e t r y g r o u p o f the interaction

matrix. Such a c l a s s i f i c a t i o n i s o f g r e a t help i n

m a t h e m a t i c a l l y a n a l y z i n g t h e model; b u t more i m p o r t a n t l y ,

the c l a s s i f i c a t i o n c o u l d b e used by the brain itself i n

d e t e c t i n q and a n a l y z i n g complex p a t t e r n s .

P h y s i o l o q i c a l l y realistic models o f i n t e r a c t i n g n e u r o n s

a r e d i f f i c u l t t o analyze b e c a u s e o f i n t r i n s i c

n o n l i n e a r i t i e s . Bore p r o g r e s s h a s been made i n a n a l y z i n g

s o l v a b l e l i n e a r w d e l s (Kohonen, 1 9 7 7 ) , b u t this work is

relatively remote f rom e x p e r i m e n t . T h e p r e sen t stochastic

model o f n o n l i n e a r l y i n t e r a c t i n g neurons corn b i n e s

c o o p e r a t i v e n o n l i n e a r i t y , which g o v e r n s the a v e r a g e peabrane

p c t e n t i a l s , m i th l i n e a r p r o c e s s i n g , which o c c u r s a t t h e

l e v e l o f membrane p o t e n t i a l c o v a r i a n c e s . B e c a u s e of this

n o v e l s y n t h e s i s many d i f f e r e n t l i n e a r systems can be

embedded i n a s i a g l e c o l l e c t i o n of n o n l i n e a r l y i n t e r a c t i n g

neurons . H i t h i n s u c h a " n o n l i n e a r l i n e a r filter"

i n f o r m a t i o n c a n be s e l e c t i v e l y st or ed and r e t r i e v e d r e l a t i v e

t o a background c o n t a x t . The p r e d i c t i o n t h a t i n f o r m a t i o n is

p r o c e s s e d a s c o r r e l a t i o n s between membrane p o t e n t i a l s c a n be

e x p e r i n e n t a l l y t e s t e d w i t h i n t r a c e l l u l a r recordings from

n e i g h b o r i n g p a i r s o f neu rons , I n the c e r e b r a l c o r t e x , f o r

example, s t r o n g ly i n t e r acting cells a r e l o c a t e d in v e r t i c a l

n c o l u m s l f v i t h r e l a t i v e l y weak h o r i z o n t a l i n t e r a c t i o n

Between n e i g h b c r i n g l tcolumnsv ( H o u n t c a s t l e , 1978) . T h e

r e s u l t s of this s t u d y s u g q e s t t h a t t h e membrane p o t e n t i a l s

o f n e u r o n s w i t h i n a c o r t i c a l ucolurnntl a r e h i g h l y c o r r e l a t e d

w i t h e a c h o t h e r and m i t h t h e s u b c o r t i c a l n e u r o n s t o which

they a r e r e c i p r o c a l l y c o n n e c t e d .

A neu ron whose r a t e of f i r i n g i s e i t h e r very low o r

v e r y h i g h i s i n s e n s i t i v e t o i n p u t c o r r e l a t i o n s . I n a h i g h l y

i n t e r c o n n e c t e d a r e a , such as c e r e b r a l c o r t e x , t h e s u b s e t of

n e u r o n s w i t h i n t e r m e d i a t e f i r i n g rates forms a " s k e l e t o n

ne tworkw o f l i n e a r l y - i n t e r a c t i n g neurons. A n e u r o n i n

v i s u a l c o r t e x , f o r example, firing briskly i n r e s p o n s e t o

i ts " t r i q g e r f e a t u r e t t i n t h e v i s u a l field, would p robab ly

c o n t r i b u t e t o t h e ' f s k e l e t o n networkw i f i t s f i r i n g r a t e were

not n e a r i t s aaximum. Under natural c o n d i t i o n s e a c h v i s n a l

s c e n e p i c k s o u t a d i f f e r e n t s u b s e t o f ' I t r i g g e r f e a t u r e s f g ,

and h e n c e a d i f f e r e n t " s k e l e t o n n e t w o r k w , The role o f

"triqcrer f e a t uresti i n a ? s k e l e t o n networkn processing

i n f o r n a t i o n i n p a r a l l e l is a n a l t e r n a t i v e t o t h e r o l e of

" t r i q g e r f e a t u r e s t f i n a s e r i a l h i e r a r c h y o f i n c r e a s i n g l y

s p e c i f i c " f e a t u r e d e t e c t o r s t f ( B a r l o w , 1968) . A l t h o u g h t h e s e r i a l c o m p u t a t i o n i n a d i g i t a l c o m p a t e r

i s v e r y d i f f e r e n t f r o m t h e p a r a l l e l c o m p u t a t i o n i n t h e

n e r v o u s s y s t e m , t h e d i s t i n c t i o n a a d e b e t w e e n h a r d w a r e a n d

s o f t w a r e c a n a l s o b e a p p l i e d t o the b r a i n , T h e p r e s e n t

study was a i m e d a t t h e i n t e r m e d i a t e l e v e l b e t w e e n n e u r a l

h a r d w a r e and b e h a v i o r a l s o f t u a r e w h i c h i n a d i g i t a l c o m p u t e r

is c a l l e d t h e i n t e r n a l " m a c h i n e l a n g u a g e i t , One resu l t i s a

c a n d i d a t e f o r t h e b r a i n ' s " m a c h i n e l a n g u a g e M : l i n e a r s y s t e m s

t h e o r y , w h i c h h a s w i d e s p r e a d a p p l i c a t i o n i n s y s t e m s

e n g i n e e r i n g , p l a y s an u n e x p e c t e d r o l e i n t h e n o n l i n e a r m o d e l

a n d nay a l s o have a s i g n i f i c a n t r o l e i n t h e o p e r a t i o n o f the

b r a i n ,

Despite t h e f o r r i d a b l e c o m p l e x i t y of t h e n e r v o u s

s y s t e m , i t s d e s i g n p r i n c i p l e s n e e d n o t b e c o m p l i c a t e d , J u s t

a s t h e d e s i g n p r i n c i p l e o f r e p r o d u c t i o n , t h e r e p l i c a t i o n o f

D N A , was d i s o v e r e d i n s p i t e of complex d e t a i l s not y e t

f u l l y u n d e r s t o o d , t h s b a s i c d e s i g n p r i n c i p l e s o f t h e b r a i n

n i g h t a l s o b e d i s c o v e r a b l e . T h e n e r v o u s systerp is a

s o l u t i o n t o numerous b i o l o g i c a l . problems i n the

c o r m u n i c a t i o n of i n f o r m a t i o n a n d t h e c o n t r o l of physical

systeas - p r o b l e n s which c o n f r o n t e d the d i s t a n t ances tors

of e x i s t i n g animals . S y s t e m s engineers confronted w i t h

s i m i l a r problems h a v e a l s o f o u n d some s o l u t i o n s , I t is

p e r h a p s c o i n c i d e n t a l that the m a t h e m a t i c a l analysis of a

s i m p l e neural moael s h o u l d r e s u l t i n e q u a t i o n s which a re

fcrmally i d e n t i c a l t o o n e s with which systems engineers are

f a ia i l i a r . This p a r a l l e l c o u l d nevertheless prove useful

both i n a n a l g z i n q e x p e r i m e n t a l d a t a a n d i n u n d e r s t a n d i n g the

design p r i n c i p l e s of t h e nervous s y s t e m .

Appendix

P o i n t - P r o c e s s H o d e l

T h e r e s u l t s s o f a r a r e b a s e d on a n o n l i n e a r model with

c o n t i n u o u s v a r i a b l e s . T h e c o n t i n u o u s model is derived h e r e

from a p o i n t - p r o c e s s model, and the v a r i a b l e s i n b o t h n o d e l s

a re q i v e n p r e c i s e i n t e r p r e t a t i o n s ,

A spike t r a i n , reqarded a s a sequence of p o i n t s in

time, can be modeled b y a stochastic p o i n t process on t h e

real l i n e (Levis, 1972) . L e t fd (t) represent the number

of spikes on t i n e i n t e r v a l 10, T ) . Assume t h a t t h e mean

number o f spikes E N ( t ) i s d i f f e r e n t i a b l e , a n d define

the "instantaneous a v e r a g e ra te t t

For example , c o n s i d e r t h e case w h e a the p o i n t process

i s P o i s s o n , T h e n t h e p r o b a b i l i t y t h a t t h e r e are ?.

spikes on t h e i n t e r v a l [ o , T ) is

w i t h

S i n c e the mean number o f spikes on the i n t e r v a l i s

the l l i n s t a n t a n e o u s a v e r a g e r a t e i * i s y k ) = ~ ( t k

T h e n i n s t a n t a n e o n s a v e r a g e rate" is related t o t h e

ninstantaneous zateBf, a n experimental variable which h a s

been u s e f u l in m e a s u r i n g t h e dynamic r e s p o n s e of t h e Limulus

retina (Knight, Toyoda & Dodge, 1970) . Given a spike t r a i n

with s p i k e s at t imes {ti) , the Nins t an t anaous r a t e v q is

defined 3 s

It i s a p p a r e n t from Fiq, 3 5 t h a t

0

v h a r e t h ; . errcr is bounded 1 ) and E E l t ) = O .

C o n s e q u e n t l y , the a y e r a g 2 of the l l i n s t a n t a n e o u s s a t e f t o v e r

an ensemble of i n d e p e n d e n t b u t i d e n t i c a l l y p r e p a r e d trials

g i v e s an estimate f o r t h e I f i n s t a n t a n e o u s average raten

If the t r a i l s i n t h e e n s e m b l e are n o t i n d e p e n d e n t , then

t h i s result may n o t be v a l i d . For e x a m p l e , when a single

eccentric c e l l i n t h e LimuZus r e t i n a is i ; l l u m i n a t e d by light

u i t h a s i n u s o i d a l l y moduf a t e d component, the response

h i s t o g r a m of t h e cell ave raqed o v e r the a o d u l a t i o n cycle is

d i f f e r e n t f r o m t h e a v e r a g e of the v t i n s t a n t a n e o u s rateH, and

particularly s o w h e n t h e m o d u l a t i o n frequency is n e a r t h e

a v e r a g e f i r i n g r a t e ( K n i g h t , 1972a , b ) . However, t h e

ensemble from which t h e response h i s tog ra ra derives i s not

composed of i n d e p e n d e n t t r i a l s since the responses o v e r

s u c c e s s i v e c y c l e s a r e h i g h l y d e p e n d e n t ,

The ftef f e c t i v e memkrane p o t e n t i a i " w h i c h was mot ivated

i n P a r t I c a n now b e more p r e c i s e l y d e f i n e d , L e t W H ) b e t h e meffbrane p o t e n t i a l o f a neuron which d o e s n o t p roduce

a n a c t i o n p o t e n t i a l b u t which receives a s p i k e train a s

i n p u t , T h e n t h e [tl~nibrane p o t e n t i a l s h o u l d s a t i s f y the

stochastic e q u a t i o n

where 8 i s t h o jump i n m a g n i t u d e o f t h e p o s t s y n a p t i c

p o t e n t i a l from a s i n g l e s y n a p t i c e v e n t . The d e r i v a t i v e of

t h e counting f u r c t i o n dN(-() is d e f i n e d by T

where [ t i ) U

a r e t h e times when i m p u l s e s occu r on

LO,T) . If F i t ) is a s t o c h a s t i c f u n c t i o n then t h e

i n t e g r a l must be i n t e r p r e t e d a c c o r d i n g t o t h e I t o c a l c u l u s

( S n y d e r , 1975) ,

The effect o f spike p r o d u c t i o n on t h e membrane

p o t e n t i a l i s shown i n Pig. 36. T h e d i f f e r e n c e b e t u e e n t h e

membrane p o t e n t i a l w i t h a n d w i t h o u t r e s e t damps

e x p o n e n t i a l l y after each impulse. The a v e r a g e of Eq. (61 )

o v e r an ensemble of p o i n t p r o c e s s e s is

where t h e a v e r a q e membrane p o t e n t i a l

14 1

F i q . 35. A typical spike t r a i n S (t) as a ftlnction

o f time, t h e c u m u l a t i v e nuiaber of spikes N (t) , and the

" i n s t a n t s n e o u s ra te t t (*) , d e f i n e d b y Eg. (59) (fro.

Se j n o w s k i , 1 9 7 7 a ) .

Fig. 3 6 . Idealized i n t r a c e l l u l a r r e c o r d i n g of t h e

membrane p o t e n t i a l (f) as a f u n c t i o n of t i m e . Upon

r e a c h i n g t h r e s h o l d , a n i n p u l s e is r e l e a s e d a n d t h e membrane

p o t e n t i a l i s reset t o a l o w e r p o t e n t i a l . I n c o n t r a s t , t h e

e f f e c t i v e men b r a n e p o t e n t i a l Y / ( t ) is c o n t i n u o u s . The

d i f f e r e n c e y&)-J(f) s u f f e r s a s t e p d i s c o n t i n u i t y

w h i c h danps e x p c n e n t i a l l y ( f r o m Sejnouski, 1 9 7 7 a ) .

corresponds t o the " e f f e c t i v e a e ~ i b r a n e p o t e n t i a L u a o t i v a t e d

i n P a r t I. T h e r e s e ~ l b l a n c e be tween E q . (62) a n d t h e

c c n t i n u o u s model ( 2 ) f u r t h e r suggests t h a t ( $ ) p r e v i o u s l y defined a s the f i r i n g r a t e t o a c o n s t a n t i n p u t

c u r r e n t , s h o u l d b e i n t e r p r e t e d a s a n " i n s t a n t a n e o u s average

r a t e . " T h u s , a n a t u r a l g e n e r a l i z a t i o n of the c o n t i n n o u s i

model is g i v e n t p t h e s t o c h a s t i c i n t e g r a l e q u a t i o n

w h e r e tha p o i n t p r o c e s s e s d M b a n d d u b s a t i s f y

The k e r n e l K a o ( t , 5 ) i s t h e t e m p o r a l r e s p o n s e o f (t) t o an i m p u l s e a t t i n e 3 f r o m a n e i q h b o r i n g neu ron , and

B a t ( T , 5 ) i s t h e r e s p o n s e f rom an e x t e r n a l input. S i n c e

va(+) and T b (e) a r e t h e m s e l v e s s t o c h a s t i c p r o c e s s e s ,

the p o i n t p r o c e s s e s i n Eq. ( 6 3 ) a r e d o u b l y s t o c h a s t i c

(Snyder , 1975) . Lf a neuron r e c e i v e s aany i n d e p e n d e n t

impulse i n p u t s , then by a c e n t r a l l i m i t t heo rem i ts membrane

p o t e n t i a l i s a ~ p r o x i m a t e l y G a u s s i a n . For example, the

p o i n t - p r o c e s s model i n Eq, ( 6 3 ) w i t h P o i s s o n i m p u l s e

q e n e r a t o r s s a t i s f i e s t h e c o n d i t i o n s of t h e c e n t r a l l i m i t

t heo rem and t h e membrane p o t e n t i a l s Y/,(t ) s a t i s f y a

l i n e a r c o v a r i a n c e e q u a t i o n s i m i l a r t o t h e one f o r t h e

c o n t i n u o u s model.

The t i a e - d e p e n d e n t c ~ u p l i n g kesnels i n this

p o i n t - p r o c e s s model take i n t o account t h e electrical

properties o f the intervening axons, synapses, and

d e n d r i t e s , including delays and decreaental decay. The

special case

corresponds to the simple model o f exponential decay

considered i n p a r t I.

A i d l e y , D, J.: The p h y s i o l o g y of e x c i t a b l e cells. Cambridqe: C a m b r i d g e U n i v e r s i t p Press 1971

A d r i a n , 3 . D.: The b a s i s o f s e n s a t i o n , London: C h r i s t o p h e r s 1 9 2 8 .

A l b u s , J, S.: A t h e o r y o f c e r b e l l a r f u n c t i o n . H a t h . B i o s c , 2, 25-61 (1971)

Amass ian , V . H . , G i b l i n , D, : P e r i o d i c c o m p o n e n t s i n s t e a d y - s t a t e a c t i v i t y o f c u n e a t e n e u r o n e s a n d their p o s s i b l e role i n s e n s o r y c o d i n g . J. P h y s i o l . 243, 353-385 (1974)

A n d e r s o n , J, A,: A s i m p l e n e u r a l n e t w o r k g e n e r a t i n g an i n t e r a c t i n g memory. B a t h , B i o s c i e n c e 14, 197-220 (1 972)

0 ~ s t r o m , K . J, : I n t r o d u c t i o n t o s t o c h a s t i c c o n t r o l t h e o r y ,

N e w York: Academic Press 1 9 7 0

B a r l o w , H . B . : S i n g l e u n i t s a n d s e n s a t i o n : a n e u r o n d o c t r i n e f o r p e r c e p t u a l p s y c h o l o g y ? P e r c e p t i o n 1, 37 1-394 ( 1 972)

Bar low, 8. B , , L e v i c k , #. R.: The nechanisrrm o f d i r e c t i o n a l l y s e l e c t i v e u n i t s i n r a b b i t ' s r e t i n a . 3 . P h y s i o l . 1 2 , 477-504 (1 965)

Bar low, R . E . : I n h i b i t o r y f i e l d s i n t h e LimuZus l a t e r a l eye . J. Gen. P h y s i o l . 54, 383-396 ( 1 9 6 9 )

B e l l , C . C . , Kawasak i , T,: R e l a t i o n among c l i m b i n g f i b e r r e s p o n s e s cf nearby P u r k i n j e c e l l s . J. M e u r o p h y s i o l . 3 5 , 155-169 (1972) -

B e r q e r , H. S. : N o n l i n e a r i t y a n d f u n c t i o n a l a n a l y s i s . N e w York: Academic Press 1977

B i r k h o f f , G . , R o t a , B.-C.: Ordinary d i f f e r e n t i a l e q u a t i o n s . 2d ed . Ual tham, Mass.: B l a i s d e l l Pub. Co. 1 9 6 9

B i s h o p , G, H.: Natural h i s t o r y o f t h e n e r v e i m p u l s e , P h y s i o l . R e v . 36, 376-399 (1956)

B i s h o p , P. 0. : E e g i n n i n g o f form v i s i o n a n d b i n o c u l a r d e p t h

d i s c r i a i n a t i o n i n c o r t e x . I n : S c h n i t t , 2. 0. (Ed-) : T h e n e n r o s c i e n c e s : Second s t u d y prograB, New P o r k : T h e R o c k e f e l l e r Press 1970

Bough, E. 9 , : S t e r e o s c o p i c v i s i o n i n the macaque monkey: a b e h a v i o r a l d e m o n s t r a t i o n , N a t u r e 225, 42-44 (19703

B r i n d l e y , G , : The use aade by the c e r e b e l l u m o f t h e i n f o r n a t i o n t h a t it r e c e i v e s f rom sense o r g a n s . Xnt. B r a i n Res. Org. B u l l . 3, 80 (19b4)

B u c y , El. S , , J o s e p h , P. D.: F i l t e r i n g f o r s t o c h a s t i c p r o c e s s s s w i t h a p p l i c a t i o n t o g u i d a n c e - N e w York: W i l e y - I n t e x s c i e n c e 1968

B u l l o c k , T . H.: Neuron d o c t r i n e a n d e l e c t r o p h y s i o l o g y . S c i e n c e 129, 997-1002 (1959)

Bullock, T. H . : I n t r o d u c t i o n t o nervous systeas. S a n F r a n c i s c o : W. 8 . Freeman & Co. 1977

Bussgang, J. 3 .: C r o s s c o r r e l a t i o n f u n c t i o n s o f a m p l i t u d e d i s t o r t e d G a u s s i a n s i g n a l s . Res. Lab, Elec. HIT Tech. Rep. 216, 1-14 (1952)

C a r p e n t e r , !I. B. : Human Neuroanatomy. Baltimore: The W i l l i a m s & W i l k i n s Co. 1976

Cooke, I , L i p k i n , H.: C e l l u l a r n e u r o p h y s i o l o g y . New York: H o l t , B i n e h a r t , and w i n s t o n 1 9 7 2

DePelice, L. 3.: F l u c t u a t i o n a n a l y s i s i n ns la robio logy . I n t . Revs. Neurobiol. 20, 169-208 (1977)

Dev, P.: P e r c e p t i o n o f d e p t h s n r f a c e s i n random-aot stereograms: a n e u r a l model . I n t . J. Han-Hachine S t u d i e s 1, 513-528 (7975)

Dickson, J, H., G e r s t e i n , G . L,: I n t e r a c t i o n s between n e u r o n s i n a u d i t o r y cortex of t h e c a t . J. N e u r o p h y s i c l . 37, 1239-126 1 (1974)

Dodge, P. A . : I n h i b i t i o n and e x c i t a t i o n i n t h e LimuZus eye. In : R e i c h a r d t , Sf. ( E d . ) : P r o c e s s i n g of o p t i c a l d a t a by o r q a n i s m s and ~ a c h i n e s , Neu York: Academic Press 1969

Dodge, P. A:, S h a p l e y , B. N., Knight, B. Y o : L i n e a r systelps a n a l y s l s o f the LimuZus retina. Behav. S c i , l5, 24-36 (197G)

Dou, B. : C e n t r a l p r o c e s s i n g o f vision: p a " r a l l e 1 p r o c e s s i n g . Fed , P r o c , 35, 54-59 (1976)

Eccles, J. C., I t o , EJ,, ~ z e n t 5 g o t h a i ~ J.: T h e c e r e b e l l u m a s a n e u r o n a l machine . Ber l in -Heide lberg-Hew Yofk: S p r i n g e r 1967

E l u l , 3 , : B r a i n u a v e s : i n t r a c e l l n l a r r e c o r d i n g and s t a t i s t i c a l a n a l y s i s h e l p c l a r i f y t h e i r p h y s i o l o g i c a l s i g n i f icaece, In: E n s l e i n , K . (Ed , ) Data a c q u i s i t i o n and p r o c e s s i n g i n b i o l o g y and n e d i c i n e . Pp. 93-115, Oxford: Pergaiaon 1968

f e n d e r , D. H. , J u l e s z , %, : E x t e n s i o n o f Panun's S u s i o n a l a r e a i n b i r o c u l a r l y s t a b i l i z e d v i s i o n , J . O p t . Soc. A m , 57, 819-830 (1967)

F e s t i n q e r , L,, Allyn , H, R., t l h i t e , C, 8 . : T h e p e r c e p t i o n of c o l o r v i t h a c h r o ~ a t i c s t i m u l a t i o n . V i s i o n Res, ll, 591-612 (1971) .

F u o r t e s , M. G. E.: I n t e g r a t i v e mechanisms i n t h e n e r v o u s system. A m . N a t u r a l i s t 43, 213-224 (1959)

F u o r t e s , H. G . F.: I n i t i a t i o n of i n p u l s e s i n v i s u a l ce l ls of LimuZus . J. P h y s i o l . 148, 14-28 (1959)

Gabor, D.: H o l o g r a p h i c model of t e m p o r a l r e c a l l . N a t u r e 217, 584 (1968) -

Gelb , A , : A p p l i e d o p t i m a l e s t i m a t i o n . Cambridge: HIT Press 1974.

G e r s t e i n , G . L. : F u n c t i o n a l a s s o c i a t i o n s o f neurons: D e t e c t i o n and i n t e r p r e t a t i o n , In: S c h m i t t , F . 0. (Ed.) : T ~ E n e u r o s c i e n c e s : Second study prograra, New

York: T h e R o c k e f e l l e r Press 1970

G i l b e r t , P. F. C,: Bow t h e c e r e b e l l u m c o u l d aeuior ize a o v e a e n t s . N a t u r e = , 688-689 (1975)

G i l b e r t , P. F. C , , Thach, #. T o : P u r k i n j e c e l l a c t i v i t y durinq moto r l e a r n i n g . B r a i n R e s e a r c h 128, 309-328 (1977)

G r a n i t , 8, : R e c e p t o r s and s e n s o r y p e r c e p t i o n . N e w Haven: Yale U n i v e r s i t y Press 1955

Gross , C, G., Rccha-Miranda, C. E,, Bender, D. Bo: V i s u a l p r o p e r t i e s o f n e u r o n s i n i n f e r i o r t e m p o r a l c o r t e x of

t h e macaque, J, Neurophps io l . 35, 96-1 1 1 (1972) .

Grossbe rg , S . : Cont ro 1 enhancement , s h o r t - t e r e i aeaory. and c o n s t a n c i e s i n r e v e r b e r a t i n g n e u r a l n e t w o r k s , S t u d i e s i n App. Hath. 52, 213-257 (1973)

Grnndfes t . H. : E l e c t r i c a l i n e x c i t a b i l i t y and so ae s o n s e q u e n c e s i n t h e c e n t r a l n e r v o u s system. P h y s i o l . Rev. 37, 337-361 (1957)

G u l i c k , 8 . L., Lawson, R , 9.: H u s a n stereopsis: a psychoph y s i c a l a n a l y s i s . Hew York: Oxford University P r e s s 1976

Haddad, A , 8 , (Ed . ) : N o n l i n e a r sys tems: p r o c e s s i n g o f random signals - c l a s s i c a l a n a l y s i s . S t r o u d s b n r g : Dowden, B u t c h i n s o n , & Ross 1975.

Hade le r , K, P , : On t h e t h e o r y o f l a t e r a l i n h i b i t i o n . K y b e r n e t i k l4, 161-165 (1974)

Haken, H. ( E d .) : C o o p e r a t i v e phenoillena. Amsterdam: North-Hol land P u b l i s h i n g Co, 1974

Ral rnos , P. B.: L e c t u r e s on e r g o d i c theory. Tokyo: M a t h e m a t i c a l S o c i e t y 1956

H a r t l i n e , H . K.: T h e nsural aechan i sms o f v i s i o n , T h e Harvey Lectures. Ser ies X X X V I I 39 (1941)

X a r t l i n e , H. R., R a t l i f f , Fa: I n h i b i t o r y interaction of t h e r e c s p t o r u n i t s i n t h e eye of LimuZus. J. Gen, P h y s i o l . 9, 1357-1376 (1957)

H a r t l i n e , ki. K . , B a t l i f f , F.: I n h i b i t o r y i n t e r a c t i o n i n t h e r e t i n a of ~imuZu6, In: P u o r t e s , ?l. G, F. (Ed.): Handbook of s e n s o r y phys io logy . Y 1112. Ber l in-Beide lberg-New York: S p r i n g e r - V e r l a g 1972

Rebb, D . 0. : T h e o r g a n i z a t i o n of b e h a v i o r . New York: Riley 1 9 4 9

H o f f e r , B . J., S i g g i n s , G. R . , O l i v e r , A, P . , Bloom, F. E. A c t i v a t i o n of t h e pathway from l o c u s c o e r u l e u s t o r a t cerebellar P u r k i n j e neurons : p h a r m a c o l o g i c a l evidence of n o r a d r e n e r g i c c e n t r a l i n h i b i t i o n , J . Pharmac. Exp . The rap . 784, 553-569 (1973)

Holden, A , V . : F o d e l s of t h e s t o c h a s t i c a c t i v i t y of neu rons , Ber l in -Heide lberq-New York : S p r i n q e r - V e r l a g 1976

Hube l , D. 8 . , U i e s e l , T, 8 . : R e c e p t i v e fields, b i n o c u l a r i n t e r a c t i o n a n d f u n c t i o n a l a r c h i t e c t u r e i n the ca t ' s v i s u a l c o r t e x . 3. ~ h y s i o l . 160, 106-154 (7962)

H u b e l , D. R,, Wiesel, 41. N.: S t e r e o s c o p i c v i s i o n i n macaque a o n k e y . Nature 225, 41-42 { I9701

Eiubel, D. H., fiiiesel, T . N.: F u n c t i o n a l a r c h i t e c t u r e of macaque v i s u a l c o r t e x . P r o c . Bop. Soc. Lond. B198, 1-59 ( 1 9 7 7 )

I t o , H : L e a r n i n g c o n t r o l m e c h a n i s m s by t h e cerebellum flocculo-vestibulo-ocular s y s t e m . I n : Tower , D. 8, : The n e r v o u s system. Vol. 1. N e w York: Raven Press 1975

Jeffress,, L. A . : L o c a l i z a t i o n of sound . I n : K e i d e l , 51. D., N e f f , 9, D . (Eds.) : Handbook o f s e n s o r y p h y s i o l o g y V/2: A u d i t o r y s y s t e m . B e r l i n - H e i d e l b e r g - N e w York: S p r i n g e r 7 9 7 5

J u l e s z , B . : B i n o c u l a r d e p t h p e r c e p t i o n o f c o e p u t e r - g e n e r a t e d p a t t e r n s . B e l l S y s t e m T e c h . J . 2, 1125-1162 ( 1 9 6 0 ) .

J u l e s z , 8 . : Towards t h e a u t o e a t i o n of b i n o c u l a r d e p t h p e r c e p t i o n and p a t t e r n r e c o g n i t i o n (ADTOHAP-1). In: P o p p l e u e l l , C. H.: P r o c e e d i n g s o f the IFIPS c o n g r e s s , Hun ich 1962. Amsterdam: W o r t h - H o l l a n d 1 9 6 3

J u f e s z , 5.: P c u n d a t i o n s o f c y c l o p e a n p e r c e p t i o n . Ch icago : U n i v e r s i t y of C h i c a g o Press 3971

J u l e s z , S.: C o o p e r a t i v e phenomena i n b i n o c u l a r d e p t h p e r c e p t i o n . A m e r i c a n S c i e n t i s t 62, 32-43 (1974)

J u l e s z , B., Chang, J. -J. : I n t e r a c t i o n b e t w e e n p o o l s of b i n o c u l a r d i s p a r i t y d e t e c t o r s t u n e d t o d i f f e r e n t d i s p a r i t i e s . B i o l . C y b e r n e t i c s 22, 107-1 1 9 (1976)

J u l e s z , B . , J o h n s o n , S . C. : f l e n t a l h o l o g r a p h y : s t e r e o g r a s s p o r t r a y i n q a m b i g u o u s l y p e r c e i v a l b l e surfaces. B e l l Systems Tech. J. 49, 2075-2083 (1968)

J u l e s z , B. P e n n i n g t o n , K. S. : E q u i d i s t r i b o t e d i n f o r m a t i o n mappinq: an a n a l o g y t o h o l o g r a m s a n d memory. J. O p t . Soc . A m . 5 6 , 6 0 4 (1965)

K a l i l , R . E,, C k a s e , R.: C o r t i c o f u g a l influence o n a c t i v i t y o f l a t e r a l g e n i c u l a t e n e u r o n s i n t h e c a t . J. N e u r o p h y s i o l . 33, 459-474 ( I 970)

Kalman, R . E , , Bucy, R. S. : New results i n l i n e a r filtering a n d p r e d i c t i o n t h e o r y . J. B a s i c E n g i n e e r i n g 95-1 08 ( n a r c h I96 1)

Kandel , E. R . : C e l l u l a r b a s i s o f b e h a v i o r , San F r a n c i s c o : 8 , H, Freeman 1976

Kandel , E. B , , Tauc, L.: H e t e r o s y n a p t i c f a c i l i t a t i o n i n n e u r o n e s o f t h e abdominal g a n g l i o n o f ApZysia depiZrms . J. P h y s i o l . E, 1-27 (1965)

Kaneko, A .: P h y s i o l o q i c a l and m o r p h o l o g i c a l i d e n t i f i c a t i o n of h o r i z o n t a l , b i p o l a r , a n d a macrine ce l l s i n g o l d f i s h r e t i n a . J, P h y s i o l . 207, 6 2 3 - 6 3 3 (1970)

Kato, T , : P e r t u r b a t i o n t h e o r y f o r l i n e a r operators. New York: S p r i n q e r - V e r l a g 1966

Katz , B , : Nerve , muscle, and s y n a p s e . N e w York: 1 4 c ~ r a w - H i l l 1966

Kety, S. S.: The b i o g e n i c amines i n t h e central nervous sys t em: t h e i r p o s s i b l e r o l e s i n a r o u s a l , emot ion , and learning. In: S c h m i t t , F* 0. (Ed. ) : T h e n e u r o s c i e n c e s : Second study proqrasa. Neu York: The R o c k e f e l l e r Press 1970

Kimble, D . P. (Ed.) : The anatomy of memory. P a l o Alto: S c i s n c e and Behav io r Books 1965

Kniqht , 3. P . : Dynamics of encod ing i n a p o p u l a t i o n of n e u r o n s . J . Gen. P h y s i o l . 59, 734-766 (1972)

Kniqht , 8 . W,: The r e l a t i o n s h i p between t h e f i r i n g r a t e o f a s i n g l e neu ron a n d t h e l e v e l of a c t i v i t y i n a p o p u l a t i o n of neu rons : e x p e r i m e n t a l e v i d e n c e f o r r e s o n a n t enhancement i n t h e p o p u l a t i o n response, J, Gen. P h y s i o l , 5 9 , 767-778 (1972)

Kniqht , B , %,: T h e h o r s e s h o e crab eye: a l i t t l e n e r v o u s system whose dynamics a r e s o l v a b l e , Lectures on Math. i n t h e L i f e Sci. 5, 11'1-144 (1973)

Kniqht, B . W . , Toyoda, J.-I., Dodge, F. A . : A q u a n t i t a t i v e d e s c r i p t i o n o f t h e d y n a ~ i c s of e x c i t a t i o n and i n h i b i t i o n i n t h e e y e of LimuZus . J . Gen. P h y s i o l . 56, 421-437 (1970)

Knudsen, E. I , , Koni sh i , M e , P e t t i g r e w , J . D,: R e c e p t i v e f i e l d s of a u d i t o r y n e u r o n s i n t h e owl. S c i e n c e 198,

Kohonen, T. : g s s o c i a t i v e memory, B e s l i n - B e i d e l b e r g - H e w York: S p r i n q e r - V e r l a g 1977

K o n i s h i , H: S p a t i a l l o c a l i z a t i o n o f s o u n d . I n : B u l l o c k , T. H, (Ed , ) : R e c o g n i t i o n o f c o a p l e l r a c o u s t i c signals. B e r l i n : Dahlern Conf e r e n z e n 1977

K u f f l e r , S. W ., F i t z H u g h , R., Bar low, 8, B.: X a i n t a i n e d a c t i v i t y i n t h e c a t ' s r e t i n a i n l i g h t a n d d a r k n e s s , J. Gen. P h y s i c l , 40, 683-702 (1957)

K u f f l e r , S. O, , N i c h o l l s , J. G.: From n e u r o n t o b r a i n . S u n d e r l a n d , #ass, : S i n a u e r Bssoc . Inc, 1976

L e v i c k , ii. R . : H a i n t a i n e d d i s c h a r g e i n t h e v i s u a l s y s t e m and its r o l e f o r i n f o r m a t i o n p r o c e s s i n g . I n : Jung, R. ( E d , ) : Handbook o f s e n s o r y p h y s i o l o g y , VII/3. B e r l i n - H e i de lberg-New Yor k: S p r i n g e r - V e r l a g 1973

L e w i s , P. A . W. (Ed . ) : S t o c n a s t i c p o i n t p r o c e s s e s . Blew York: J o h n Wiley 1972

~ l i n g s , R e R . (Ed. ) : N e u r o b i o l o g y o f c e r e b e l l e b e v o l u t i o n a n d d e v e l o p m e n t . Chicago: Amer. Bed. Assoc . 1969

link, R. R.: The cortex of t h e c e r e b e l l u m . S c i . Am. Pp. 5 6 - 7 1 J a n u a r y 1 9 7 5

L o n g u e t - Hiqqens , H. C. : H o l o g r a p h i c model o f t e m p o r a l r e c a l l , Nature 277, 104 ( 1 9 6 8 )

L o n q u e t - H i q q i n s , H. C. , g i l l s h a w , D. J,, Buneman, 0. P.: Theories o f a s s o c i a t i v e r e c a l l . Q u a r t . Revs. Biophys, 2, 223-244 (1970)

Lorente d e 3 6 , B, : A n a l y s i s o f t h e a c t i v i t y of c h a i n s of i n t e r n u n c i a l n e u r o n s . J . ~ e u r o ~ h ~ s i o l , -* 1 207-244 (1 938)

~ c C u l l o c h , 9. S . , Pitts, #. 8. : A l o g i c a l c a l c u l u s o f i d e a s immanen t i n n e r v o u s a c t i v i t y . B u l l , m a t h . B i o p h y s . & 115-133 (1 543)

Marr, D . : A t h e o r y o f c e r e b e l l a r cortex. 3. ~ h y s i o l , 202, 437- 470 (1 969)

Harr , D, , P o g q i o , T . : C o o p e r a t i v e c o m p u t a t i o n o f stereo d i s p a r i t y . S c i e n c e 194, 283-287 (1976)

# a r r , D, , Poqgio, T, : F r o a u n d e r s t a n d i n g co rnpa ta t ion t o u n d e r s t a n d i n g n e u r a l circuitry. Neurosc. Bes. Prog. B u l l . 15, - 470-488 (1977)

a i l e s , E, A , : The primate f l o c c u l u s and eye-hand c o o r d i n a t i o n . I n : Brooks, 3 , A , , Ba j andas , P, J, [Eds.) : E y e movements: ARVO Symposium 3976. PIer Yorlr:

Plenum 1977

Moore, G. P., P e r k e l , D o H . , Segundo* J. P.: Statistical a n a l y s i s and f u n c t i o n a l i n t e r p r e t a t i o n o f n e u r o n a l s p i k e d a t a . Ann, Revs. P h y s i o l . 28, 493-522 (1966)

Hori, I(,-I,, Kodode, !la, Asada, He: An i t e r a t i v e prediction and c o r r e c t i o n method f o r a u t o m a t i c s t e r e o c o m p a r i s o n . Corap. G r a p h i c s 8 Image Proc . 2, 393-401 (1973)

flarrell, Pa: Electrical signs of s e n s o r y c o d i n g , In: Q u a r t o n , G . C,, ne lnechuk , T,, Schmi t t , F. 0. (Edso) : The n e u r o s c i e n c e s : a s t u d y program, New York: The R o c k e f e l l e r D n i s e r s i t y Press 1967

H o r r e l l , F,: I c t e q r a t i v e p r o p e r t i e s o f parastriate neurons . In: Karczmar, A. G . , Eccles, J . C. (Eds . ) : B r a i n and human b e h a v i o r . B e r l i n - H e i d e l b e r g - New Pork: S p r i n q e r - V ~ r l a g 1972

i l o u n t c a s t l e , 'J. E.: T h e p roblem o f s e n s i n g and n e u r a l cod ing . In : Q u a r t o n , G. C., HeLnechuk, T., Schmitt, P* 0. (Eds .) : T h e n e u r o s c i e n c e s : a s t u d y program, New York: T h e Bockefeller U n i v e r s i t y Press 1967

H o u n t c a s t l e , 8 , B.: An o r g a n i z i n g p r i n c i p l e f o r c e r e b r a l f u n c t i o n : t h e u n i t module a n d the d i s t r i b u t e d system. In: S c h m i t t , P. C. (Ed.) : T h e n e u r o s c i e n c e s : f o u r t h study program, Canbr idge : aIT P r e s s 7978

Neher, E., S t e v e n s , C. F.: Conductance f l u c t u a t i o n s and i o n i c p o r e s i n m ~ m b r a n e s , Ann, R e v s , 3 iophys . Bioeng. 6 , 345-382 (1977) -

Nelson, J . I, : G l o b a l i t y and s t e r e o s c o p i c v i s i o n , J. Theor . Biol. 49, 1-88 (7975)

Norman, D, A. : Memory and a t t e n t i o n . 2d e d i t i o n . New York: Biley 1976

Offenloch, K.: What i s t h e EEG composed o f ? Aspects of EEG e l e c t r o g e n e s i s . In: CEAN -Compute r i zed EXG a n a l y s i s . S t u t t q a r t : Gustav Fischer 1975

Olson, L., P u x e , R. : On p r o j e c t i o n s from the l o c u s c c w r u l e a s n o r a d r e n e r q i c neu rons : t h e c e r e b e l l a r innervations. B r a i n Bes. 28, 165-171 (1971)

P a l a y , S . L., C han-Palay, 8 , : C e r e b e l l a r c o r t e x : cytology and o r g a n i z a t i o n . Berlin-Beidelberg-New Pork: S p r i n g e r 1974

P e a r s o n K. G,: Nerve c e l l s w i t h o u t a c t i o n p o t e n t i a l s . In: P e n t r e s s , J. C. (Ed.) : S i m p l e r ne tworks and b e h a v i o r . S u n d e r l a n d , Mass. : S i n a u e r A s s o c i a t e g 1976

P e l l i o n i s z , A,, ~ l i n i s , R. , P e r k e l , D. 3.: A c o n p u t e r node1 o f t h e c e r e b e l l a r c o r t e x o f the f rog . I e n r o s c i e n c e 2, 19-35 (1977)

P e r k e l , D, R., E u l l o c k , I, H,: Heural coding. Beurosc. Bes. P s o ~ . B u l l . & 221-348 (1968)

P f a f f e l h u b e r , E.: C o r r e l a t i o n memory models - a first a p p r o x i m a t i o n i n a g e n e r a l l e a r n i n g scheme. B io l . C y b e r n e t i c s 18, 277-223 (1975)

Poqg io , G. P ., P i s c h e r , B.: B i n o c u l a r i n t e r a c t i o n and d e p t h sensitivity i n s t r i a t e and p r e s t r i a t e c o r t e x of b e h a v i n g r h e s u s monkeys. J. N e u r o p h y s i o l , 40, 1392-1405 (1977)

P u r p l e , B. I..: T h e i n t e g r a t i o n of e x c i t a t o r y and i n h i b i t o r y i n f l u e n c e s i n the eccentric cell i n t h e eye of LimuZus . Thesis: The R o c k e f e l l e r I n s t i t u t e 1964

P u r p l e , R. L., Dodqe, F. A , : I n t e r a c t i o n o f e x c i t a t i o n and i n h i b i t i c n i n t h e e c c e n t r i c c e l l i n the eye of LimuZus . Cold S p r i n g Harbor Symp. on Quant. Biol, 30, 529-537 (1 965)

P r i b r a m , K. H , , N u w e r , fl., Baron, 3. J.: The h o l o g r a p h i c hypothesis of memory s t r u c t n r e i n b r a i n f u n c t i o n and p e r c e p t i c n . In : Krantz , D, H., A tk inson , R, C., Luce, R . D., s u p p e s , P. ( E d s , ) : Contemporary deve lopmen t s i n m a t h e m a t i c a l p sycho logy , Vol. 11 I Heasur ement psychop hysics a n d n e u r a l i nf o r n a t i o n p r o c e s s i n g , San F r a n c i s c o : 9 , H, Freeman 1974

Rakic , P. (Ed.) : L o c a l c i r c u i t neurons . Neurosc i ence Bes. Prog. B u l l , l3, 289-446 (1975)

~ a m 6 n y C a j a l , S.: Histologic du systsrne ne rveux d e l'homme et d e s v e r t g b r g s . Tom. XI. P a r i s : H a l o i n e 1955,

Madrid : Cocse j o S u p e r i o r d e I n e e s t i g a c i o n e s C i e n t i f icas 1 9 1 1

R a t l i f f , F. : Mach bands: q u a n t i t a t i v e studies on neord l n e t w o r k s i n t h e retina. S a n P r a n c i s c o : Bolden-Day Pnc. 1965

R a t l i f f , F. (Ea,): Studies i n e x c i t a t i o n and inhibition i n t h e r e t i n a , Neu York: T h e R o c k e f e l l e r U n i v e r s i t y Press 1974

R a t l i f f , F., H a r t l i n e , H, K,: T h e r e s p o n s e s o f LimZtls o p t i c n e r v e f i b e r s t o p a t t e r n s of i l l u m i n a t i o n on t h e r e c e p t o r mosa ic . J . Gen, P h y s i o l , 6, 1241-1255 (1959)

R a t l i f f , F., H a r t l i n e , H. K , , Hiller, W. 8,: S p a t i a l and t e m p o r a l a s p e c t s o f r e t i n a l i n h i b i t o r y i n t e r a c t i o n , J. O p t . Soc . Am. 53, 110-120 (1963)

Robinson, D. A*: A d a p t i v e g a i n c o n t r o l o f vestibule-ocular reflex by t h e c e r s h e l l u m , J. ~ e u r o p h y s i o l . z, 954-969 (1976)

Rodiek , -9, 5 i . : l a i n t a i n e d a c t i v i t y of c a t retinal g a n g l i o n ce l l s . J, Neurophys io l . 3, 1043-1071 (1967)

Rosenzweig, B R. , Bennett, E. L. (2ds.) : Neural mechanisms of l e a r n i n g and memory. Cambridge: 8 I T Press 1976

Ross , J: S t e r e o p s i s by k i n o c u l a r d e l a y . N a t u r e 248, 363-364 (1974)

S c h m e i d l e r , 8.: Linear o p e r a t o r s i n A i l b e r t s p a c e . Nev York: Academic Press 1965

S e j n o w s k i , T. J.: On g l o b a l p r o p e r t i e s o f n e u r o n a l i n t 3 r a c t i o n . B i c l , C y b e r n e t i c s 22, 85-95 (1976)

S e j n o w s k i , T , J,: On t h e s t o c h a s t i c dynamics of n e u r o n a l i n t e r a c t i o n , B io l . C y b e r n e t i c s 22, 203-21 1 (7976)

S e j n o w s k i , T. J.: S t o r i n g c o v a r i a n c e w i t h nonlinearly i n t e r a c t i n q n e u r o n s . J. ?lath. B i o l . - 4, 303-327 (1977)

Se j n o v s k i , T. 3. : S t a t i s t i c a l c o n s t r a i n t s on synaptic p l a s t i c i t y . J . Theo. B i o l . , 385-389 ( I 977)

Shannon, C , E , , McCarthy, J . fl, (Eds.): Automata s t u d i e s . P r i n c e t o n : P r i n c e t o n U n i v e r s i t y Press 1956

S h e p h e r d , G . if. : The s y n a p t i c o r g a n i z a t i o n of t h e b r a i n . Oxford: Oxford U n i v e r s i t y Press 1974

S h e p h e r d , G. H,: ~ i c r o c i r c u i t s i n t h e n e r v o u s s y s t e m , Sci. Am. Pp. 93-103 F e b r u a r y 1 9 7 8

S h i m a b a r a , T. , Tauc , L e : I d e n t i f i c a t i o n o f a n e u r o n inducing h e t e r o s y n a p t i c f a c i l i t a t i o n on a spec i f i c s y n a p s e i n ApZysia . E r a i n Bes. 118, 142- 146 ( 1 976)

S n y d e r , D. L.: Eandom p o i n t p r o c e s s e s . N e w Pork: V i l e y 1975

S t e i n b u c h , K. : Die l e r n n t a t r i x . K y b e r n e t i k - 1 , 36-45 (1961)

S p e k r e i j s e , H . : R e c t i f i c a t i o n i n t h e goldfish r e t i n a : a n a l y s i s by s i n u s o i d a l and a u x i l i a r y s t i m u l a t i o n , V i s i o n Res. 9, 1461-1472 (1969)

S p e k r e i j s e , H., O o s t i n g , H,: L i n e a r i z i n g : a m e t h o d f o r a n a l y z i n g and s y n t h e s i z i n g n o n l i n e a r sy ste MS. K y b a r n e t i k 2, 22-31 (1970)

Sperling, G. : E i n o c u l a r f u s i o n : a p h y s i c a l and neural t h e o r y , A m . J, Psych. 83, 461-532) ( 1 9 7 0 )

S t e n t , G. S.: A p h g s i o l o g i c a l mechanism f o r H e b b l s p o s t u l a t e o f l e a r n i n g . P r o c , Natn . Acad. S c i . U.S.A. 997- 1 0 0 1 ( 1973)

S t e v e n s , C, F,: A q u a n t i t a t i v e t h e o r y o f n e u r a l i n t e r a c t i o n s : t h e o r e t i c a l a nd e x p e r i m e n t a l i n v e s t i g a t i o n s . Thesis: T h e R o c k e f e l l e r I n s t i t u t e 1964

S t e v e n s , I=. P. : N e u r o p h y s i o l o g y : A p r i m e r . N e w York: John U i l e y 1966

S t e v e n s , C. P.: S t u d y o f merttbrane p e r m e a b i l i t y c h a n g e s by f l u c t u a t i o n a n a l y s i s . N a t u r e 270, 391-396 (1977)

S t e v e n s , 3 . K. , G e r s t e i n , G. L. : I n t e r a c t i o n s b e t w e e n c a t l a t e r a l g e n i c u l a t e neurons. J. N e u r o p h y s i o l . t 39 239-256 (1 976)

~ z e n t s g o t h a i , J.: Structure-functional c o n s i d e r a t i o n o f t h e c e r e b e l l a r n e u r o n n e t w o r k . P r o c . IEEE 56, 960-968 (1968)

~ z e n t / a g o t h a i , J.: A r c h i t e c t u r e o f t h e . cerebral c o r t e x , I n : Jasper , H. H., Ward, A . A , , Pope, A. (Eds.): B a s i c m e c h a n i s m s o f t h e e p i l e p s i e s . B o s t o n : L i t t l e , Brown

T o m i t a , T. : E l e c t r o p h y s i o l o g i c a l study of the a e c h a n i s ~ t s s u b s e r v i n g c o l o r c o d i n g i n the f i s h r e t i n a . Cold Sprinq Harbor Sysp. Quan t . B i o l . 30, 559-566 [ I %5)

T s y p k i n , Ya. 2.: F o u n d a t i o n s of t h e theory o f l e a r n i n g systems. New York: Acadertiic P r e s s 1973

Verveen, A . A . , Defelice, L . J.: B e a b r a n e n o i s e , P r o g . B i o p h y s , f l o l e c . B i o l . 28, 189-265 (1974)

Verveen, A , A . , D e r k s e n , H. 2 . : F l u c t o a t i o a s in w m b r a n e p o t e n t i a l of axons a n d the problem of coding. K y b e r n e t i k 2, 152-160 (1965)

Werbl in , F . S . , D o w l i n g , J. E.: O r g a n i z a t i o n of t h e retina o f the mudpuppy, Necturus macuZosus: 11. I n t r a c e l l u l a r r e c o r d i n g . J. H e u r o p h y s i o l . 32, 339-355 (1969)

S e s t l a k e , P, . : T h e p o s s i b i l i t i e s of n e u r a l h o l o g r a p h i c processes within t h e b r a i n . K y b e r n e t i k 2, 129-153 (1970)

w h i t e , : The f o r m a t i o n of cel l a s s e m b l i e s . B u l l . Hath. B i o p h y s . - 23, 43-53 ( 1961 )

Wiener, N . : E x t r a p o l a t i o n , i n t e r p o f a t i o n , a n d s i a o o t h i n g of s t a t i o n a r y t i m e series. Cambridge: MIT Press 1949

Wiesel, T. N, , H u b e l , D . H . : S i n g l e - c e l l responses i n s t r i a t e c o r t e x o f k i t t e n s d e p r i v e d o f v i s i o n i n one eye. J . N e u r o p h y s i o l . - 2 6 , 1003-10 17 (1963)

Biqner, E. P. : G r o u p t h e o r y a n d i ts applications t o t h e q u a n t u m m e c h a n i c s o f a t o m i c s p e c t r a , New Pork: Academic Press 1 9 5 9 .

i4igstrom, B.: A neural mode l w i t h l e a r n i n g c a p a b i l i t y and i ts r e 1 a t i . c n t o m e c h a n i s m s o f a s s o c i a t i o n . K y b e r n e t i k 12, 204-215 (1973) -

P i l s o n , B . R , , Cowan, J. D.: E x c i t a t o r y and i n h i b i t o r y i n t e r a c t i o n s i n localized p o p u l a t i o n s of reode l n e u r o n s . Biophys. J. 12, 1 - 2 4 ( 1 9 7 2 )