Stochastic Synchrony of Chaos in a Pulse Coupled Neural

Network with Both Chemical and Electrical Synapses

among Inhibitory Neurons

Takashi Kanamaru† and Kazuyuki Aihara‡,††

† Department of Innovative Mechanical Engineering,Faculty of Global Engineering, Kogakuin University,139 Inume, Hachioji-city, Tokyo 193-0802, Japan

‡ Institute of Industrial Science, University of Tokyo,4-6-1 Komaba, Meguro-ku, Tokyo 153-8505, Japan

†† ERATO, JST, Japan

Neural Computation, vol.20, no.8 (2008) pp.1951-1972.Java Applet: http://brain.cc.kogakuin.ac.jp/˜kanamaru/Chaos/e/sSync/

Abstract

The synchronous firing of neurons in a pulse coupledneural network composed of excitatory and inhibitoryneurons is analyzed. The neurons are connected by bothchemical synapses and electrical synapses among the in-hibitory neurons. By introducing electrical synapses, pe-riodically synchronized firing as well as chaotically syn-chronized firing is widely observed. Moreover, we findstochastic synchrony where the ensemble-averaged dy-namics shows synchronization in the network but eachneuron has a low firing rate and the firing of the neuronsseems to be stochastic. Stochastic synchrony of chaoscorresponding to a chaotic attractor is also found.

1 Introduction

Since the 1980s, oscillations and synchronization of theensemble-averaged dynamics in neuronal assemblies havebeen found in many areas of the brain, and their rolesin information processing have been discussed (For a re-view, see e.g. Gray (1994)). For example, when visualstimulation was given to cats, oscillations of 40 Hz ap-peared in the local field potential in the visual cortex.Moreover, when correlated inputs were given to the re-ceptive fields of each assembly, synchronization amongdistant (∼ 7mm) assemblies appeared (Gray & Singer,1989). It has been proposed that such synchronizationamong neuronal assemblies might solve the binding prob-lem (Gray, 1999). On the other hand, various oscillationsare known to exist in the hippocampus, such as the sharpwave of 200 Hz (Buzsaki et al., 1992), the theta rhythmof 8 Hz (Csicsvari, Hirase, Czurko, & Buzsaki, 1998), andthe gamma rhythm of 40 Hz (Bragin et al., 1995), and

the correlated firing caused by such oscillations mightbe related to regulation of learning in the hippocampus(Buzsaki, 2006). However, the above discussions are justbased on hypotheses, and possible roles and mechanismsof the oscillations and synchronization are still contro-versial.

Moreover, weak synchronization, where the ensemble-averaged dynamics in neuronal assemblies shows oscilla-tions but each neuron has a low firing rate, is also knownto exist in the visual cortex (Gray & Singer, 1989), inthe hippocampus (Buzsaki et al., 1992; Csicsvari, Hi-rase, Czurko, & Buzsaki, 1998; Fisahn, Pike, Buhl, &Paulsen, 1998; Whittington et al., 2000), and in the cere-bellar nucleoolivary pathway (Lang, Sugihara, & Llinas,1996). Brunel & Hansel (2006) and Tiesinga & Jose(2000) called such weak synchronization as stochasticsynchrony. Stochastic synchrony was found both inmodeling studies based on experimental data (Traub,Miles, & Wong, 1989) and in theoretical modeling stud-ies (Brunel, 2000; Brunel & Hakim, 1999; Brunel &Hansel, 2006; Kanamaru, 2006a; Kanamaru & Sekine,2004), and its relationship to information processing hasattracted attention. One of the mechanisms of stochasticsynchrony might be oscillations with small amplitudes ofthe ensemble-averaged dynamics in the network. Let usconsider the situation where the dynamics averaged inan assembly of neurons shows an oscillation, and this os-cillation becomes a feedback input to this network. If theamplitude of this feedback input is sub-threshold for eachneuron, the firing of each neuron becomes stochastic, andstochastic synchrony takes place (Kanamaru & Sekine,2004). This mechanism is similar to that of stochasticresonance (Gammaitoni, Hanggi, Jung, & Marchesoni,1998; Longtin, 1993).

1

To our knowledge, in modeling studies, stochastic syn-chrony has been realized only by periodic oscillations.On the other hand, in the present study, we foundstochastic synchrony that was generated by chaotic os-cillations. This paper is organized as follows. In section2, a pulse coupled neural network is defined. This net-work is composed of excitatory neurons and inhibitoryneurons, and their parameters such as synaptic time con-stants are set as asymmetric between excitatory and in-hibitory populations in order to reflect biological con-ditions. Besides chemical synapses, we also introduceelectrical synapses among the inhibitory neurons. To ex-amine the dynamics of this network, the Fokker-Planckequation obtained in the limit of an infinite numberof neurons is numerically analyzed, and the bifurcationof the system is investigated. By introducing electri-cal synapses, chaotic oscillations with small amplitudesare observed. In section 3, it is found that the oscil-lations with small amplitudes correspond to stochasticsynchrony, namely, weak synchronization. If such oscilla-tions are chaotic, stochastic synchrony of chaos appears.In section 4, the roles of types of connections betweenassemblies are examined. The final section provides dis-cussions and conclusions.

2 A Network Composed of Exci-

tatory Neurons and InhibitoryNeurons

In the following sections, we consider a pulse coupledneural network composed of excitatory neurons with in-ternal states θ(i)E (i = 1, 2, · · · , NE) and inhibitory neu-rons with internal states θ(i)I (i = 1, 2, · · · , NI) (Kana-maru, 2006b) that are represented as follows:

τE˙θ(i)E = (1− cos θ(i)E ) + (1 + cos θ(i)E )

×(rE + ξ(i)E (t) + gEEIE(t)− gEIII(t)), (2.1)

τI˙θ(i)I = (1− cos θ(i)I ) + (1 + cos θ(i)I )

×(rI + ξ(i)I (t) + gIEIE(t)− gIIII(t) + ggapI(i)gap(t)),

(2.2)

IX(t) =1

2NX

NX∑j=1

∑k

1κX

exp

(− t− t

(j)k

κX

), (2.3)

I(i)gap(t) =1NI

NI∑j=1

sin(θ(j)I (t)− θ(i)I (t)

), (2.4)

〈ξ(i)X (t)ξ(j)Y (t′)〉 = DδXY δijδ(t− t′), (2.5)

where X and Y each denote the excitatory assembly Eor the inhibitory assembly I, t(j)k is the kth firing timeof the jth neuron in assembly X , and the firing time isdefined as the time at which θ(j)X exceeds π in the pos-itive direction. In addition to the connections IX(t) bychemical synapses in which the post-synaptic potential

is written by an exponential function, there are connec-tions I(i)gap(t) with connection strength ggap by electri-cal synapses among the inhibitory neurons. Previousexperimental studies showed that there are rich electri-cal synapses among inhibitory neurons in many areasof the brain such as the cortex (Galarreta & Hestrin,1999, 2001; Gibson, Beierlein, & Connors, 1999) andthe hippocampus (Fukuda & Kosaka, 2000; Katsumaru,Kosaka, Heizmann, & Hama, 1988; Strata et al., 1997;Zhang et al., 1998). The electrical synapses are real-ized by structures called gap junctions (Nicholls, Mar-tin, Wallace, & Fuchs, 2001), and such connections cor-respond to the diffusive couplings in physical systems. Itis known that they facilitate synchronous firing amongneurons (Ermentrout, 2006).Note that the model of neurons with θ = (1− cos θ)+

(1 + cos θ)r is called the theta neuron model (Ermen-trout & Kopell, 1986; Ermentrout, 1996), and this isthe canonical model of class 1 neurons. Without theelectrical synapses, our network is based on the canon-ical model of class 1 networks connected by chemicalsynapses with exponential functions (Izhikevich, 1999,2000). For simplicity, the restrictions, gEE = gII ≡ gint

and gEI = gIE ≡ gext, are placed, where gint is the in-ternal connection strength in an assembly and gext is theexternal connection strength between excitatory and in-hibitory assemblies. Note that the introduction of theelectrical synapses to the theta model is not straightfor-ward because the transformation of the variable of themembrane potential to phase θ is singular when the neu-ron fires (Ermentrout, 2006). We use the conventionaldefinition of the diffusive coupling for simplicity. The re-sult similar to that obtained in this paper was also found(data not shown) using the class 1 Morris-Lecar neurons(Ermentrout, 1996; Morris & Lecar, 1981); therefore, ourresults might be found widely in the networks of class 1neurons.The membrane time constants are set as τE = 1 and

τI = 0.5 in order to take into account the physiologi-cal fact that fast spiking cells are dominant among theinhibitory neurons in the cortex. The synaptic time con-stants are set as κE = 1 and κI = 5.In the absence of inputs IX(t) from assembly X and

noise ξ(i)X (t), a single neuron shows a self-oscillation whenrX > 0. When rX < 0, this neuron becomes an excitablesystem with a stable equilibrium point defined by

θ0 = − arccos1 + rX1− rX , (2.6)

in which θ0 is close to zero for rX ∼ 0. In the following,we use values of the parameter rX < 0 and consider thedynamics of a network of excitable neurons.As shown in Appendix A, the ensemble-averaged dy-

namics in this network can be analyzed using the Fokker-Planck equation (Gerstner & Kistler, 2002; Kuramoto,1984), which is obtained in the limit of NE , NI → ∞.When there are no correlations of firing among neuronsin the network, the stable solution of the Fokker-Planck

2

0

2

4

6

8

10

0.001 0.01 0.1

gext

B ggap

=0.15

D

Hopf

SN

SNL

ChaosSN

HB

0

2

4

6

8

10

0.001 0.01 0.1

gext

A ggap

=0

D

Hopf

SN

SN

crisis HB

Chaos

Figure 1: Bifurcation sets in the (D, gext) plane for (A)ggap = 0 and (B) ggap = 0.15. The excitatory and in-hibitory neurons are connected by chemical synapses.Synchronous firing of neurons is generally observed whenthe values of D and gext are in the area enclosed bythe Hopf and HB (or SNL) bifurcation lines. In (B),by introducing the electrical synapses among the in-hibitory neurons, synchronous firing exists even whengext > gint = 5. The solid, dotted, and dash-dotted linesdenote the Hopf, saddle-node, and homoclinic bifurca-tions, respectively. The areas where a chaotic solutionexists are roughly sketched. SN, the saddle-node bifur-cation; SNL, the saddle-node on limit cycle bifurcation;HB, the homoclinic bifurcation.

equation is the equilibrium state. When there are somecorrelations of firing among neurons, the solution is time-varying, namely, the ensemble-averaged dynamics in thenetwork can be represented by a limit cycle, a chaoticattractor, and so on. In the following, when the solu-tion of the Fokker-Planck equation is time-varying, weconsider the firing of the neurons as synchronous.

Bifurcation sets of the network obtained by numeri-

cal analyses of the Fokker-Planck equation (Kanamaru,2006a, 2006b) based on the method given in AppendixB, are shown in Figure 1. The noise intensity D and theexternal connection strength gext are chosen as the bi-furcation parameters. The bifurcation set for a networkwithout electrical synapses is shown in Figure 1A. Gen-erally, synchronous firing of neurons is observed whenthe values of D and gext are in the area enclosed bythe Hopf and HB (or SNL) bifurcation lines. A simi-lar analysis was presented in Kanamaru (2006b), wherethe membrane time constants and the synaptic time con-stants were set as uniform. On the other hand, in thepresent study, the time constants of the excitatory andinhibitory assemblies are set to be asymmetric, namely,τE = 1, τI = 0.5, κE = 1, and κI = 5. With thissetting, it was found that the range of the parametervalues where synchronous firing exists became narrowerthan that in the previous analysis. However, the struc-ture of the bifurcations was almost the same with thatof the previous analysis; therefore, it can be concludedthat the structure of the bifurcations is robust againstchanges in the membrane time constants and synaptictime constants. It is also observed that chaos exists nearthe homoclinic bifurcation set, but the area is very nar-row because chaos disappears by the crisis.It should be also noted that synchronous firing can

be observed only when gext < gint = 5, and this fact isin agreement with a previous study (Kanamaru, 2006b).Let us consider the dynamics in a network with largegext by examining the instantaneous firing rates JE andJI of the excitatory and inhibitory assemblies, respec-tively, which can be calculated using the Fokker-Planckequation as shown in Appendix A. Generally, JE andJI decrease as gext increases; therefore, the equilibriumpoint (JE , JI) approaches the origin. The dependenceof the equilibrium of JE on gext is shown in Figure 2A,where JE decreases as gext increases.By introducing electrical synapses among the in-

hibitory neurons, the area where synchronous firing canbe observed widened and synchronous firing was ob-served even for gext > gint = 5, as shown in Figure1B. Synchronous firing in a network with gext > gint wasalso observed when the bifurcation parameters rE and rIwere set as asymmetric (Kanamaru, 2006a). This simi-larity might have arisen because electrical synapses onlyin the inhibitory assembly introduced asymmetry to thenetwork. Moreover, in Figure 1B, it is also observed thatthe area where a chaotic solution exists widened by in-troducing the electrical synapses. The dependence of JE

on gext in a network with ggap = 0.15 is shown in Fig-ure 2B. Similarly to Figure 2A, JE tended to decreaseas gext increased. Moreover, both periodic solutions andchaotic solutions existed in networks with gext of up to4.5, and periodic solutions appeared again in networkswith gext > 8.35. It is also observed that in networkswith relatively large gext, the periodic oscillations andthe chaotic oscillations had small amplitudes near theorigin. Typical chaotic oscillations in the (JE , JI) plane

3

-0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0 2 4 6 8 10

gext

JE

D=0.006, g =0.15gap

6 7 8 9 100

0.004

0.008

Hopf

Hopf

Hopf

Hopf

B

-0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0 2 4 6 8 10

gext

JE

D=0.006, g =0gap

Hopf

Hopf

SN

SN

A

Figure 2: Dependence of the instantaneous firing rateJE of the excitatory assembly on gext. The solid anddotted lines denote stable and unstable equilibria, re-spectively. When a stable limit cycles or a chaotic at-tractor exists in the network, their maxima and minimaare also plotted. The values of the parameters are setas (A) D = 0.006 and ggap = 0 and (B) D = 0.006and ggap = 0.15. The inset in (B) shows an enlargementin the range 6 ≤ gext ≤ 10 where the vertical axis wasexpanded.

with gext = 3.9 or gext = 4.4 are shown in Figures 3Aand 3B, respectively. Note that the chaotic attractor inFigure 3B is smaller than that in Figure 3A. This is be-cause the value of gext in Figure 3B is larger than thatin Figure 3A.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 0.04 0.08 0.12 0.16

JE

JI

A gext =3.9, D=0.006

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 0.04 0.08 0.12 0.16

JE

JI

B g ext =4.4, D=0.0045

Figure 3: Chaotic attractors in the (JE , JI) plane forgint = 5 and ggap = 0.15. The values of the other pa-rameters are set as (A) gext = 3.9 and D = 0.006 and(B) gext = 4.4 and D = 0.0045. It is observed that theattractor decreases in size as gext increases.

3 Stochastic Synchrony of Chaos

In the previous section, it was found that introduction ofelectrical synapses in the inhibitory assembly had the fol-lowing effects on the network. First, the ranges of the pa-rameters where periodically or chaotically synchronizedfiring can be observed widened. Second, the periodic orchaotic solution of the network persisted even when gext

was relatively large, and such solution tended to havesmall amplitudes near the origin.Note that oscillations with small amplitudes close

to the origin correspond to weakly synchronized firing(Kanamaru, 2006a; Kanamaru & Sekine, 2004), wherethe ensemble-averaged dynamics shows synchronizationin the network but each neuron has a low firing rateand the firing of each neuron seems to be stochastic. In

4

0

5

10

15

20

0 100 200 300 400 500

0

0.2

0.4

0.6

0 100 200 300 400 500

0

5

10

15

20

0 100 200 300 400 500

0

0.04

0.08

0.12

0.16

0 100 200 300 400 500

t

JE

JI

inde

x of

neu

rons

(I)

inde

x of

neu

rons

(E

)A

B

C

D

gext =3.9, D=0.006 f all =0.038

f 1 =0.041

f all =0.046

f 1 =0.034

Figure 4: The firing of neurons in a finite network withNE = NI = 1000, gint = 5, gext = 3.9, ggap = 0.15,and D = 0.006. Note that the values of the parame-ters are the same with those used in Figure 3A. (A),(C) Temporal change in the instantaneous firing rate ofthe excitatory assembly and the inhibitory assembly, re-spectively. (B), (D) Raster plot of the firing times of theneurons in the excitatory assembly and in the inhibitoryassembly, respectively. Each excitatory neuron fires atleast once at each peak of JE ; therefore, stochastic syn-chrony is not observed in the excitatory assembly. Onthe other hand, some inhibitory neurons do not fire evenwhen JI takes peak values, and this phenomenon is asign of stochastic synchrony.

Brunel & Hansel (2006) and Tiesinga & Jose (2000), sim-ilar phenomena were called stochastic synchrony. To ourknowledge, stochastic synchrony in previous studies wasbased on periodic oscillations, and stochastic synchrony

0

0.02

0.04

0.06

0.08

0 100 200 300 400 500

0

0.04

0.08

0.12

0.16

0 100 200 300 400 500

0

5

10

15

20

0 100 200 300 400 500

0

5

10

15

20

0 100 200 300 400 500

t

JE

JIin

dex

of n

euro

ns (

I)in

dex

of n

euro

ns (

E)

A

B

C

D

gext =4.4, D=0.0045 f all =0.046

f 1 =0.017

f all =0.040

f 1 =0.013

Figure 5: The firing of neurons in a finite network withNE = NI = 1000, gint = 5, gext = 4.4, ggap = 0.15,and D = 0.0045. Note that the values of the parame-ters are the same with those used in Figure 3B. (A), (C)Temporal change in the instantaneous firing rate of theexcitatory assembly and the inhibitory assembly, respec-tively. (B), (D) Raster plot of the firing of the neurons inthe excitatory assembly and in the inhibitory assembly,respectively. Stochastic synchrony of chaos is observedin both assemblies.

that corresponds to chaotic oscillations, in other words,stochastic synchrony of chaos, had not been observed.Here we present the stochastic synchrony of chaos ob-served in our network.First, the synchronous firing that corresponds to the

chaotic attractor shown in Figure 3A, is shown in Fig-ure 4. The trajectory shown in Figure 3A is a solu-tion of the Fokker-Planck equation which holds in the

5

limit of an infinite number of neurons, and the dynamicsshown in Figure 4 is the behavior of a finite network withNE = NI = 1000. The stochastic differential equations2.1 and 2.2 are integrated numerically in Stratonovich’ssense based on the method in Klauder & Petersen (1985).The instantaneous firing rate JX in assembly X is de-fined as

JX(t) ≡ 1NXd

NX∑i=1

∑j

Θ(t− t(i)j ), (3.1)

Θ(t) ={

1 for 0 ≤ t < d0 otherwise , (3.2)

where d = 1.0. As shown in Figure 4B, each excitatoryneuron fires at least once at each peak of JE . Thus,stochastic synchrony is not observed in the excitatoryneurons in the network shown in Figure 4B. To quantifythis fact, we calculated the power-spectrum PE(f) ofJE(t), defined by

PX(f) =1N

N∑j=1

1T

∣∣∣∣∣∫ tj+T

tj

JX(t)e−2πiftdt

∣∣∣∣∣2

, (3.3)

where X = E or I, t1 = 0, tj+1 = tj + T , T = 2048,and N = 21. Note that the mean value of N samples iscalculated to obtain smooth PX(f), and the frequencyf has an order of 1/t although both f and t are dimen-sionless in our model. PE(f) is shown in Figure 6A,and it is observed that it has a broad spectrum becausechaos and noise coexist in JE(t). Note that the peaksat small frequencies f = 0.018 and f = 0.036 denote theslow dynamics of JE(t), and the peaks at large frequen-cies f = 0.075 and f = 0.1 denote the fast dynamics ofJE(t); therefore, the mean frequency of JE(t) would belocated in the range between the frequencies which de-note the slow dynamics and the fast dynamics. Based onthe method shown in Appendix C, the mean frequencyfall of JE(t) is calculated as fall = 0.038. On the otherhand, the frequency f1 of the firing of excitatory neuronsis calculated as f1 = 0.041. Because f1 is close to fall,it can be concluded that stochastic synchrony of chaosdoes not take place in the dynamics of the excitatory as-sembly shown in Figures 4A and 4B. On the other hand,some inhibitory neurons do not fire even when JI takespeak values as shown in Figure 4D. This phenomenonis a sign of stochastic synchrony. The mean frequencyfall of JI(t) is calculated as fall = 0.046, and f1 of thefiring of inhibitory neurons is calculated as f1 = 0.034.Because f1 is smaller than fall, it can be concluded thatstochastic synchrony of chaos takes place in the dynam-ics of the inhibitory assembly shown in Figures 4C and4D.The synchronous firing that corresponds to the chaotic

attractor shown in Figure 3B, is shown in Figure 5. Itis observed that both the firing of the excitatory neu-rons and that of the inhibitory ones are sparse and lookrandom, and that very few neurons fire even when JE

0 0.04 0.08 0.12 0.16 0.2

1e-4

0.001

0.01

0.1

1e-5

1e-4

0.001

0.01

0.1

f

P (f)E

P (f)I

1e-5f =0.0341

f =0.038allf =0.0411

A gext =3.9, D=0.006

0 0.04 0.08 0.12 0.16 0.2

1e-5

1e-4

0.001

0.01

0.1

1e-5

1e-4

0.001

0.01

0.1

f

P (f)E

P (f)I

f =0.0131 f =0.040all

f =0.046all

f =0.0171

B gext =4.4, D=0.0045

f =0.046all

periodic structures

periodic structures

Figure 6: The power-spectra PE(t) and PI(f) of JE(t)and JI(t) in the case where (A) gext = 3.9 and D =0.006, and (B) gext = 4.4 and D = 0.0045, which cor-respond to the firing patterns shown in Figure 4 and 5,respectively. Note that the frequency f has an orderof 1/t although both f and t are dimensionless in ourmodel. The white arrows show the positions of the pe-riodic structures. The positions of the mean frequencyfall of the assembly and the frequency f1 of the firingof each neuron are indicated by thin vertical arrows.The stochastic synchrony of chaos is observed in the in-hibitory assembly in (A), and it is observed in both theassemblies in (B).

and JI take peak values. These firing patterns repre-sent stochastic synchrony that corresponds to a chaoticattractor; therefore, we call these firing patterns asstochastic synchrony of chaos. The power-spectra ofJE(t) and JI(t), the mean frequency fall of each assem-bly, and the frequency f1 of the firing of each neuronare shown in Figure 6B. In both the assemblies, it is ob-served that f1 is smaller than fall; therefore, it can beconcluded that stochastic synchrony of chaos takes placein both the assemblies.Stochastic synchrony is realized when the oscillations

of the ensemble-averaged dynamics in the network have

6

small amplitudes as shown in Figure 3B. In such a case,the amplitude of the feedback input from the networkis also small, and it causes sub-threshold oscillations foreach neuron in the network and stochastic synchrony(Kanamaru & Sekine, 2004). This mechanism is similarto that of stochastic resonance (Gammaitoni, Hanggi,Jung, & Marchesoni, 1998; Longtin, 1993), which is re-alized when a small periodic signal and an appropriateamount of noise are injected to an excitable element. Inour model, the signal and noise are defined by 2.3 and2.5, respectively, and the periodic or chaotic signal isgenerated by the internal dynamics of the network.

4 Roles of Types of Connectionsamong Neuronal Assemblies

In the previous sections, various kinds of synchronousfiring including chaotic synchrony were found in the net-work of excitatory and inhibitory neurons by regulatingthe external connection strength gext. In this section,the roles of the connection strengths gEE , gIE , gEI , andgII are examined.In the previous sections, the restrictions gEE = gII =

gint and gIE = gEI = gext were placed in order to ana-lyze the properties of the synchronous firing in the net-work. In this section, we fix gint and gext at values atwhich synchronous firing exists, and then we change thevalue of only one of gEE , gIE , gEI , or gII while keep-ing the other three values fixed. By analyzing the de-pendence of the average 〈JE〉 of JE over time and thevariance Var(JE) of JE on each connection strength, theroles of each connection could be understood. Particu-larly, the variance Var(JE) takes the value close to zerowhen the firing pattern of neurons is fully asynchronous,and it takes non-zero values when the firing of each neu-ron is synchronized. Thus, it can be used to measure thedegree of synchronization.First, let us analyze the case where there is no elec-

trical synapse in the network (ggap = 0). The param-eters are initially fixed at gEE = gII = gint = 5,gIE = gEI = gext = 3, and D = 0.006, where peri-odically synchronized firing is observed in the network.The dependence of 〈JE〉 and Var(JE) on the connec-tion strength was analyzed by changing the value of oneconnection strength, and the results are shown in Fig-ures 7A and 7B, respectively. Note that the vertical axisin Figure 7B is log-scaled, and Var(JE) takes the valueclose to zero when there is no plot. The following prop-erties are observed in Figure 7. First, synchronous fir-ing disappears (i.e., Var(JE)∼ 0) when one connectionstrength is set to zero. In other words, all four typesof connections are required for a genesis of synchroniza-tion. Second, strong synchronization is observed whengIE and gEI are smaller than gEE and gII . Third, ad-justment of the values of gIE and gEI is required to ob-serve synchronous firing because Var(JE) takes non-zerovalues only in some ranges of gIE and gEI as shown in

1e-07

1e-06

1e-05

0.0001

0.001

0.01

0 2 4 6 8 10g

Var(J )E

0

0.1

0.2

0.3

0.4

0.5

0 2 4 6 8 10g

A

B

gIEgEI gII

gEE

gIEgEI gII

gEE

<J >Eg

gap=0

Figure 7: The dependence of (A) the average 〈JE〉 ofJE over time and (B) the variance Var(JE) of JE oneach connection strength gEE , gII , gIE , or gEI , in anetwork without electrical synapses. The initial valuesof the parameters are set as gEE = gII = 5, gIE = gEI =3, ggap = 0, and D = 0.006. The value of only oneconnection strength, gEE , gII , gIE , or gEI , was varied.

Figure 7B. On the other hand, as for gEE and gII , syn-chronous firing can be observed with sufficiently stronggEE and gII . Fourth, in networks with large gEE, 〈JE〉increases as gEE increases as shown in Figure 7A. On theother hand, in networks with large gII , 〈JE〉 remainedat a nearly constant value. The above properties of syn-chronous firing also held for different values of gint, gext,and D.Next, we analyze the roles of the connections when

there are electrical synapses in the network (ggap =0.15). The parameters are initially fixed at gEE = gII =gint = 5, gIE = gEI = gext = 4.4, and D = 0.0045,where stochastic synchrony of chaos is observed in thenetwork. Note that these values of the parameters arethe same with those used in Figures 3B and 5. Thedependence of 〈JE〉 and Var(JE) on each connectionstrength is shown in Figures 8A and 8B, respectively.There are similarities and differences compared with the

7

0

0.1

0.2

0.3

0.4

0.5

0 2 4 6 8 10

1e-07

1e-06

1e-05

0.0001

0.001

0.01

0 2 4 6 8 10g

g

A

B

gIEgEI

gII

gEE

gIEgEI gII

gEE

<J >E

Var(J )E

ggap

=0.15

Figure 8: The dependence of (A) the average 〈JE〉 of JE

over time and (B) the variance Var(JE) of JE on eachconnection strength gEE, gII , gIE, or gEI , in a networkwith electrical synapses. The initial values of the pa-rameters are set as gEE = gII = 5, gIE = gEI = 4.4,ggap = 0.15, and D = 0.0045. Plateau-like structuresthat indicate stochastic synchrony are observed in (B)for systems with small gII , large gIE, or large gEI .

case of ggap = 0 in Figure 7. The plateau-like struc-tures of Var(JE) observed for small gII , large gIE , orlarge gEI show stochastic synchrony. This phenomenoncan be understood as follows. Stochastic synchrony canbe observed when gext is large as shown in Figure 1B.Therefore, it is natural that stochastic synchrony existswhen gIE or gEI is large. Moreover, Figure 8 shows thata decrease in gII has similar effects as an increase in gext

in the network.

5 Discussion and Conclusions

In the present study, we analyzed the synchronous fir-ing in a pulse coupled neural network of excitatoryand inhibitory neurons that are connected by chemicalsynapses. Electrical synapses among the inhibitory neu-

rons were introduced to this network. Bifurcation struc-ture of the ensemble-averaged dynamics of neurons in thenetwork was analyzed with the Fokker-Planck equationwhich is obtained in the limit of an infinite number ofneurons. It was concluded that by introducing the elec-trical synapses to the network, the range of the parame-ter values where periodically or chaotically synchronizedfiring can be observed widened. Moreover, the periodicor chaotic solution observed in the network with electri-cal synapses persisted even for networks with large gext,and it was found that the solutions for networks withlarge gext had small amplitudes near the origin. Theoscillations with small amplitudes near the origin cor-responded to stochastic synchrony where the ensemble-averaged dynamics shows synchronization in the networkbut each neuron has a low firing rate and the firing seemsto be stochastic. In the present study, we found stochas-tic synchrony that corresponded to a chaotic attractor,and we called this phenomenon stochastic synchrony ofchaos.The roles of the four types of connections among ex-

citatory and inhibitory assemblies gEE , gIE , gEI , andgII were also investigated. It was found that in a net-work without electrical synapses, all four types of con-nections were required for a genesis of synchronous firing.This result seems to contradict the previous finding thatthe pulse coupled self-oscillating neurons can perfectly orpartially synchronize with each other without inhibitoryneurons (Hansel, Mato, & Meunier, 1995; Kuramoto,1991; Mirollo & Strogatz, 1990; Tsodyks, Mitkov, &Sompolinsky, 1993; van Vreeswijk, 1996) or without ex-citatory neurons (Golomb & Rinzel, 1993; Kopell, 2000;van Vreeswijk, Abbott & Ermentrout, 1994). This dis-crepancy is caused by the fact that our network is com-posed of excitable, not self-oscillating neurons. In addi-tion, it was found that adjustment of gEI and gIE was re-quired. As to networks with electrical synapses, stochas-tic synchrony was observed in networks with small gII ,large gIE , or large gEI .Although stochastic synchrony is often observed in

sparsely connected networks (Brunel, 2000; Brunel &Hakim, 1999; Traub, Miles, & Wong, 1989), all neuronsin our network were connected with each other and theconnections were uniform. Thus, it can be concludedthat randomness of connections is not required for a gen-esis of stochastic synchrony. Moreover, there are cases inwhich stochastic synchrony is observed both in the exci-tatory assembly and in the inhibitory assembly (Figures5B and 5D). This was observed because the peak valuesof both JE and JI were small. It is inferred that therewould be cases where stochastic synchrony is observedeither in the excitatory assembly or in the inhibitory as-sembly according to the shape of the attractor.When stochastic synchrony exists in a network, the

contribution of a single neuron in the network to thesynchronization is small. Therefore, this neuron mightalso contribute to form other synchronous networks si-multaneously. If such dynamics is realized, they can be

8

interpreted as the stochastic realization of the dynamicalcell assemblies (Fujii et al., 1996; Hebb, 1949). Investi-gation of such dynamics is an important future problem.

Acknowledgement

This study was partially supported by a Grant-in-Aid forEncouragement of Young Scientists (B) (No. 17700226)and a Grant-in-Aid for Scientific Research on PriorityAreas (No. 17022012) from The Ministry of Education,Culture, Sports, Science and Technology of Japan.

A The Fokker-Planck Equation

for the System

To analyze the dynamics of the network, we use theFokker-Planck equations (Gerstner & Kistler, 2002; Ku-ramoto, 1984) which are represented as

∂nE

∂t= − ∂

∂θE(AEnE)

+D

2∂

∂θE

{BE

∂

∂θE(BEnE)

}, (A.1)

∂nI

∂t= − ∂

∂θI(AInI)

+D

2∂

∂θI

{BI

∂

∂θI(BInI)

}, (A.2)

AE(θE , t) =1τE

(1− cos θE) +1τE

(1 + cos θE)

×(rE + gEEIE(t)− gEIII(t)), (A.3)

AI(θI , t) =1τI(1− cos θI) +

1τI(1 + cos θI)

×(rI + gIEIE(t)− gIIII(t) + ggapIgap(θI , t)),(A.4)

BE(θE , t) =1τE

(1 + cos θE), (A.5)

BI(θI , t) =1τI(1 + cos θI), (A.6)

Igap(θI , t) = 〈sin θI〉 cos θI − 〈cos θI〉 sin θI , (A.7)

〈f(θI)〉 =∫ 2π

0

f(θI)nI(θI , t) dθI , (A.8)

for the normalized number densities of excitatory andinhibitory neurons, in which

nE(θE , t) ≡ 1NE

∑δ(θ(i)E − θE), (A.9)

nI(θI , t) ≡ 1NI

∑δ(θ(i)I − θI), (A.10)

in the limit of NE, NI → ∞. The probability flux foreach assembly is defined as

JE(θE , t) = AEnE − D

2BE

∂

∂θE(BEnE),(A.11)

JI(θI , t) = AInI − D

2BI

∂

∂θI(BInI), (A.12)

respectively. In the limit of NX → ∞, IX(t) in equation2.3 follows the following differential equation,

˙IX(t) = − 1κX

(IX(t)− 1

2JX(t)

), (A.13)

where JX(t) ≡ JX(π, t) is the probability flux at θX = π.By integrating the Fokker-Planck equations A.1 and

A.2 and the differential equation A.13 simultaneously,the dynamics of the network that is governed by equa-tions 2.1 and 2.2 can be analyzed.

B Numerical Integration of the

Fokker-Planck Equations

In this section, we provide a method on the numericalintegration of the Fokker-Planck equations A.1 and A.2.Because the normalized number densities given by equa-tions A.9 and A.10 are 2π-periodic functions of θE andθI , respectively, they can be expanded as

nE(θE , t) =12π

+∞∑

k=1

(aEk (t) cos(kθE) + b

Ek (t) sin(kθE)),

(B.1)

nI(θI , t) =12π

+∞∑

k=1

(aIk(t) cos(kθI) + b

Ik(t) sin(kθI)),

(B.2)

and, by substituting them, equations A.1 and A.2 aretransformed into a set of ordinary differential equationsof aX

k and bXk as follows:

da(X)k

dt= −(rX + IX + 1)

k

τXb(X)k

−(rX + IX − 1)k

2τX(b(X)

k−1 + b(X)k+1)

− Dk8τ2

X

f(a(X)k )

+πggapk

4τX(−b1g1(b(X)

k ) + a1g2(a(X)k ))δXI ,

(B.3)

db(X)k

dt= (rX + IX + 1)

k

τXa(X)k

+(rX + IX − 1)k

2τX(a(X)

k−1 + a(X)k+1)

− Dk8τ2

X

f(b(X)k )

+πggapk

4τX(b1g1(a

(X)k ) + a1g2(b

(X)k ))δXI ,

(B.4)f(xk) = (k − 1)xk−2 + 2(2k − 1)xk−1 + 6kxk

+2(2k + 1)xk+1 + (k + 1)xk+2, (B.5)g1(xk) = xk−2 + 2xk−1 + 2xk + 2xk+1 + xk+2,(B.6)

9

g2(xk) = xk−2 + 2xk−1 − 2xk+1 − xk+2, (B.7)IX ≡ gXEIE − gXIII , (B.8)

a(X)0 ≡ 1

π, (B.9)

b(X)0 ≡ 0, (B.10)

a(X)−n ≡ a(X)

n , (B.11)

b(X)−n ≡ −b(X)

n , (B.12)

where X = E or I. Using a vector x =(IE , II , aE

1 , bE1 , a

I1, b

I1, a

E2 , b

E2 , a

I2, b

I2, · · ·)t, the ordinary

differential equations x = f(x) are defined by A.13,B.3, and B.4. By integrating this ordinary differentialequations numerically, the time series of the probabilityfluxes JE and JI are obtained. For numerical calcula-tions, each Fourier series is truncated at the first 40 or60 terms.The bifurcation sets of the Hopf bifurcation and the

saddle-node bifurcation in Figure 1 were obtained as fol-lows. A stationary solution xs was numerically obtainedby the Newton method (Press, Flannery, Teukolsky, &Vetterling, 1988), and the eigenvalues of the Jacobianmatrix Df(xs) that had been numerically obtained byusing the QR algorithm (Press, Flannery, Teukolsky, &Vetterling, 1988), were examined to find the bifurcationsets. On the other hand, the bifurcation sets of thehomoclinic bifurcation were obtained by observing thelong-time behaviors of the solutions of x = f(x).

C Derivation of the Mean Fre-quency of the Assembly

In this section, we provide a method on the numericalcalculation of the mean frequency fall of the assembly inthe network with a finite number of neurons. As shownin Figures 4A, 4C, 5A, and 5C, JX(t) (X = E or I) ofthe finite network is noisy. To eliminate noise, we usethe the low-pass filter written by

xout = −1τxout +

1τxin, (C.1)

where xin and xout are the input and the output of thisfilter, respectively, and its cutoff frequency is defined by

fc =1

2πτ. (C.2)

The cutoff frequency fc is set at the maximal fre-quency where the peak of the power-spectrum is ob-served, namely, fc = 0.1 for gext = 3.9 and D = 0.006 inFigure 6A, and fc = 0.06 for gext = 4.4 and D = 0.0045in Figure 6B. After applying this low-pass filter to JX(t)twice, we count the number of peaks of JX(t). JX(t∗)at t = t∗ is considered as a peak when two conditionsJX(t∗) > JX(t∗ ± ∆t) and JX(t∗) > 〈JX(t)〉 are sat-isfied, where ∆t is the time step of data, and 〈JX(t)〉is the time-average of JX(t). The second condition was

required to avoid spurious peaks caused by noise. Aftercounting the peaks, we define the mean frequency fall ofthe assembly as the number of peaks of JX(t) per unittime.

ReferencesBragin, A., Jando, G., Nadasdy, Z., Hetke, J., Wise,K., & Buzsaki, G. (1995). Gamma (40-100 Hz)oscillation in the hippocampus of the behaving rat. TheJournal of Neuroscience, 15, 47–60.

Brunel, N. (2000). Dynamics of sparsely connectednetworks of excitatory and inhibitory spiking neurons.Journal of Computational Neuroscience, 8, 183–208.

Brunel, N. & Hakim, V. (1999). Fast global oscillationsin networks of integrate-and-fire neurons with low firingrates. Neural Comput., 11, 1621–1671.

Brunel, N. & Hansel, D. (2006). How noise affects thesynchronization properties of recurrent networks ofinhibitory neurons. Neural Comput., 18, 1066–1110.

Buzsaki, G. (2006). Rhythms of the brain, (OxfordUniversity Press, New York).

Buzsaki, G., Horvath, Z., Urioste, R., Hetke, J., &Wise, K. (1992). High-frequency network oscillation inthe hippocampus. Science, 256, 1025–1027.

Csicsvari, J., Hirase, H., Czurko, A., & Buzsaki, G.(1998). Reliability and state dependence of pyramidalcell-interneuron synapses in the hippocampus: anensemble approach in the behaving rat. Neuron, 21,179–189.

Ermentrout, B. (1996). Type I membranes, phaseresetting curves, and synchrony. Neural Comput., 8,979–1001.

Ermentrout, B. (2006). Gap junctions destroypersistent states in excitatory networks. Phys. Rev. E,74, 031918.

Ermentrout, G. B., & Kopell, N. (1986). Parabolicbursting in an excitable system coupled with a slowoscillation. SIAM J. of Appl. Math., 46, 233–253.

Fisahn, A., Pike, F. G., Buhl, E. H., & Paulsen, O.(1998). Cholinergic induction of network oscillations at40 Hz in the hippocampus in vitro. Nature, 394,186–189.

Fujii, H., Ito, H., Aihara, K., Ichinose, N., & Tsukada,M. (1996). Dynamical cell assembly hypothesis –

10

Theoretical possibility of spatio-temporal coding in thecortex. Neural Networks, 9, 1303–1350.

Fukuda, T., & Kosaka, T. (2000). Gap junctionslinking the dendritic network of GABAergicinterneurons in the hippocampus. The Journal ofNeuroscience, 20, 1519–1528.

Galarreta, M. & Hestrin, S. (1999). A network offast-spiking cells in the neocortex connected byelectrical synapse. Nature, 402, 72–75.

Galarreta, M. & Hestrin, S. (2001). Electrical synapsesbetween GABA-releasing interneurons. Nature ReviewsNeuroscience, 2, 425–433.

Gammaitoni, L., Hanggi, P., Jung, P., & Marchesoni,F. (1998). Stochastic resonance. Reviews of ModernPhysics, 70, 223–287.

Gerstner, W. & Kistler, W. (2002). Spiking NeuronModels (Cambridge Univ. Press, Cambridge).

Gibson, J. R., Beierlein, M., & Connors, B. W. (1999).Two networks of electrically coupled inhibitory neuronsin neocortex. Nature, 402, 75–79.

Golomb, D. & Rinzel, J. (1993). Dynamics of globallycoupled inhibitory neurons with heterogeneity. PhysicalReview E, 48, 4810–4814.

Gray, C. M. (1994). Synchronous oscillations inneuronal systems: mechanisms and functions. J.Comput. Neurosci., 1, 11–38.

Gray, C. M. (1999). The temporal correlationhypothesis of visual feature integration: still alive andwell. Neuron, 24, 31–47.

Gray, C. M. & Singer, W. (1989). Stimulus-specificneuronal oscillations in orientation columns of catvisual cortex. Proc. Natl. Acad. Sci. USA, 86,1698–1702.

Hansel, D., Mato, G., & Meunier, C. (1995) Synchronyin excitatory neural networks. Neural Comput., 7,307–337.

Hebb, D. O. (1949). The Organization of Behavior – aneuropsychological theory (John Wiley, New York).

Izhikevich, E. M. (1999). Class 1 neural excitability,conventional synapses, weakly connected networks, andmathematical foundations of pulse-coupled models.

IEEE Trans. Neural Networks, 10, 499–507.

Izhikevich, E. M. (2000). Neural excitability, spikingand bursting. Int. J. Bifurcation and Chaos, 10,1171–1266.

Kanamaru, T. (2006a). Analysis of synchronizationbetween two modules of pulse neural networks withexcitatory and inhibitory connections. Neural Comput.,18, 1111–1131.

Kanamaru, T. (2006b). Blowout bifurcation and on-offintermittency in pulse neural networks with multiplemodules. Int. J. Bifurcation and Chaos, 16, 3309–3321.

Kanamaru, T. & Sekine, M. (2004). An analysis ofglobally connected active rotators with excitatory andinhibitory connections having different time constantsusing the nonlinear Fokker-Planck equations. IEEETransactions on Neural Networks, 15, 1009–1017.

Katsumaru, H., Kosaka, T., Heizmann, C. W., &Hama, K. (1988). Gap junctions on GABAergicneurons containing the calcium-binding proteinparvalbumin in the rat hippocampus (CA1 region).Experimental Brain Research, 72, 363–370.

Klauder, J. R. & Petersen, W. P. (1985). Numericalintegration of multiplicative-noise stochastic differentialequations. SIAM J. Numer. Aanal., 22, 1153–1166.

Kopell, N. (2000). We got rhythm: dynamical systemsof the nervous system. Notices of the AMS, 47, 6–16.

Kuramoto, Y. (1984). Chemical oscillations, waves,and turbulence (Springer, Berlin).

Kuramoto, Y. (1991). Collective synchronization ofpulse-coupled oscillators and excitable units. PhysicaD, 50, 15–30.

Lang, E. J., Sugihara, I., & Llinas, R. (1996).GABAergic modulation of complex spike activity bythe cerebellar nucleoolivary pathway in rat. Journal ofNeurophysiology, 76, 255–275.

Longtin, A. (1993). Stochastic resonance in neuronmodels. Journal of Statistical Physics, 70, 309–327.

Mirollo, R. E. & Strogatz, S. H. (1990).Synchronization of pulse-coupled biological oscillators.SIAM Journal of Applied Mathematics, 50, 1645–1662.

Morris, C. & Lecar, H. (1981). Voltage oscillations inthe barnacle giant muscle fiber. Biophys. J., 35,

11

193–213.

Nicholls, J. G., Martin, A. R., Wallace, B. G., & Fuchs,P. A. (2001). From Neuron to Brain. SinauerAssociates Inc. Publishers, Massachusetts.

Press, W. H., Flannery, B. P., Teukolsky, S. A., &Vetterling, W. T. (1988). Numerical Recipes in C,Cambridge Univ. Press, New York.

Strata, F., Atzori, M., Molnar, M., Ugolini, G.,Tempia, F., & Cherubini, E. (1997). A pacemakercurrent in dye-coupled hilar interneurons contributes tothe generation of giant GABAergic potentials indeveloping hippocampus. The Journal of Neuroscience,17, 1435–1446.

Tiesinga, P. H. E., & Jose, J. V., (2000). Robustgamma oscillations in networks of inhibitoryhippocampal interneurons. Network, 11, 1-23.

Traub, R. D., Miles, R., & Wong, K. S., (1989). Modelof the origin of rhythmic population oscillations in thehippocampal slice. Science, 243, 1319–1325.

Tsodyks, M., Mitkov, I., & Sompolinsky, H. (1993).Pattern of synchrony in inhomogeneous networks ofoscillators with pulse interactions. Physical ReviewLetters, 71, 1280–1283.

van Vreeswijk, C. (1996) Partial synchronization inpopulations of pulse-coupled oscillators. PhysicalReview E, 54, 5522–5537.

van Vreeswijk, C., Abbott, L. F., & Ermentrout, G. B.(1994). When inhibition not excitation synchronizesneural firing. Journal of Computational Neuroscience,1, 313–321.

Whittington, M. A., Traub, R. D., Kopell, N.,Ermentrout, B., & Buhl, E. H., (2000).Inhibition-based rhythms: experimental andmathematical observations on network dynamics.International Journal of Psychophysiology, 38, 315–336.

Zhang, Y., Velazquez, J. L. P., Tian, G. F., Wu, C.-P.,Skinner, F. K., Carlen, P. L., & Zhang, L. (1998). Slowoscillations (≤ 1 Hz) mediated by GABAergicinterneuronal networks in rat hippocampus. TheJournal of Neuroscience, 15, 9256–9268.

12