parametric and non-parametric modeling of short …1 introduction short-term synaptic plasticity...
TRANSCRIPT
Parametric and non-parametric modeling of short-termsynaptic plasticity. Part I: computational study
Dong Song & Vasilis Z. Marmarelis &
Theodore W. Berger
Received: 21 November 2007 /Revised: 8 April 2008 /Accepted: 1 May 2008# Springer Science + Business Media, LLC 2008
Abstract Parametric and non-parametric modeling meth-ods are combined to study the short-term plasticity (STP) ofsynapses in the central nervous system (CNS). Thenonlinear dynamics of STP are modeled by means: (1)previously proposed parametric models based on mecha-nistic hypotheses and/or specific dynamical processes, and(2) non-parametric models (in the form of Volterra kernels)that transforms the presynaptic signals into postsynapticsignals. In order to synergistically use the two approaches,we estimate the Volterra kernels of the parametric modelsof STP for four types of synapses using synthetic broad-band input–output data. Results show that the non-parametric models accurately and efficiently replicate theinput–output transformations of the parametric models.Volterra kernels provide a general and quantitative repre-sentation of the STP.
Keywords Facilitation . Depression . Nonlinear modeling .
Poisson random train . Volterra kernels
1 Introduction
Short-term synaptic plasticity (STP) is the use-dependentalteration in the strength of synaptic transmission over atime scale of milliseconds to seconds, whereby themagnitudes of postsynaptic responses are dynamicallyaffected by the pattern of recent input impulses (Zuckerand Regehr 2002). This is illustrated in Fig. 1, where a trainof presynaptic action potentials (inputs) is shown to elicitpostsynaptic responses (outputs) with varying amplitudesdepending on the temporal pattern of previous presynapticactivity. This phenomenon may also be viewed as adynamic nonlinearity (Sclabassi et al. 1988; Berger et al.1994). STP is considered a critical component of synaptictransmission and information processing in the brain(Orban et al. 1985; Reid et al. 1991; Abbott et al. 1997;Lisman 1997; Zador and Dobrunz 1997).
The study on STP dates back to the 1940s (Eccles et al.1941; Feng 1941). Many underlying mechanisms of STPhave been inferred from numerous studies involvingvarious experimental techniques and preparations (Katzand Miledi 1968; Creager et al. 1980; Debanne et al. 1996;Dobrunz and Stevens 1997; Dittman and Regehr 1998;Hanse and Gustafsson 2001). These mechanisms aregenerally classified into two major categories: facilitation(when the presence of previous presynaptic events causesincreased postsynaptic response at present) and depression(when the opposite effect occurs). The most widelyaccepted biological mechanisms for such STP effects arethe residual calcium hypothesis of facilitation and thedepletion model of depression (Liley and North 1953; Katzand Miledi 1968; Betz 1970). According to these hypoth-eses, facilitation is caused by the accumulation of residualcalcium in the presynaptic bouton, while depression is dueto the depletion of the neurotransmitter vesicle pool.
J Comput NeurosciDOI 10.1007/s10827-008-0097-3
Action Editor: David Golomb
D. Song (*) :V. Z. Marmarelis : T. W. BergerDepartment of Biomedical Engineering,University of Southern California,403 Hedco Neuroscience Building,Los Angeles, CA 90089, USAe-mail: [email protected]
T. W. BergerProgram in Neuroscience, University of Southern California,Los Angeles, USA
D. Song :V. Z. Marmarelis : T. W. BergerCenter for Neural Engineering, University of Southern California,Los Angeles, USA
Additionally, the recovery of the depleted vesicle pool hasbeen shown to be dependent on the residual calcium con-centration (Dittman and Regehr 1998; Wang and Kaczmarek1998; Hosoi et al. 2007). Depletion-independent mechanismsof depression were also reported (Hsu et al. 1996; Dobrunzet al. 1997; Kraushaar and Jonas 2000; Waldeck et al. 2000;Gover et al. 2002; Kirischuk et al. 2002; Pedroarena andSchwarz 2003; Fuhrmann et al. 2004). Furthermore, post-synaptic mechanisms (e.g., the desensitization of AMPAreceptors) are potential contributors to synaptic depression(Trussell and Fischbach 1989; Trussell et al. 1993; Jones andWestbrook 1996).
Many STP models have been developed with distinctscientific aims (Mallart and Martin 1967; Friesen 1975;Magleby and Zengel 1975; Krausz and Friesen 1977;Zengel and Magleby 1982; Yamada and Zucker 1992; Senet al. 1997; Tsodyks and Markram 1997; Varela et al. 1997;Tsodyks et al. 1998; Hunter and Milton 2001). In oneapproach, the model is built to explain the physiologicalmechanisms underlying synaptic transmission. One excel-lent example of such model was proposed by Dittman et al.(2000). In their model, the STP dynamics is described bythe interplay between facilitation, depression and residualcalcium at the presynaptic terminal. This type of model canbe termed “parametric model” due to the fact that it isexpressed with very specific model structures/functionsinspired by physiological mechanisms, and a number ofadjustable parameters that can be related to certain biologicalprocesses. It has a predictive power in uncovering the natureof the underlying physiological processes/mechanisms. Forexample, by altering the values of its key model parameters,e.g., initial release probability of synapse, one can replicateseveral distinct forms of STP dynamical characteristics in aphysiologically interpretable manner. In terms of reproducingthe STP dynamics, such models are often qualitative due tothe aim it is developed for.
The other class of STP models were developed toquantitatively identify/replicate the nonlinear dynamicalinput–output characteristics of STP (Krausz and Friesen1977; Zengel and Magleby 1982; Sen et al. 1997; Varela etal. 1997). This can be done in a parametrical fashion: the
model structure can be determined based on the principlephysiological mechanisms, e.g., facilitation and depletionfor STP. The values of the unknown parameters can then beestimated from experimental data so that the resulted modelreplicated the nonlinear dynamical input–output transfor-mation with minimum error. However, for this specificmodeling aim, the parametric modeling approach has theinherent limitations associated with their fixed modelstructure that is determined a priori based on the partialknowledge and assumptions to the modeled system, andthus is subject to potential biases. For example, there almostcertainly exist unknown mechanisms/processes that are notdescribed by the parametric model. A secondary source oferrors may be the accuracy of the required parameterestimation under realistic input characteristics if theparameter estimation is performed on the basis of input–output data obtained with highly specific experimentalprotocols that contains built-in biases. The latter provideonly limited information about the functional characteristicsof the system and may bias the estimated parameter values.
An alternative approach is the use of “non-parametric”models that are obtained directly from input–output datacollected under broader experimental conditions (e.g.random stimuli that probe the system function with abroader repertoire of inputs) within the framework of ageneral model form (the functional Volterra series). In thenon-parametric modeling approach, no specific assump-tions are made a priori about the model structure, since themodel takes a general form that is applicable to almostall causal systems (Krausz and Friesen 1977; Marmarelisand Marmarelis 1978; Bishop 1995; Marmarelis 2004).Instead of searching for the proper parameter values withina postulated model structure, the non-parametric approachsearches for the optimal functions (Volterra kernels) con-tained within the general model that represents the input–output relationship of the system. The nonlinear dynamicsunderlying synaptic STP can be represented quantitativelyby the kernels of the Volterra model that has predictivecapability for broadband data. The key point is that thenon-parametric model utilizes a general model form(Volterra kernels), thus avoids potential errors in thepostulation of the model structure—as required in theparametric modeling approach. The unknown quantitiesin the non-parametric Volterra model are the kernels thatare estimated from input–output data collected underbroadband conditions (e.g. random inputs). Thus, thenon-parametric model, being inductive on the basis ofbroadband data, has the potential to captures thecomplete input–output nonlinear dynamical characteristicsof STP, as determined by all relevant biologicalprocesses and underlying mechanisms active under thebroadband conditions (Marmarelis and Marmarelis 1978;Berger et al. 1988a; Marmarelis 2004).
ki
synapse
Fig. 1 Illustration of the input–output data representation for thesynapse model
J Comput Neurosci
As stated above, since parametric and non-parametricmodels are developed for distinct aims and lay emphasis ondifferent aspects of the modeled system, they are comple-mentary in nature. The main aim of these two papers is tocombine both parametric and non-parametric modelingmethods in a synergistic manner to study the STP. In thispaper (part I), we introduce a recently developed variant ofVolterra modeling that employs Laguerre expansions andallows efficient high-order model representation and accu-rate kernel estimation with short input–output datasets(Marmarelis 1993). This method was applied to overcomethe major difficulty of the non-parametric model—itsrepresentational complexity caused by the possible high-order kernels required for completeness. We estimated non-parametric models of four different types of centralsynapses using synthetic broadband input–output datasimulated with published parametric models of STP (Varelaet al. 1997; Dittman et al. 2000). Results show that the non-parametric models accurately capture the input–outputrelations defined by the respective parametric models ofSTP for a broad repertoire of input patterns similar to thoseencountered under physiological conditions. The effects ofspecific parameters of the parametric models on the systeminput–output transformation are reflected directly on spe-cific features of the estimated kernels.
More importantly, since the non-parametric (Volterra)model constitutes a canonical and complete representationof the system (nonlinear) dynamics and it is not restrictedby any prior assumptions, it forms a model-free, nearlydirect representation of the input–output data themselves.The non-parametric model estimated from experimentaldata can be used as the “ground truth” to evaluateparametric models of the system in terms of their input–output transformational properties. Furthermore, the non-parametric model may suggest specific modifications in thestructure of the respective parametric model. This combinedutility of parametric and non-parametric modeling methodsis presented in the companion paper (part II).
2 Materials and methods
The input–output data used in this study result from the simu-lation of the parametric models described below. The data areanalyzed according to the Volterra modeling methodology andits recent refinements (i.e. Laguerre expansions of the Volterrakernels) that are also described below.
2.1 Representation of the input–output data
In actual experiments, the input signals at the synapses aretrains of action potentials propagating along the axon anddelivered at the presynaptic sites of the axon terminals.
Since all action potentials have very short duration (1–2ms)and almost identical shapes, they are simplified forpurposes of processing and analysis as sequences ofdiscrete impulses (Kronecker delta functions) with inter-impulse intervals encoding the input information. Theoutput signals are the quantities of neurotransmitterreleased from the vesicles of the presynaptic bouton inresponse to each action potential arriving at the presynapticsite of the axon terminal. Experimentally, the strengths ofneurotransmitter release events are quantified as amplitudesof the excitatory postsynaptic currents (EPSCs). The EPSCtrains are simplified as sequences of discrete impulses withvarying amplitudes through deconvolution with an EPSCtemplate (see companion paper for more details). Thelatencies of the EPSCs are short and approximatelyconstant (typically a fraction of 1ms) and thus can beignored. The EPSCs are recorded under voltage-clampconditions and after pharmacological manipulations so thatthe contribution of postsynaptic processes which mayintroduce nonlinearities (e.g., NMDA current and nonlineardendritic integration) are diminished. However, strongnonlinearities due to the processes of presynaptic facilita-tion and depression are maintained, as evidenced by thepresented results (see next section) of nonlinear modelingof the input–output data. The computational form ofsequences of discrete impulses for the input (fixedamplitude) and output (variable amplitude) is used for thegeneration of the simulation data. An illustration of theinput–output data representation is shown in Fig. 1.
2.2 Parametric models of the four central synapses
The parametric models of four types of CNS synapseshaving different characteristics of STP are included in thisstudy: (1) the hippocampal CA3-to-CA1 Schaffer collateralsynapse (SC); (2) the cerebellar granule cell parallel fiber-to-Purkinje cell synapse (PF); (3) the inferior olive climbingfiber-to-Purkinje cell synapse (CF); and (4) the excitatorysynapse in layer 2/3 of the visual cortex (VC).
The EPSC amplitudes for the first three synapses arecalculated through simulation of the FD model based on theresidual calcium hypothesis (Dittman et al. 2000) underconditions of random stimulation (Poisson random impulsetrains) using the following equations with different setsof model parameters for each synapse. The EPSCs of theVC synapse are simulated using the FD1D2 model (Varelaet al. 1997).
The output EPSC of the FD model is described by thefollowing equation:
EPSC tð Þ ¼ a � NT � F tð Þ � D tð Þ: ð1Þwhere α is the average mEPSC size, NT is the total numberof release sites, F(t) is the dynamical facilitation factor and
J Comput Neurosci
D(t) is the dynamical depression factor. The product FD isequal to the release probability of the synapses. Thedynamical facilitation and depression factors are describedby the equations:
F tð Þ ¼ F1 þ 1� F1
1þ KF=CaXF tð Þ ; ð2Þ
@D
@t¼ 1� D tð Þð Þ � kre cov CaXDð Þ � D t0ð Þ � F t0ð Þ
� δ t � t0ð Þ; ð3Þ
krecov CaXDð Þ ¼ kmax � k01þ KD=CaXD tð Þ þ k0; ð4Þ
where F1 is the initial probability of release, CaXF andCaXD are two putative calcium-bound molecules responsi-ble for facilitation and depression, respectively, KF is theaffinity of CaXF for the respective release site, KD is theaffinity of CaXD for the respective release site, and krecov isthe residual calcium-dependent recovery of depletion that isbounded by the baseline rate of recovery from refractorystate k0 and the maximum rate of recovery from refractorystate kmax. Eq. (2) defines an instantaneous, nonlinearsigmoidal relation between CaXF and F that bounds Fbetween F1 and 1. Eq. (3) describes the dynamic changes ofthe depression factor caused by depletion of the releasesites. When a release happens at t0, the depression factor Ddecreases by the amount of release D(t0)F(t0) since theserelease sites fall into the refractory period. The rate ofrecovery krecov is determined by the dynamical concentra-tion of CaXD according to Eq. (4) that models the residualcalcium-dependent recovery of depletion and mathemati-cally bounds krecov between k0 and kmax.
The dynamical concentrations of CaXF and CaXD are de-scribed by the following two first-order differential equations:
@ CaXF
@t¼ �CaXF tð Þ=τF þ $F � δ t � t0ð Þ; ð5Þ
@ CaXD
@t¼ �CaXD tð Þ=τDþ$D � δ t � t0ð Þ: ð6Þ
In Eqs. (5) and (6), the concentrations CaXF and CaXD
are modeled as two linear processes that decay exponen-tially with time constants τF and τD, after an impulsivechange ΔF and/or ΔD of the facilitation and/or depressionfactor, respectively—triggered by an action potentialarriving at the presynaptic site at time t0.
Since the modeled STP nonlinear dynamics depend onlyon dimensionless parameters, without loss of generality, thetwo scalars, α and NT, in Eq. (1) and the impulsive changesΔF and ΔD are set equal to 1. Note that in Table 1, theequivalent parameter for maximum paired-pulse facilitationratio ρ is listed instead of KF (Dittman et al. 2000). Theirrelation is given by:
KF=$F ¼ 1� F1
F1= 1� F1ð Þð Þ � ρ� F1� 1; ð7Þ
where ρ explicitly models the paired-pulse facilitation ratiofor the minimum inter-impulse interval and is the keyparameter of facilitation.
In the case of the fourth synapse, the excitatory synapsein layer 2/3 of the visual cortex (VC), the EPSCs arecalculated through simulation of the FD1D2 model underconditions of random stimulation (Poisson random impulsetrains) using the following equations (Varela et al. 1997):
A ¼ A0 � F � D1 � D2; ð8Þ
Table 1 Parameter values used for the simulation of the models of the four types of synapses
SC PF CF VC
Parametric model FD-residual calcium model FD1D2 modelΡ 2.2 3.1 – A0 1F1 0.24 0.05 0.35 τF 94 msτF 100 ms 100 ms – τD1 380 msτD 50 ms 50 ms 50 ms τD2 9200 msk0 2 s−1 2 s−1 0.7 s−1 f 0.917kmax 30 s−1 30 s−1 20 s−1 d1 0.416KD 2 2 2 d2 0.975
Non-parametric model PV kernel modelsL 4 4 4 10M 2000 ms 2000 ms 2000 ms 20 sα 0.984 0.984 0.990 0.998N 400 400 400 2000
J Comput Neurosci
F ! F þ f ; ð9Þ
D ! D� d; ð10Þ
tF@F
@t¼ 1� F; ð11Þ
tD@D
@t¼ 1� D: ð12Þ
The EPSC amplitude is modeled in Eq. (8) as theproduct of the initial EPSC amplitude A0 with thefacilitation factor F and the two depression factors D1 andD2. After each stimulating action potential, F is increasedby a constant f [see Eq. (9)] while D is multiplied by aconstant factor d [see Eq. (10)]. Between successive stimuli(i.e. arriving action potentials), F and D recover exponen-tially with time constants τF and τD, respectively [see Eqs.(11) and (12)]. D1 and D2 have different recovery rates.Note that, in contrast to the previous FD-residual calciummodel, A0 does not represent the release probability and isset equal to 1 in the simulations for simplicity.
2.3 Estimation of the non-parametric modelsof the four synapses
The estimation of the equivalent non-parametric (Volterra)models requires a broadband input, such as a random point-process input like a Poisson random impulse train (RIT), toprobe fully the dynamics and the nonlinearities of theparametric models of the four synapses. The inter-pulseinterval (IPI) of a Poisson RIT is a random variable with anexponential distribution whose mean value equals theinverse of the exponent. The first set of simulations usedRIT inputs with data-record length of 400 input/outputevent pairs and a mean firing rate (MFR) of 2Hz (i.e., theIPI mean value is 500ms) covering an IPI range from 2 to5,000ms, which is consistent with the physiological firingcharacteristics of many types of central neurons (Berger etal. 1988a; Barnes et al. 1990). This length of input/outputdata is adequate to obtain accurate kernel estimates of therespective non-parametric (Volterra) models following themethodology presented below. Note that it is necessary toincrease the input–output data-record length to 2,000 eventpairs in the case of the VC synapse. In a second set ofsimulations, Poisson RIT inputs of the same length asbefore and with various MFRs (from 0.5 to 100Hz) areused to simulate the four synaptic parametric models. Theresulting input–output data are used to estimate the kernelsof the respective non-parametric models in order toexamine their ability to emulate the functional propertiesof the parametric models at these higher firing rates.
2.4 Volterra modeling and kernel estimation using Laguerreexpansions
According to the theory of Volterra modeling, as adapted tothe case of point-process inputs and contemporaneousvarying-amplitude outputs (Volterra 1959; Marmarelis andMarmarelis 1978; Marmarelis 2004), the value of the outputy(t) of a nonlinear time-invariant system with finite memoryM, can be represented at the time ti by the followingPoisson–Volterra (PV) model:
y tið Þ ¼ k1 þX
ti�M<tj<ti
k2 ti � tj� �
þX
ti �M < tj1 < titi �M < tj2 < t
Xk3 ti � tj1 ; ti � tj2� �
þX
ti �M < tj1 < titi �M < tj2 < titi �M < tj3 < ti
XXk4 ti � tj1 ; ti � tj2 ; ti � tj3� �þ . . .
ð13Þ
where k1, k2, k3 and k4 are the first, second, third and fourthorder PV kernels of the system. The summations in Eq. (13)take place over all time of input events within a past epochM (termed the “system memory”) prior to ti. Thesesummations are constructed in a hierarchy of risingcombinations of multiple preceding events. The first-orderPV kernel, k1, represents the amplitude of the EPSCattributed to a single input impulse (i.e., in the absence ofany preceding input impulses within the memory extent M).The second-order PV kernel, k2, represents the change inthe EPSC amplitude caused by second-order interactionsbetween the present input impulse and each of the pastinput impulses within the memory extent M. The third-order PV kernel, k3, represents the change in the EPSCamplitude caused by third-order interactions between thepresent input impulse and any two (not necessarilydifferent) preceding input impulses within M. The fourth-order PV kernel, k4, represents the change caused by fourth-order interactions between the present input impulse andany three (not necessarily different) preceding inputimpulses within M and so on for higher-order kernels.
The form of these summations in Eq. (13) results fromthe reduced Volterra model (Marmarelis 2004), where onedimension of each kernel is collapsed because the input andoutput events are contemporaneous (they fall in the sameevent bin or have a constant delay), and the input single x(t)is the following point-process (Poisson random sequence ofaction potentials):
x tð Þ ¼XNi¼1
d t � tið Þ; ð14Þ
where δ(t − ti) denotes the discrete impulse (Kroneckerdelta) at the time ti, N is the total number of input–outputevents.
J Comput Neurosci
In order to facilitate the estimation of the PV kernelsfrom broadband input–output data, we use the Laguerreexpansion technique (Watanabe and Stark 1975; Marmarelis1993; Song et al. 2007) which yields better estimatesthan the conventional kernel estimation technique of cross-correlation (Lee and Schetzen 1965; Sclabassi et al. 1988)and allows reliable kernel estimation from short and noisydatasets. This improved performance results from theutilization of the orthonormal basis of discrete-time Laguerrefunctions to expand the kernels and reduce the number ofunknown parameters to be estimated.
According to this methodology, the ith-order PV kernelki is expanded in terms of L discrete-time Laguerre basisfunctions bj(τ) as:
ki τ1; . . . ; τ i�1ð Þ
¼XLj1¼1
. . .XLji�1¼1
ci j1; . . . ; ji�1ð Þbj1 τ1ð Þ . . . bji�1 τ i�1ð Þ; ð15Þ
where ci are the kernel expansion coefficients. The PVmodel of Eq. (13) can be rewritten as:
y tið Þ ¼ c1 þXLj¼1
c2 jð Þvj tið Þ þXLj1¼1
XLj2¼1
c3 j1; j2ð Þvj1 tið Þvj2 tið Þ
þXLj1¼1
XLj2¼1
XLj3¼1
c4 j1; j2; j3ð Þvj1 tið Þvj2 tið Þvj3 tið Þ þ . . . ;
ð16Þwhere
vj tð Þ ¼X
t�M<ti<t
bj t � tið Þ; ð17Þ
bj τð Þ ¼�1ð Þτα j�τð Þ=2 1� αð Þ1=2Pτ
k¼0�1ð Þk τ
k
� �jk
� �ατ�k 1� αð Þk 0 � τ < jð Þ
�1ð Þjα τ�jð Þ=2 1� αð Þ1=2Pjk¼0
�1ð Þk τk
� �jk
� �αj�k 1� αð Þk j � τ � Mð Þ
8>><>>: : ð18Þ
The Laguerre parameter α (0 < α < 1) determines therate of exponential asymptotic decline of the Laguerre basisfunctions and in practice is selected through successivetrials so that the prediction mean-square error is minimized.The utilization of basis functions for kernel expansionimproves the kernel estimation because it reduces thenumber of free parameters to be estimated from the data(since the number L of basis functions is typically muchsmaller than the number M of samples within the systemmemory, i.e., the length of each kernel dimension.
The unknown kernel expansion coefficients ci areestimated through least-squares fitting of the input–outputdata in Eq. (15) using singular value decomposition toavoid numerical instabilities. A 1ms bin size is used togenerate the discrete Laguerre basis functions. The obtainedestimates of the expansion coefficients can be used toconstruct the PV kernel estimates according to the Eq. (15).
The prediction accuracy of the obtained PV models wasevaluated with the computed out-of-sample normalized rootmean-square error (NRMSE) of the output prediction:
NRMSE ¼PNi¼1
y tið Þ �by tið Þð Þ2
PNi¼1
y tið Þ2
0BBB@
1CCCA
1=2
ð19Þ
where by is the output predicted by the estimated PV model fora novel RIT input different from the one used for kernelestimation, y is the output of the respective parametric synapsemodel for the novel RIT input. The use of different RIT inputsfor kernel estimation and prediction evaluation is necessary toavoid overfitting the PV kernels/model to the data.
The number of basis function (L) and the Laguerreparameter α are optimized using the criterion of out-of-sample NRMSE. For a given model order, optimal α issearched in the range of (0, 1) for increasing L. The com-bination of α and L that gives the smallest out-of-sampleNRMSE is chosen to construct the PV kernels.
2.5 PV kernels and response descriptors
Each of the aforementioned PV kernels quantifies interac-tions of a specific number of preceding input impulses that isdetermined by its order. However, a certain number ofpreceding input impulses (within the epoch of systemmemory) make contributions to the output through all thePV kernels of the system. For instance, a single precedingimpulse makes a contribution through the second-order PVkernel (k2) but also through all other higher-order PV kernelspresent in a specific system. In other words, k2 is not thesame paired-pulse facilitation/depression function measuredthrough the popular paired-pulse stimulation protocol.
J Comput Neurosci
To examine the effects of a given number of precedinginput impulses within the epoch of system memory, weintroduce the notion/measure of “response descriptors”(RD). The ith-order RD, ri, exclusively represents the ith-order modulatory effect of any i-1 different preceding inputimpulses on the present output. There exist simplemathematical relations between the PV kernels and theith-order RD. For instance, in the case of a third-order PVmodel (Fig. 2):
r1 ¼ k1 ð20Þ
r2 tð Þ ¼ k2 tð Þ þ k3 t; tð Þ ð21Þ
r3 t1; t2ð Þ ¼ 2k3 t1;t2� � ð22Þ
Note that, for a triplet of input impulses, the output isgiven by:
y t1; t2ð Þ ¼ r1 þ r2 t1ð Þ þ r2 t2ð Þ þ r3 t1; t2ð Þ ð23Þ
where y(τ1, τ2) is the output predicted by the third-orderPV model for the two preceding input impulses with timelags from the present output τ1 and τ2, respectively (seeFig. 2E). This mathematical formalism can be extended toany order of PV model. Equations (20)–(23) show that theset of PV kernels and the set of RDs are equivalent interms of output prediction. The RDs will be used forillustration of the results in this paper and for their com-parative evaluation vis-a-vis previously reported results onsynaptic STP, because they more closely relate to theestablished notion of paired-pulse facilitation/depression.An illustrative example of PV kernels and RDs for a third-order PV model obtained from the simulated data ofthe parametric FD model of the SC synapse is shown inFig. 2. Similarly, the RDs of a fourth-order PV kernelmodel can be expressed as (Fig. 7):
r1 ¼ k1 ð24Þ
r2 tð Þ ¼ k2 tð Þ þ k3 t; tð Þ þ k4 t; t; tð Þ ð25Þ
0 200 400 600 800 1000-100
-50
0
50
100
150
200
250
300
0 200 400 600 800 1000-100
-50
0
50
100
150
200
250
300
0
500
1000
0
500
1000-400
-300
-200
-100
0
0
500
1000
0
500
1000-400
-300
-200
-100
0
(%)
(%)
(%)
(%)
k1 = 0.24k2
k3
r1 = 0.24r2
r3
τ 1
τ 2
r2(τ2)
r2(τ1)r3(τ1,τ 2)
r1
y(τ1,τ 2)
τ1 τ2
τ1 τ2
r3(τ 1,τ 2)
r2(τ2)
r2(τ1)
(A) (C)
(D)(B)
(E)
Fig. 2 The Poisson-Volterra(PV) kernels and correspondingresponse descriptors (RDs) of thethird-order PV model of the SCsynapse. (A) The second-orderPV kernel k2 representing thesecond-order interactions with apreceding impulse (exhibitingfacilitation in this case), wherethe abscissa axis is the inter-pulseinterval (IPI) values of the pre-ceding impulse. (B) The third-order PV kernel k3 representingthe third-order interactions withtwo preceding impulses (exhibit-ing depression in this case),where the two abscissa axes arethe IPI values of the two preced-ing impulses. (C) The second-order RD r2 that is equal to thepaired-pulse response function.(D) The third-order RD r3 thatcan be viewed as the triple-pulseresponse function. The separationof paired-pulse facilitation anddepression characteristics is evi-dent in (C) and (D), respectively.(E) The response to a three-pulsestimulus with IPIs τ1 and τ2 is thesummation of r1, r2(τ1), r2(τ2)and r3(τ1, τ2)
J Comput Neurosci
r3 t1; t2ð Þ ¼ 2k3 t1; t2ð Þ þ 3k4 t1; t1; t2ð Þþ 3k4 t2; t2; t1ð Þ ð26Þ
r4 t1; t2; t3ð Þ ¼ 6k4 t1; t2; t3ð Þ ð27Þand it can be proven that,
y τ1; τ2; τ3ð Þ ¼ r1 þ r2 τ1ð Þ þ r2 τ2ð Þ þ r2 τ3ð Þþ r3 τ1; τ2ð Þ þ r3 τ1; τ3ð Þ þ r3 τ2; τ3ð Þþ r4 τ1; τ2; τ3ð Þ ð28Þ
The RDs may also be directly estimated from the input–output data with separate consideration of each output withrespect to the number of impulses within its memory.However, the current method is computationally moreconvenient since the estimation of PV kernels only involvescombinations of v instead of the combinations of impulsesin the system memory (M).
3 Results
The parametric models of the aforementioned four syn-apses (SC, PF, CF and VC) are simulated for the param-eter values shown in Table 1 and Poisson RIT inputs withMFR of 2Hz. Using the resulting synthetic input–outputdata, we estimate the PV kernels of the correspondingnon-parametric models employing the methodology de-scribed in the previous section.
A Volterra kernel model is expressed as a functionalpower series of input with progressively higher-orderkernels capable of describing higher-order nonlinear dy-namics. Theoretically, it can replicate arbitrary systemnonlinearity. However, the number of coefficients to beestimated grows exponentially with increases in modelorder, and thus makes it impractical to keep all the highorder terms. So in practice, the first issue in kernelestimation is to find out how many terms (orders) areneeded for accurate replication of the system nonlinearityand then to truncate the unnecessary high order terms. Thisis typically done using statistical model order selectioncriteria (MOSC) (Marmarelis 2004). However, existingnoise-based statistical MOSCs are not applicable to theproblem in this study since the modeled systems (paramet-ric models) are noise-free. The residual errors are purelydue to the difference between the kernel model and theparametric model, and their statistical significance thus cannot be evaluated using statistical criteria. Therefore, wechoose to use a simple and straightforward model orderselection approach in this study: if the residual error(NRMSE) is below a certain level (5%) for a given model
order, the model is considered a “practically complete” (asopposed to “mathematically complete”) model since themajority of the system nonlinearity is accounted for.
We estimate first, second, third and fourth order PVmodels of the four synapses. The out-of-sample NRMSEsof model predictions are given in Table 2. The first-orderkernel model is a scalar, which is equal to the mean ofoutput amplitudes. No nonlinear dynamic is included andits model predictions are constant. However, the NRMSEsof first-order kernel models show that all four synapseshave strong output nonlinearities given the unitary inputamplitudes. The second, third and fourth-order kernelmodels are capable of modeling up to second, third andfourth-order nonlinear systems, respectively. In PF, CF andVC synapses, most of the NRMSEs are diminished (<5%)when the model order is increased to two; in SC synapse,there is still a large portion of residual error in the secondorder kernel model. The model order should be increased tothree to obtain a NRMSE below 5%. These results showthat for Poisson random inputs with a 2Hz MFR, PF, CFand VC are primarily second-order nonlinear systems,whereas the SC synapse is a third-order nonlinear system,given the order of the system is defined as the order of thesystems nonlinearity.
3.1 Estimated PV kernels for the parametric modelsof the four central synapses
The four synapses included in this paper show differentforms of STP. This could be revealed by a simple, fixed-interval train (FIT) simulation (Fig. 3). For instance, duringa 100Hz train (10 pulses), the EPSC amplitudes of SCsynapse peak at the second impulse and then decline in therest of the impulses; the EPSC amplitudes of PF synapsekeep increasing from the second impulse to the fourthimpulse and then plateau; whereas in both CF and VCsynapses, EPSC amplitudes monotonically decline. ThisFIT simulation only shows the behavior of the synapseswith respect to 10ms intervals. In the kernel modelsestimated with random-interval trains, the four differentforms of STP are fully quantified with respect to arbitraryIPIs and explicitly visualized in the kernels. To make thekernel results comparable, third-order PV kernel models are
Table 2 The NRMSEs of the estimated PV models for the four typesof synapses (using the parameter values in Table 1) for Poisson RITinputs different from the ones used for PV kernel estimation
Model order SC PF CF VC
1st 27.98 40.27 13.1 32.722nd 15.32 3.82 4.82 4.353rd 4.72 0.27 2.36 3.664th 1.89 0.21 1.74 2.23
J Comput Neurosci
used in all four synapses. These models precisely predictthe outputs of the synapses for 2Hz Poisson random inputs(Fig. 4).
3.1.1 SC synapse
Figure 2 and the first column of Fig. 5 shows the estimatedRDs for the third-order non-parametric model of the SCsynapse. The first-order RD (r1) is equal to the initialrelease probability of the synapse (r1= k1 = F1 = 0.24). Thesecond-order RD, r2(τ), represents the modulatory effect ofa single preceding impulse upon the present response as afunction of the IPI value τ. It is seen that this effect ispositive (i.e., facilitatory) for all IPI values up to about400ms, diminishing for larger IPI values. Maximumsecond-order facilitation is observed for the shortest IPIand it is less than half the respective value of the second-order PV kernel, k2(τ), that does not take into account thedepressive effect of third-order dynamics (represented byk3(τ, τ), which is solely depressive for this synapse; seeFig. 2(A) and (C) for comparison). Note that r2(τ) dependson k2 (τ) and k3(τ, τ), according to Eq. (21) and itsmaximum value at the shortest IPI (120%) matches themaximum paired-pulse facilitation ratio in the parametricmodel of the SC synapse (ρ − 1 = 1.2; Fig. 2(C). The third-order RD, r3(τ1, τ2), describes the joint contribution of twopreceding impulses to the present response as a function ofthe two IPI intervals τ1 and τ2. It is shown to be solelydepressive [with double the values of the third-order PVkernel, according to Eq. (22); see Fig. 2(B) and (D) forcomparison] with the most negative value at the shortest IPIinterval being almost three times the value of k1. This result
shows that the SC synapse has a strong depressive third-order nonlinearity. For an input of three impulses, e.g., thefirst three impulses in the 100Hz FIT (Fig. 3, first row), theresponse y(τ1, τ2) to the most recent impulse (termed“present”) is facilitated by the positive second-ordercontributions of each preceding impulse, r2(τ1) and r2(τ2),and depressed by the negative third-order joint contributionof the two preceding impulses r3(τ1, τ2; Fig. 2E). When theabsolute value of r3(τ1, τ2) is larger than r2(τ2), the thirdresponse becomes smaller than the second response andforms a peak in the second response, which is preciselywhat happens in the 100Hz FIT (Fig. 3, first row). Theestimated PV kernels and the corresponding RDs, r2 and r3,fully quantify the effects of paired-pulse facilitation andtriple-pulse depression for arbitrary IPIs in SC synapses.
The small prediction errors imply accurate estimation ofthe PV kernels and RDs. In addition, the estimated RDs arealso directly validated with the RDs numerically calculatedthrough the parametric model. The estimated r2 and r3 wellapproximate the actual r2 (Fig. 2(C), dashed line) and r3(not shown) with arbitrary IPIs.
3.1.2 PF synapse
The second column of Fig. 5 shows the obtained results forthe PF synapse. As indicated for the SC synapse model, r1(=k1) is equal to the initial release probability (F1 = 0.05),which is smaller than that of the SC synapse. The obtainedRDs for the PF synapse have similar qualitative character-istics with the SC synapse, i.e. r2 is positive for all IPIs(solely facilitatory) and r3 is negative for all IPIs (solelydepressive). However, the magnitude of r2 is much largerthan that of the SC synapse because of the much higher
SC
CF
PF
VC
20 ms
Fig. 3 The EPSC responses simulated with the parametric modelsof the four synapses for a fixed-interval input train of ten impulses(IPI of 10 ms or 100 Hz)
SC
CF
PF
VC
2 sFig. 4 The EPSC responses simulated with the parametric models ofthe four synapses (bars) for a Poisson random-interval train (RIT)input with 2 Hz mean firing rate (MFR), and the predictions of thethird-order PV models (circles) accurately replicating the fourdifferent forms of STP
J Comput Neurosci
paired-pulse facilitation ratio in the PF synapse (ρ = 3.1).Also, the magnitude of r3 becomes much smaller comparedto that of the SC synapse model. This confirms the NRMSEresult showing that the majority of the PF synapsenonlinearity is of second order (Table 2). In a triple-pulsetrain, e.g., the first three impulses in the 100Hz FIT (Fig. 3,second row), the third response y(τ1, τ2) is stronglyfacilitated by the second-order contributions of each of thepreceding impulses, r2(τ1) and r2(τ2), and only weaklydepressed by the negative third-order joint contribution ofthe two preceding impulses r3(τ1, τ2). The train thus showsa nearly monotonic-increasing pattern.
3.1.3 CF synapse
The third column of Fig. 5 shows the obtained results forthe CF synapse. The value of r1 is equal to the initialrelease probability 0.35, which is larger than in the previoustwo synapses. The values of r2 are negative for all IPIs, andthey are smaller in absolute value than the previous twosynapses. The values of r3 are also negative and smallrelative to r2, indicating that the second-order nonlinearitiesare dominant in the CF synapse, as in the case of the PFsynapse. Contrary to the SC and PF synapses, the CFsynapse shows negative r2 values that quantify thedepressive effect of each preceding impulse upon thepresent response as a function of the IPI. The absolutevalues of r2 decline rapidly for IPI < 140ms and moreslowly for longer IPIs until they diminish around 2,000ms,indicating that we cannot fit the waveform of r2 with asimple exponential function. We also observe that the formof r3 is biphasic, crossing from negative (depressive) to
positive (facilitatory) values around IPI of 100ms. The peakvalue of r2 (−35% at the minimum IPI) is equal to the initialrelease probability of this synapse.
3.1.4 VC synapse
Similar to the CF synapse, the VC synapse has a dominantsecond-order suppressive kernel/RD (Fig. 5, fourth col-umn). However, unlike the CF synapse, the small values ofr3 are positive for the VC synapse. Note that r1 (not plotted)is equal to A0 (=1), which is defined as the baselineresponse amplitude in the FD1D2 model. We observe thatthe waveform of r2 is comprises of an early fast-recoveringcomponent that is strongly negative and a subsequent slow-recovering component that has weaker negative values.This result shows that the two depressive components (D1
and D2 in Eq. 8) in the parametric model of the VC synapseare captured by the second-order kernel of the non-parametric PV model. The facilitation determined by F inthe parametric model of the VC synapse is masked by D1
and D2 in r2, since the overall effect of D1, D2 and F issuppressive. The small values of r3 indicate that thenonlinearity of the VC synapse is primarily of second order.
3.2 Interpretation and comparison of the non-parametricmodels of the four synapses
The first three synapses (SC, PF and CF) share the sameparametric model structure but have different parametervalues. The corresponding non-parametric PV kernelmodels provide an elucidating way to appreciate how thekey model parameters affect the input–output transforma-
0 1000 2000
0
100
200
300
01000
2000
01000
2000
-300-200-100
0
0 1000 2000
0
100
200
300
01000
2000
01000
2000
-300-200-100
0
0 1000 2000-40
-20
0
20
01000
2000
01000
2000
-40
-20
0
20
0 1 2-60
-40
-20
0
20
01
2
01
2-60-40-20
020
Time (msec) Time (msec) Time (msec) Time (10 sec)
r1
r2
(%)
r3
(%)
SC PF CF VC
0.24 0.05 0.35 -
Fig. 5 The estimated RDs of the four types of synapses. Each column shows the RDs of one type of synapse. Note that in the SC, PF and CFsynapses, r1 is equal to the initial release probability, but in the VC synapse, r1 is equal to unity by definition and thus not shown
J Comput Neurosci
tional properties of each synapse (Fig. 5). For instance, thenonlinearities of the PF synapse are primarily determinedby the residual calcium-dependent facilitation mechanism.The depression mechanism is relatively weak due to thefact that PF synapse has a very small initial releaseprobability (0.05) which limits the depression caused byvesicle depletion. These are reflected by the prominentpositive r2 and relatively small negative r3. By contrast, theSC synapse has a much higher initial release probability(0.24) which causes a higher level of depletion, leading tolarger negative values of r3, i.e., in a triple-pulse train, thejoint contribution of pairs of preceding impulses (exclusiveof their second-order contributions) to the present outputbecomes strongly negative because of the small number ofavailable vesicles after two consecutive releases. Nonethe-less, the modulatory effect of each single preceding impulseis still facilitatory (as indicated by the positive values of r2).
The CF synapse lacks the facilitation mechanism(Dittman et al. 2000). In its FD model, the F factorremains constant through time regardless of the inputpattern (Table 1). Its nonlinearity is exclusively determinedby the kinetics of recovery from vesicle depletion which issignificant due to the high initial release probability (0.35),thus its r2 values are negative. Negative also are the r3values for short IPIs, but become slightly positive forIPIs longer than about 100ms.
The VC synapse has a different parametric modelstructure involving two depression factors in addition tothe facilitation factor. The respective non-parametric PVmodel provides a general representation of its input–outputtransformational properties and makes it possible to assessthe functional similarities and differences with the differentSTP characteristics of the other three types of synapses—even though the latter have different forms of parametricmodels. The main distinguishing characteristic of the para-metric VC synapse model is its double-component depres-sion (D1 and D2) which results in a double-exponentialshape for r2 (with time courses approximately matched bythe time-constants τD1 and τD2 given in Table 1). The fastfacilitation factor (F) of the parametric model of the VCsynapse is reflected on the positive values of r3. Note thatthe FD model of the CF synapse also yields a double-exponential shape for r2 (see third column of Fig. 3) whichis due to the residual calcium-dependent recovery ofdepletion, resulting in different recovery rates for differentIPIs—i.e. when the IPI is short, the residual calcium causedby a preceding impulse has a high concentration and resultsin a fast recovery rate; while for long IPIs, the residualcalcium decays to lower levels that result in a slow recoveryrate. This mechanism is modeled differently in the twoparametric models but its effects on the STP are representedsimilarly in the respective non-parametric models, assistingthe comparison of their functional characteristics.
In summary, the results above show that each form ofsynaptic STP has its own characteristic pattern that isreflected quantitatively on the shapes, polarities and timedurations of its PV kernels and corresponding RDs. It isevident that the PV kernels and the corresponding RDscontain quantitative and biologically interpretable informa-tion about the dynamics and nonlinearities underlying theSTP processes in these four types of synapses.
3.3 Higher-order PV models for higher input firing rates
It is shown above that, for Poisson RIT inputs with MFR of2Hz, the third-order PV models are able to highlyaccurately predict the outputs of all four types of synapses.For such inputs, the IPI values are random independentsamples drawn from an exponential distribution with meanvalue of 500ms. In this study, we use an absolute refractoryperiod of 2ms and the IPI values are generated in the rangeof 2–5,000ms. The importance of using Poisson RIT inputis that it includes many of the temporal patterns typical ofCNS synapses in physiological conditions. The MFR of2Hz is selected on the basis of the firing characteristics ofmany central neurons, e.g., hippocampal CA1 and CA3neurons of free-moving rats (Berger et al. 1983; Barnes etal. 1990; Deadwyler et al. 1996). However, different firingcharacteristics with higher or lower average rates and/ordifferent distributions also exist. The effect of the differentfiring rates on the obtained PV models is discussed furtherin this section.
In order to examine the effect of higher firing rates onthe efficacy of the non-parametric modeling approach, wesimulate the parametric models with inputs of higher MFRsand estimated the respective PV kernels to examine thepredictive capabilities of the resulting non-parametricmodels for these input characteristics. The PV kernels ofthe parametric models of the SC, PF and CF synapses areestimated with data obtained from Poisson RIT inputs withvarious MFRs in the range of 0.5–100Hz. Because of theirrandom character, these inputs contain practically allpossible input sequences that can naturally occur underphysiological conditions. Figure 6 shows the predictionNRMSE of the estimated PV models of various orders(from first to fourth order) for the various input MFRs. TheNRMSE of the first-order PV model quantifies the averageoutput variations around the mean response of the synapsesfor each input MFR and can be used as a baseline to judgethe performance of the higher-order PV models. In the caseof the PF and CF synapses, it is seen that the second-orderPV models can account for a significant portion of theoutput variations and the third-order PV models can predictthe outputs very well over the entire range of input MFRs(up to 100Hz) with the fourth-order PV models offeringinsignificant improvement. In the SC synapse, the second-
J Comput Neurosci
order PV model gave poor predictions for MFRs higherthan 1Hz, as already indicated in the results of the 2HzMFR presented above, but the NRMSE of the third-orderPV model was reduced to levels below 5% for MFRs up to2Hz (as shown above) and remained below 10% for MFRsless than 30Hz. The NRMSE of the third-order predictionincreased significantly for MFRs larger than 30Hz, reachingalmost 20% for the maximum MFR of 100Hz. The use of afourth-order PV model reduced the NRMSE further,keeping it below 5% for MFRs up to 6Hz and below 10%for MFRs less than 50Hz. A steady increase of the fourth-order prediction NRMSE with increasing input MFR isobserved that indicates the need for even higher-order PVkernels to capture better the system nonlinearities andimprove the model prediction for higher input MFRs.
Figure 7 shows the PV kernels and RDs of the fourth-order PV model of the SC synapse estimated with a PoissonRIT input of 2Hz MFR. The fourth-order term (k4/r4) in thefourth-order PV kernel model contributes to the predictionof the outputs with more than two preceding inputimpulses. For example, for a quad-pulse train (Fig. 7G),the fourth output is predicted by the fourth-order PV kernelmodel as the summation of the first-order term contributed
by the present impulse (r1), second-order terms from eachpreceding impulse [r2(τ1), r2(τ2) and r2(τ3)], third-orderterms contributed by each pair of preceding impulses [r3(τ1,τ2), r3(τ1, τ3) and r3(τ2, τ3)] and the fourth-order termjointly determined by all three preceding impulses [r4(τ1,τ2, τ3), Fig. 7G]. k4 and r4 are positive 3D matrices (M × M× M) with peak value at minimum intervals. They allow thefourth-order model to more accurately capture the systemsnonlinearities associated with more-than-two precedingimpulses in the system memory window.
In the fourth-order PV model, the estimated k1 was thesame as that of the third-order PV model, while k2 and k3were different from those of the third-order PV model(compare Figs. 2 and 7). This is due to the fact that both PVmodels are mathematically incomplete as shown in theirresidual errors, although they are practically completeaccording to the chosen NRMSE threshold (5%). Inmathematically incomplete PV models, the extra high orderterms, e.g., k4 in the fourth-order PV model of the SCsynapse, are nonzero. These high order terms project to thelower-order PV kernels (k2 and k3) of the lower-order PVmodels, e.g., the third-order PV model of the SC synapse inthis case, and make these lower-order PV kernels (k2 andk3) different in the two models. Direct comparison of thePV kernels is generally non-informative in the mathemat-ically incomplete PV models of different model orders. Bycontrast, the PV kernels of the mathematically completemodels do not change as the model order changes sincethese PV models have the same non-zero terms and theadditional higher-order PV kernel in the higher-order PVmodel will not interfere with the lower-order PV kernelssince they are equal to zero. However, it is remarkable thatthe RDs of the two practically complete PV models arenearly identical (Figs. 2 and 7). Such favorable property ofthe RD representation can be explained as the following: asshown in the Method section, RDs, which are derived fromthe PV kernels, have a simple interpretation, i.e., the ith-order RD, ri, exclusively represents the ith-order modula-tory effect of any i-1 different preceding input impulses onthe present output. ith-order RD thus does not contribute tothe prediction of the responses with i-1 or fewer impulses.When the MFR of the Poisson random input is low, theprobability of having many input impulses in the modeledsystem’s memory window (M) is small. The majorities ofthe input patterns (isolated by intervals larger than M) aresingle impulses, pairs and triplets. Practically completemodels, e.g., third and fourth-order PV models of SCsynapse, achieve low NRMSE mainly by accuratelypredicting those commonly encountered patterns. The first,second and third-order RDs, which correspond to theprediction of single impulses, pairs and triplets respectively,are all very close to their true values (as indicated by thesmall NRMSE and validated by the direct calculation) and
0 5 100
5
10
15
20
25
30
f (Hz)
NR
MSE
(%
)
0 5 100
5
10
15
20
25
30
35
40
45
f (Hz)
0 5 100
2
4
6
8
10
12
14
16
18
20
f (Hz)
SC PF CF
2nd
3rd
4th
1st
2nd
3rd 4th
1st
2nd
3rd
4th
1st
1 10 1000
10
20
30
40
50
60
1 10 1000
5
10
15
20
25
30
35
40
45
50
1 10 1000
5
10
15
20
25
30
35
40
NR
MSE
(%
)
2nd
3rd
4th
1st
2nd
3rd4th
1st
2nd 3rd
4th
1st
Fig. 6 Predictive accuracies of the non-parametric PV models of threetypes of synapses (SC, CF, PF) for various MFRs of the RIT input.Upper panel: input MFRs in the range of 0.5–10 Hz. Lower panel:input MFR in the range of 0.5–100 Hz (plotted in log scale)
J Comput Neurosci
therefore remain stable across different model orders. In thePV kernel representation, by contrast, ith-order PV kernelalso contributes to the predictions of the responses with i-1or fewer impulses. The estimates of the lower-order PVkernels, e.g., k2 and k3, are thus altered by the inclusion ofhigher-order PV kernels and consequently show differentvalues across different model orders.
Figure 8 shows the changes in the RDs of first, secondand third order that are based on the PV kernel estimates of
the second, third and fourth-order PV models of the SCsynapse, obtained for various values of input MFR in therange from 0.5 to 10Hz. The obtained r1 values show asignificant dependence on input MFR for the second-ordermodel, but no significant dependence for the fourth-ordermodel. For the third-order model, which was selected forthe main stimulation protocol with MFR of 2Hz, thedependence of the r1 values on the input MFR is significantonly for MFR > 4Hz. The obtained r2 values exhibited a
0 200 400 600 800 1000-100
0
100
200
300
400
0 200 400 600 800 1000-100
-50
0
50
100
150
200
250
300
0
500
1000
0
500
1000-500
-400
-300
-200
-100
0
0
500
1000
0
500
1000-500
-400
-300
-200
-100
0
k1 = 0.24
Time (ms) Time (ms)
02000
02000
0
50
100
10 200 4000
20000
2000
0
300
600
10 200 400
r1
= 0.24
k3
k2
r2
r3
k4
r4
τ1
τ2
r2(τ2)
r2(τ1)
r3(τ1,τ2)
r1
y(τ1,τ2,τ3)
r2(τ3)
τ3
r3(τ1,τ3)
r3(τ2,τ3)r4(τ1,τ2,τ3)
τ1 τ2
r2(τ2)
r2(τ1)
τ3
r2(τ3)
τ2τ3τ1
τ 3
r3(τ1,τ3)
r3(τ1,τ2)
r3(τ2,τ3)
(A) (D)
(E)(B)
(C) (F)
(G)
Fig. 7 The PV kernels andcorresponding RDs of thefourth-order PV model of theSC synapse. (A) The second-order PV kernel k2. (B) Thethird-order PV kernel k3. (C)Slices of the fourth-order PVkernel k4 at three selected lags(from 10 to 400 ms). (D) Thesecond-order RD r2. E Thethird-order RD r3. (F) Slices ofthe fourth-order RD r4 at threeselected lags (from 10 to 400ms). (G) RD contributions tothe present output
J Comput Neurosci
strong dependence on input MFR for the second-order PVmodel, changing from positive to negative values around3Hz. Significant dependence was also observed for thethird-order PV model when MFR was higher than 4Hz, butno significant dependence was observed in the case of thefourth-order PV model for MFR up to 8Hz. The obtained r3values exhibited a strong dependence on input MFR for thethird-order PV model (especially for MFR > 3Hz), but lessdependence for the fourth-order PV model. These resultsshow that the kernel estimates and the corresponding RDsof higher-order PV models not only provide better outputpredictions as indicated in the NRMSE results, but also aremore robust to the changes of input MFR. When the inputMFR is further increased (e.g., higher than 15Hz),significant changes occur even for the fourth-order PV
model (not shown), indicating that higher-order models areneeded to fully represent the system dynamic nonlinearitiesin this higher firing rates.
In addition to the input characteristics (e.g., MFR), theother determinant of the required model order, obviously, isthe system nonlinearity to be modeled. For example, for asecond-order nonlinear system, no matter how the MFR isenhanced, second-order PV model will always be complete.The number of impulses in the system memory windowonly provides the upper-bound of the required model order.This is shown in the PF synapse models (Fig. 9). Thesecond order kernel model of PF synapse was constant inthe range of 0.5–5Hz showing that in this range, PFsynapse could be sufficiently characterized as second-ordernonlinear system; the third and fourth order kernel models
0
5
10
0
500
1000-100
0
100
200
0
5
10
0
500
1000
-100
0
100
200
0
5
10
0
500
1000
-400
-200
0
200
0
5
10
0
500
1000-100
0
100
200
0
5
10
0
500
1000
-400
-200
0
200
0 2 4 6 8 100
0.25
0.5
f (Hz)
2nd
(%)
(%)
(%)
4th
(%)
3rd
(%)
r2(τ
τ
τ τ τ
τ τ
)
r2( )
r2( ) r3( , )
r3( , )
r1
2nd3rd
4th
τ (ms) f (Hz)f (Hz) τ (ms)
(A) (B)
(D)(C)
(E) (F)
Fig. 8 The dependence of theRDs of the SC synapse on theinput MFR and PV model order.(A), (C) and (E) The second-order RD r2 for the second, thirdand fourth order PV models as afunction the input MFR f. (B)The first-order RD r1 for thesecond, third and fourth orderPV models as a function theinput MFR f. (D) and (F) Thediagonal slice of the third-orderRD r3 for the third and fourthorder PV models as a functionthe input MFR f. Note that r1, r2and r3 are less variant withchanging input MFR for thefourth order PV model
J Comput Neurosci
were constant in the whole range of 0.5–10Hz indicatingthat in this broader range, third-order model was sufficient.This result shows that the PF synapse is intrinsically lessnonlinear than the SC synapse, in the sense that for a giveninput range, it can be fully characterized by lower-orderkernel models.
In summary, the required model order to adequatelycharacterize a system is jointly determined by both theinput characteristics and the intrinsic system nonlinearity.Higher-order kernel models are more complete androbust to the change of input characteristics (e.g.,MFR) in general. RDs of practically complete modelsare insensitive to the model order and thus are moreappropriate for representations and interpretations of thesystem nonlinearity.
4 Discussion
In this paper, we applied a general, non-parametricmodeling approach to the study of STP in CNS synapses.In this approach, the input–output transformational proper-ties of various synapses are probed with broadband stimuli(Poisson RITs). The synaptic nonlinearities, associated withSTP, are represented by non-parametric models utilizing PVkernels estimated from input–output data, which character-ized fully the transformation of the presynaptic sequence ofimpulses into the postsynaptic EPSCs. Using a recentlydeveloped technique, the PV kernels can be accurately andreliably estimated from short input–output datasets. In thefirst half of this paper, we estimated the PV kernels usingthe input–output data simulated with established parametric
0
5
10
0
500
1000-100
0
100
200
0
5
10
0
500
1000-100
0
100
200
0
5
10
0
500
1000-150
-100
-50
0
50
0
5
10
0
500
1000-100
0
100
200
0
5
10
0
500
1000-150
-100
-50
0
50
0 2 4 6 8 100
0.05
0.1
0.15
0.2
2nd
3rd4th
f (Hz)
2nd
(%)
(%)
(%)
4th
(%)
3rd
(%)
r2(τ)
r2(τ)
r2(τ) r3(τ,τ)
r3(τ,τ)
r1
f (Hz)τ (ms) f (Hz)τ (ms)
(A) (B)
(D)(C)
(E) (F)
Fig. 9 The dependence of theRDs of the PF synapse on theinput MFR and PV model order.(A), (C) and (E) The values ofr2 from the second, third andfourth order PV models fordifferent Poisson input MFRs.(B) The values of r1 from thesecond, third and fourth orderPV models for different Poissoninput MFRs. (D) and (F) Diag-onal slice of r3 from the thirdand fourth order PV models fordifferent Poisson input MFRs.No significant dependence isobserved for the third and fourthorder model, indicating com-pleteness of the PV model forinput MFR up to 10 Hz
J Comput Neurosci
models of several CNS synapses for Poisson randomimpulse trains (RITs) with MFR of 2Hz. It was found thatthird-order PV models (involving first, second and third-order PV kernels) could predict accurately the outputs ofthese parametric synapse models to novel input patterns (solong as the MFR did not exceed significantly 2Hz). Thenonlinear dynamics defined by the specific mathematicalform and associated parameter values of the parametricmodels were fully and reliably captured by the respectivenon-parametric models. The PV kernels and the associatedRDs quantitatively described the transformational input–output properties of the synapses and facilitated interpreta-tion of their functional properties. In the second half of thispaper, we further investigated the predictive accuracy ofthese non-parametric models for RIT inputs with higherMFRs and showed that this approach can be successfullyapplied to this case, but only if we increase sufficiently theorder of the PV models to capture the higher-orderdynamics and nonlinearities associated with higher inputMFRs.
4.1 On Poisson–Volterra kernel model
Non-parametric method was first used to model synaptictransmission by Krausz and Friesen (Krausz and Friesen1977). In their study, they estimated Wiener kernels of thelobster cardiac ganglion synapse using cross-correlationtechnique. The approach used in this study has severalimportant differences and improvements compared to theirapproach. First, instead of Wiener kernels, we estimatedVolterra kernels, which renders the kernel estimatesinvariant to input characteristics when a mathematicallycomplete model is used. The corresponding RDs of theestimated PV kernels can be directly related to conventionalelectrophysiological measurements, e.g., r1 is baselinepostsynaptic response amplitude and r2 is the paired-pulsefacilitation/depression function. The utilization of RDsrelaxes the requirement of mathematical completeness ofthe model in representation, since they remain invariant inpractically complete models. By contrast, Wiener kernelshighly depend on the input characteristic, e.g., MFRs of thePoisson input train. For example, first-order Wiener kernelis the mean of the response amplitudes during the Poissontrain, which obviously depends on MFR. Second, in thisstudy, the PV kernels of the non-parametric models areestimated using the Laguerre expansion technique thatallows accurate and efficient kernel estimation with shortdatasets (Marmarelis 1993). Most of PV kernels in thispaper are estimated with only 400 input–output pairs,which is a significant advantage when we consider that theconventional cross-correlation technique usually requiresthousands of input–output pairs for comparable accuracy(Krausz and Friesen 1977; Berger et al. 1988a, 1988b;
Sclabassi et al. 1988). In addition, the Laguerre expansiontechnique allows the estimation of PV models of higherorders (i.e., third and fourth order) which is extremelydifficult using the conventional binning method, since thelatter involves much larger number of kernel coefficients tobe estimated. Compared to previously obtained second-order PV kernel models (Gholmieh et al. 2001), higher-order models provide more accurate output predictions,because they capture the higher-order nonlinearities associ-ated with more complex input–output properties and/orhigher input MFRs. This is demonstrated clearly in the SCsynapse result, where there is a dramatic drop of predictionerror when we increase the model order from second tothird. For this specific system, only PV models of orderhigher than second are practically complete, in the sensethat they capture the major effect of system nonlinearity. Bycontrast, the second-order PV model, being an incompletemodel, fails to give accurate prediction. Furthermore,because of the prominent third-order nonlinearity of theSC synapse, the kernel estimates of the second-order PVmodel are biased by the presence of the higher-ordernonlinearities. Thus, the second-order model yielded valuesof r2 that are markedly different from the actual paired-pulse facilitation/depression function (Fig. 8).
It is important to point out again that the practicalcompleteness of the PV model is also dependent on theMFR of the random inputs. When input MFR increases, thepossibility of higher-order nonlinear interactions increases.In SC synapses, when the MFR of the Poisson RIT is 2Hz,the third-order PV model is practically complete, but whenthe MFR is increased to 10Hz, the fourth-order PV model ispractically complete. This raises one important aspect of thenon-parametric modeling approach: namely their guaran-teed ability to predict the output is only for input patternsthat are within the range of the inputs with which they areestimated. Although this is often true for parametricmodeling as well, it is more critical for non-parametricmodeling since non-parametric model is purely data-drivenand in general lacks the extrapolation power. Therefore inexperimental design, it is crucial to use broadband inputsthat cover the patterns of physiological interest.
Another form of quantitative model of STP in neuro-muscular junctions of a crustacean was introduced by Senet al. (1997). This model consists of a cascade of threecomponents: a linear filter K2, followed by a staticnonlinearity F, followed by another linear filter K1. FilterK1 describes the shape of the postsynaptic response, e.g.,excitatory junctional current/potential (EJC/EJP), and isequivalent to the “typical EPSC waveform” that we usedhere to deconvolve the EPSC train in Part II of this study(see companion paper for details). The other filter K2 is thefirst component of the cascade and serves to integrate theinput activity over a longer past epoch. Sen et al. showed
J Comput Neurosci
that it has an exponential form (with a negative exponent)extending over several seconds. The output of K2 (i.e. theintegrated input past activity) is transformed by the staticnonlinearity F, which was shown to exhibit a supra-linearform (exponential-like, with a positive exponent). Thismodel form represents the linear-nonlinear (LN) cascademodel studied extensively in the 70s in connection withvarious sensory systems (for a review, see Marmarelis andMarmarelis 1978). The PV kernels estimated in this studycorrespond to the cascade of K2 and F in the sense that theydescribe the nonlinear dynamic transformation of a point-process input (presynaptic spike train) to varying-amplitudeoutputs (EPSC or EJC/EJP amplitudes) which representsthe combined effect of K2 and F. It was shown that thismodel can be successfully applied to several crustaceanneuromuscular junctions. However, it should be noted thatthis LN model does not constitute a complete representationof arbitrary nonlinear dynamics like the Volterra modelingapproach. It relies on the strong assumption that thenonlinear dynamics of a particular system can be separatedinto a cascade of a single linear filter followed by a singlestatic nonlinearity. The general Volterra representation isequivalent to the Wiener–Bose model (composed ofmultiple parallel linear filters feeding into a multi-inputstatic nonlinearity) or it can be decomposed into multipleparallel LN cascades termed the “principal dynamic modes”(Marmarelis 2004). We note that Sen et al. (1997) adoptedthe LN approach for one important and valid aim, namely,developing a methodology that is “as simple as possibleboth to apply and to describe” while retaining a certaindegree of generality and interpretability. This aim isdifferent from the aim of the non-parametric modeling inthis study, since we seek a general and complete represen-tation of STP that allows us to reveal the subtle features ofthe nonlinear dynamical characteristics and use them toguide the modification of the parametric model.
4.2 Parametric vs. non-parametric modeling
Before using experimental input–output (stimulus–re-sponse) data, in this paper, kernel models are estimatedusing input–output data simulated with four parametricsynapse models. There are several reasons for this: first, theparametric synapse models included in this paper representseveral distinct forms of STP determined by the principlephysiological mechanisms/processes. They include a vari-ety of input–output properties and make it possible to testthe generality of the non-parametric approach within abroad context. Second, the parametric models are thor-oughly known, stationary, nonlinear systems that allowdirect and unambiguous validation of the non-parametricmodels. Third, the FD model capture the basic mechanismsof STP, i.e., residual calcium-dependent facilitation and
depletion-based depression. As shown in Section 3, manyinsights are gained by studying the relationships between itskey parameters, which represent fundamental biologicalprocesses, and the kernels, which quantitatively describethe functional input–output properties of the system.
Parametric and non-parametric approaches are developedfor distinct aims and have their relative strengths andweaknesses in modeling a biological system (Berger et al.1994), and thus can be used in a synergistic manner. In thispaper, we show that the non-parametric PV model providesa quantitative description of the synaptic input–outputtransformation without requiring prior assumptions aboutthe model structure. Furthermore, non-parametric modelextracts the system nonlinearities and expresses them in auniform and often simpler form. For example, the paramet-ric FD model involves nonlinear differential equations;Analytical solution of output for a given input pattern is notobvious. On the other hand, using the non-parametric PVmodel, prediction of outputs for a given input pattern onlyrequires simple arithmetical operations such as multiplica-tion and summation. The uniformity and simplicity ofmodel mathematical form is critical for large-scale simu-lations (Traub and Miles 1991) and hardware implementa-tions (Tsai et al. 1998; Berger et al. 2001).
Most importantly, non-parametric models estimateddirectly from the input–output experimental data can beused to evaluate parametric models under broadband inputconditions in terms of their input–output transformationalproperty. The differences between the non-parametricmodel estimated from experimental data and the equivalentnon-parametric model of the parametric model may exposeunknown physiological mechanisms/processes and suggestspecific modification to the parametric model. This isshown in the companion paper (part II).
Acknowledgments This research was supported by the NIH/NIBIB(BMSR), NSF ERC (BMES), DARPA (HAND), ONR, and NSF(BITS). We thank the two anonymous reviewers for their insightfulcomments on this manuscript.
References
Abbott, L. F., Varela, J. A., Sen, K., & Nelson, S. B. (1997). Synapticdepression and cortical gain control. Science, 275, 220–224.
Barnes, C. A., Mcnaughton, B. L., Mizumori, S. J. Y., Leonard, B. W.,& Lin, L.-H. (1990). Comparison of spatial and temporalcharacteristics of neuronal activity in sequential stages ofhippocampal processing. In J. Zimmer & O. P. Ottersen (Eds.),Understanding the brain through the hippocampal region as amodel for studying structure and function (p. 287). Amsterdam:Elsevier Science.
Berger, T. W., Baudry, M., Brinton, R. D., Liaw, J. S., Marmarelis, V.Z., Park, A. Y., et al. (2001). Brain-implantable biomimeticelectronics as the next era in neural prosthetics. Proceedings ofthe IEEE, 89, 993–1012.
J Comput Neurosci
Berger, T. W., Chauvet, G., & Sclabassi, R. J. (1994). A biologicalbased model of functional properties of the hippocampus. NeuralNetworks, 7, 1031–1064.
Berger, T. W., Eriksson, J. L., Ciarolla, D. A., & Sclabassi, R. J.(1988a). Nonlinear systems analysis of the hippocampal perfo-rant path-dentate projection. II. Effects of random impulse trainstimulation. Journal of Neurophysiology, 60, 1076–1094.
Berger, T. W., Eriksson, J. L., Ciarolla, D. A., & Sclabassi, R. J.(1988b). Nonlinear systems analysis of the hippocampal perfo-rant path-dentate projection. III. Comparison of random train andpaired impulse stimulation. Journal of Neurophysiology, 60,1095–1109.
Berger, T. W., Rinaldi, P. C., Weisz, D. J., & Thompson, R. F. (1983).Single-unit analysis of different hippocampal cell types duringclassical conditioning of rabbit nictitating membrane response.Journal of Neurophysiology, 50, 1197–1219.
Betz, W. J. (1970). Depression of transmitter release at theneuromuscular junction of the frog. Journal of Physiology, 206,629–644.
Bishop, C. M. (1995). Neural networks for pattern recognition.Oxford: Oxford University Press.
Creager, R., Dunwiddie, T., & Lynch, G. (1980). Paired-pulse andfrequency facilitation in the CA1 region of the in vitro rathippocampus. Journal of Physiology, 299, 409–424.
Deadwyler, S. A., Bunn, T., & Hampson, R. E. (1996). Hippocampalensemble activity during spatial delayed-nonmatch-to-sampleperformance in rats. Journal of Neuroscience, 16, 354–372.
Debanne, D., Guerineau, N. C., Gahwiler, B. H., & Thompson, S. M.(1996). Paired-pulse facilitation and depression at unitarysynapses in rat hippocampus: Quantal fluctuation affects subse-quent release. Journal of Physiology, 491(Pt 1), 163–176.
Dittman, J. S., Kreitzer, A. C., & Regehr, W. G. (2000). Interplaybetween facilitation, depression, and residual calcium at threepresynaptic terminals. Journal of Neuroscience, 20, 1374–1385.
Dittman, J. S., & Regehr, W. G. (1998). Calcium dependence andrecovery kinetics of presynaptic depression at the climbing fiberto Purkinje cell synapse. Journal of Neuroscience, 18, 6147–6162.
Dobrunz, L. E., Huang, E. P., & Stevens, C. F. (1997). Very short-termplasticity in hippocampal synapses. Proceedings of the NationalAcademy of Sciences of the United States of America, 94, 14843–14847.
Dobrunz, L. E., & Stevens, C. F. (1997). Heterogeneity of releaseprobability, facilitation, and depletion at central synapses.Neuron, 18, 995–1008.
Eccles, J. C., Katz, B., & Kuffler, S. W. (1941). Nature of the“endplate potential” in curarized muscle. Journal of Neurophys-iology, 4, 362–387.
Feng, T. (1941). The changes in the end-plate potential during andafter prolonged stimulation. Chinese Journal of Physiology, 13,79–107.
Friesen, W. O. (1975). Antifacilitation and facilitation in cardiacganglion of spiny lobster Panulirus interruptus. Journal ofComparative Physiology, 101, 207–224.
Fuhrmann, G., Cowan, A., Segev, I., Tsodyks, M., & Stricker, C.(2004). Multiple mechanisms govern the dynamics of depressionat neocortical synapses of young rats. Journal of Physiology, 557,415–438.
Gholmieh, G., Soussou, W., Courellis, S., Marmarelis, V., Berger, T.,& Baudry, M. (2001). A biosensor for detecting changes incognitive processing based on nonlinear systems analysis.Biosensors & Bioelectronics, 16, 491–501.
Gover, T. D., Jiang, X. Y., & Abrams, T. W. (2002). Persistent,exocytosis-independent silencing of release sites underlies homo-synaptic depression at sensory synapses in Aplysia. Journal ofNeuroscience, 22, 1942–1955.
Hanse, E., & Gustafsson, B. (2001). Paired-pulse plasticity at thesingle release site level: an experimental and computationalstudy. Journal of Neuroscience, 21, 8362–8369.
Hosoi, N., Sakaba, T., & Neher, E. (2007). Quantitative analysis ofcalcium-dependent vesicle recruitment and its functional role atthe calyx of Held synapse. Journal of Neuroscience, 27, 14286–14298.
Hsu, S. F., Augustine, G. J., & Jackson, M. B. (1996). Adaptation ofCa(2+)-triggered exocytosis in presynaptic terminals. Neuron, 17,501–512.
Hunter, J. D., & Milton, J. G. (2001). Synaptic heterogeneity andstimulus-induced modulation of depression in central synapses.Journal of Neuroscience, 21, 5781–5793.
Jones, M. V., & Westbrook, G. L. (1996). The impact of receptordesensitization on fast synaptic transmission. Trends in Neuro-sciences, 19, 96–101.
Katz, B., & Miledi, R. (1968). The role of calcium in neuromuscularfacilitation. Journal of Physiology, 195, 481–492.
Kirischuk, S., Clements, J. D., & Grantyn, R. (2002). Presynaptic andpostsynaptic mechanisms underlie paired pulse depression atsingle GABAergic boutons in rat collicular cultures. Journal ofPhysiology, 543, 99–116.
Kraushaar, U., & Jonas, P. (2000). Efficacy and stability of quantalGABA release at a hippocampal interneuron-principal neuronsynapse. Journal of Neuroscience, 20, 5594–5607.
Krausz, H. I., & Friesen, W. O. (1977). The analysis of nonlinearsynaptic transmission. Journal of General Physiology, 70, 243–265.
Lee, Y. W., & Schetzen, M. (1965). Measurement of the Wienerkernels of a nonlinear system by crosscorrelation. InternationalJournal of Control, 2, 237–254.
Liley, A. W., & North, K. A. (1953). An electrical investigation ofeffects of repetitive stimulation on mammalian neuromuscularjunction. Journal of Neurophysiology, 16, 509–527.
Lisman, J. E. (1997). Bursts as a unit of neural information:Making unreliable synapses reliable. Trends in Neurosciences,20, 38–43.
Magleby, K. L., & Zengel, J. E. (1975). A quantitative description oftetanic and post-tetanic potentiation of transmitter release at thefrog neuromuscular junction. Journal of Physiology, 245, 183–208.
Mallart, A., & Martin, A. R. (1967). An analysis of facilitation oftransmitter release at the neuromuscular junction of the frog.Journal of Physiology, 193, 679–694.
Marmarelis, V. Z (1993). Identification of nonlinear biological systemsusing Laguerre expansions of kernels. Annals of BiomedicalEngineering, 21, 573–589.
Marmarelis, V. Z. (2004). Nonlinear dynamic modeling of physiolog-ical systems. Hoboken: Wiley-IEEE Press.
Marmarelis, V. Z., & Marmarelis, P. Z. (1978). Analysis ofphysiological systems: The white-noise approach. New York:Plenum.
Orban, G. A., Hoffmann, K. P., & Duysens, J. (1985). Velocityselectivity in the cat visual system. I. Responses of LGN cells tomoving bar stimuli: A comparison with cortical areas 17 and 18.Journal of Neurophysiology, 54, 1026–1049.
Pedroarena, C. M., & Schwarz, C. (2003). Efficacy and short-termplasticity at GABAergic synapses between Purkinje andcerebellar nuclei neurons. Journal of Neurophysiology, 89,704–715.
Reid, R. C., Soodak, R. E., & Shapley, R. M. (1991). Directionalselectivity and spatiotemporal structure of receptive fields ofsimple cells in cat striate cortex. Journal of Neurophysiology, 66,505–529.
Sclabassi, R. J., Eriksson, J. L., Port, R. L., Robinson, G. B., &Berger, T. W. (1988). Nonlinear systems analysis of the
J Comput Neurosci
hippocampal perforant path-dentate projection. I. Theoretical andinterpretational considerations. Journal of Neurophysiology, 60,1066–1076.
Sen, K., Gibson, J., Fost, J., Abbott, L. F., Nelson, S. B., & Hunter, J.D. (1997). Synaptic heterogeneity and stimulus-induced modu-lation of depression in central synapses. Journal of Neuroscience,17, 7926–7940.
Song, D., Chan, R. H., Marmarelis, V. Z., Hampson, R. E.,Deadwyler, S. A., & Berger, T. W. (2007). Nonlinear dynamicmodeling of spike train transformations for hippocampal-corticalprostheses. IEEE Transactions on Biomedical Engineering, 54,1053–1066.
Traub, R. D., & Miles, R. (1991). Neuronal networks of thehippocampus. New York: Cambridge University Press.
Trussell, L. O., & Fischbach, G. D. (1989). Glutamate receptordesensitization and its role in synaptic transmission. Neuron, 3,209–218.
Trussell, L. O., Zhang, S., & Raman, I. M. (1993). Desensitization ofAMPA receptors upon multiquantal neurotransmitter release.Neuron, 10, 1185–1196.
Tsai, R. H., Sheu, B. J., & Berger, T. W. (1998). A VLSI neuralnetwork processor based on hippocampal model. Analog Inte-grated Circuits and Signal Processing, 15, 201–213.
Tsodyks, M. V., & Markram, H. (1997). The neural code betweenneocortical pyramidal neurons depends on neurotransmitterrelease probability. Proceedings of the National Academy ofSciences of the United States of America, 94, 719–723.
Tsodyks, M., Pawelzik, K., & Markram, H. (1998). Neural networkswith dynamic synapses. Neural Computation, 10, 821–835.
Varela, J. A., Sen, K., Gibson, J., Fost, J., Abbott, L. F., & Nelson, S.B. (1997). A quantitative description of short-term plasticity atexcitatory synapses in layer 2/3 of rat primary visual cortex.Journal of Neuroscience, 17, 7926–7940.
Volterra, V. (1959). Theory of functionals and of integral and integro-differential equations. New York: Dover.
Waldeck, R. F., Pereda, A., & Faber, D. S. (2000). Properties andplasticity of paired-pulse depression at a central synapse. Journalof Neuroscience, 20, 5312–5320.
Wang, L. Y., & Kaczmarek, L. K. (1998). High-frequency firing helpsreplenish the readily releasable pool of synaptic vesicles. Nature,394, 384–388.
Watanabe, A., & Stark, L. (1975). Kernel method for nonlinearanalysis: Identification of a biological control system. Mathe-matical Biosciences, 27, 99–108.
Yamada, W. M., & Zucker, R. S. (1992). Time course of transmitterrelease calculated from simulations of a calcium diffusion model.Biophysical Journal, 61, 671–682.
Zador, A. M., & Dobrunz, L. E. (1997). Dynamic synapses in thecortex. Neuron, 19, 1–4.
Zengel, J. E., & Magleby, K. L. (1982). Augmentation and facilitation oftransmitter release. A quantitative description at the frog neuromus-cular junction. Journal of General Physiology, 80, 583–611.
Zucker, R. S., & Regehr, W. G. (2002). Short-term synaptic plasticity.Annual Review of Physiology, 64, 355–405.
J Comput Neurosci