an equation-free probabilistic steady-state approximation ... · of chemical or biochemical...

THE JOURNAL OF CHEMICAL PHYSICS 123, 214106 �2005�

An equation-free probabilistic steady-state approximation: Dynamicapplication to the stochastic simulation of biochemical reaction networks

Howard Salis and Yiannis N. Kaznessisa�

Department of Chemical Engineering and Materials Science, and Digital Technology Center, University ofMinnesota, Minneapolis, Minnesota 55455

�Received 9 June 2005; accepted 4 October 2005; published online 6 December 2005�

Stochastic chemical kinetics more accurately describes the dynamics of “small” chemical systems,such as biological cells. Many real systems contain dynamical stiffness, which causes the exactstochastic simulation algorithm or other kinetic Monte Carlo methods to spend the majority of theirtime executing frequently occurring reaction events. Previous methods have successfully applied atype of probabilistic steady-state approximation by deriving an evolution equation, such as thechemical master equation, for the relaxed fast dynamics and using the solution of that equation todetermine the slow dynamics. However, because the solution of the chemical master equation islimited to small, carefully selected, or linear reaction networks, an alternate equation-free methodwould be highly useful. We present a probabilistic steady-state approximation that separates the timescales of an arbitrary reaction network, detects the convergence of a marginal distribution to aquasi-steady-state, directly samples the underlying distribution, and uses those samples to accuratelypredict the state of the system, including the effects of the slow dynamics, at future times. Thenumerical method produces an accurate solution of both the fast and slow reaction dynamics while,for stiff systems, reducing the computational time by orders of magnitude. The developed theorymakes no approximations on the shape or form of the underlying steady-state distribution and onlyassumes that it is ergodic. We demonstrate the accuracy and efficiency of the method using multipleinteresting examples, including a highly nonlinear protein-protein interaction network. Thedeveloped theory may be applied to any type of kinetic Monte Carlo simulation to more efficientlysimulate dynamically stiff systems, including existing exact, approximate, or hybrid stochasticsimulation techniques. © 2005 American Institute of Physics. �DOI: 10.1063/1.2131050�

I. INTRODUCTION

The stochastic simulation of systems of chemical or bio-chemical reactions has become an important tool in quanti-tatively describing the behavior of “small” chemical or bio-chemical systems1–4 where the number of reacting moleculesmay be few and the frequency of reaction occurrences maybe rare. These simulations produce realizations of a jumpMarkov process with discrete states, a mathematical repre-sentation that provides an accurate mesoscopic description ofmany physical and chemical processes, including reactionand diffusion.5 The original stochastic simulation algorithm6

and improved variants7,8 produce an exact solution to thestochastic dynamics of a well-stirred system of coupled bio-chemical or chemical reactions. However, the computationalcosts of these algorithms increase proportionally with thenumber of executed reaction events, causing the simulationof systems which contain frequently occurring, or “fast,” re-actions to be expensive. Systems with both fast and “slow”reactions are commonly found in biological organisms,where the regulation of relatively infrequently expressedgenes is directly affected by fast enzymatic reactions andother protein interactions. Any reduction in the computa-tional costs of these methods will be multiplied manyfold,given their repeated usage in other useful methods such as in

a�
Fax: 612 626 7246. Electronic mail: [email protected]
0021-9606/2005/123�21�/214106/16/$22.50 123, 2141

Downloaded 07 Dec 2005 to 134.84.164.143. Redistribution subject to

sensitivity analysis,9 global optimization,10–13 and the cre-ation of bifurcation diagrams of mesoscopic and microscopicsystems.14,15

Several approximations have been proposed to decreasethe computational cost of stochastic simulations. By approxi-mating fast reactions as either a discrete Poisson process16,17

or a discrete binomial process,18,19 “bundles” of reaction oc-currences are executed with either a Poisson or a binomialdistribution. Fast reactions with many participating mol-ecules may also be validly approximated as a continuousMarkov process,20 where the dynamics are described by achemical Langevin equation. Hybrid methods combiningthese approximations with the original jump Markov processhave been developed,21,22 with the latest work23 both theo-retically and numerically quantifying the global error andintroducing an additional approximation that reduces the costwhen simulating systems with many thousands of reactions.

However, stiffness or a separation in time scales is still afrequent problem for stochastic simulators and has been pre-viously noted.24,25 If there is stiffness in the subset of fast/continuous reactions, then one may approximate them as acontinuous Markov process and derive a system of stiff sto-chastic differential equations �SDEs� that may be numeri-cally integrated using an adaptive,26,27 semi-implicit,28 orbalanced implicit29 method. While work has progressed in

24
creating similar methods for Poisson-driven SDEs, the un-
© 2005 American Institute of Physics06-1

AIP license or copyright, see http://jcp.aip.org/jcp/copyright.jsp

http://dx.doi.org/10.1063/1.2131050

http://dx.doi.org/10.1063/1.2131050

http://dx.doi.org/10.1063/1.2131050

214106-2 H. Salis and Y. N. Kaznessis J. Chem. Phys. 123, 214106 �2005�

derlying numerical theory30 is only beginning to be devel-oped. Consequently, it would be useful to develop a methodthat specifically addresses the problem of stiffness, eithersubstituting or complementing existing methods.

One common solution to stiffness is to remove the fastdynamics by assuming that they quickly relax to a steady-state value. The quasi-steady-state approximation �QSSA� isa well-known method of converting a subset of differentialequations into a system of nonlinear algebraic constraints byassuming that their solution proceeds to its steady state muchfaster than the time scales of the other differential equationsin the system.31 A probabilistic version of this approximationhas been applied to the chemical master equation by elimi-nating selected intermediate species from small enzymaticreaction networks, assuming their distributions are time in-dependent and resulting in some familiar Michaelis-Menten-type kinetics.32 Two recent approaches by Cao et al.33 and byGoutsias34 partition a system of reactions and species intofast and slow ones, solve a chemical master or differentialequation that describes the fast dynamics, and use the solu-tion to compute the probabilistic rates of the slow reactions.All of these methods have shown that it is possible to applya probabilistic steady-state approximation �PSSA� to jumpMarkov processes and substantially reduce the computa-tional cost of simulation. In fact, it is possible to determinethe exact solution of at least the first two moments of thechemical master equation for any linear system.35 However,for nonlinear chemical kinetic systems, there is no general-ized way to compute for the solution of the chemical masterequation. Because many interesting systems are both nonlin-ear and contain fast dynamics, a method that both avoids thesolution of the corresponding chemical master or differentialequation and applies the probabilistic steady-state approxi-mation is desired. Here, we intend to use an equation-freeapproach to generalize these ideas for any system of chemi-cal or biochemical reactions, including nonlinear ones, addi-tionally develop the theory, and propose a numerical methodthat implements these ideas. The equation-free method dy-namically applies the probabilistic steady-state approxima-tion when it is valid, computes the resulting steady-state dis-tribution from only sampled trajectory data, and uses thissteady-state distribution to accurately simulate the next slowreaction, effectively leaping ahead in simulation time. Wemake no assumptions about the form or shape of the steady-state distribution. Instead, we only assume that it is ergodic.

The method is considered equation-free, similar in prin-ciple to other ones,36 because we do not compute the steady-state probability distribution by solving an evolution equa-tion, such as a chemical master or differential equation, nordo we make any approximations on the continuity or differ-entiability of the transitions and derive a Fokker-Plank orordinary differential equation. Instead, we simulate the for-ward stochastic dynamics of the jump Markov process usinga kinetic Monte Carlo method and detect the existence of aquasi-steady-state marginal distribution. We then samplestates directly from the distribution, alleviating the need tomake moment truncations or other approximations.

The method is dynamically applied on top of any kinetic
Monte Carlo algorithm or stochastic simulator that drives the

system dynamics forward in time, such as the original sto-chastic simulation algorithm, and may be used on any systemof chemical or biochemical reactions described as a jumpMarkov process. It is most frequently and effectively appliedwhen there is a subset of fast/discrete reactions, or any tran-sitions, that causes a subset of species to quickly converge toa probabilistic stable steady state. For simplicity, we restrictourselves to nonspatial, well-stirred systems.

This paper first presents a simple illustrative example todemonstrate the dynamic usage of a probabilistic steady-stateapproximation, followed by a generalization of those prin-ciples for any system of coupled chemical or biochemicalreactions. A detailed method is then proposed, four examplesare analyzed, and the computational costs and the accuracy,in terms of their moment and distribution errors, are com-pared to an optimized variant of the stochastic simulationalgorithm. We conclude with a critical discussion and thelimitations of the method.

II. AN ILLUSTRATIVE EXAMPLE

Consider the following nonlinear toy reaction network ina bacterial-sized volume of 1.0E−15 L with an initial condi-tion of #A=45 molecules, #B=#C=#D=25 molecules, and#E=0 molecules:

A + B → C, k1 = 1.3284 �molecules s�−1, �1�

C → A + B, k2 = 80 s−1, �2�

C + D → E, k3 = 3.32E − 4 �molecules s�−1. �3�

While this is just an example, this system has many com-monalities to real biological systems. Species C and E maybe multimer proteins of monomer proteins A, B, and D. Pro-tein heterodimer C may bind to mRNA regulatory bindingsite D, forming a complex E. The number of participatingmolecules in this system is few, requiring us to represent it asa jump Markov process. The reactions �1� and �2� are revers-ible and fast transitions, while reaction �3� is relatively slow.

The stochastic simulation algorithm and any of its vari-ants will spend the majority of its time resolving the indi-vidual occurrences of the fast reactions and will only rarelyexecute the occurrence of the slow one. If the Poisson orbinomial approximation is applied, the inherent time step ofthe simulation is inversely proportional to the sum of thereaction propensities, resulting in a slow simulation whenmany fast reactions exist. If we are interested in the long-time behavior of this simple system, we would need to spenda large amount of computational time to resolve both the fastand slow reactions. In Fig. 1, we have simulated a trajectoryof the system for 1000 s with the stochastic simulation algo-rithm.

The species evolve over time according to two dominanttime scales. The first time scale consists of the effects of thefast reactions, �1� and �2�, and the second is the effect of theoccasional occurrence of reaction �3�. After each occurrenceof reaction �3�, the fast time scale causes the species A, B,and C to quickly reach a probabilistic steady state. The num-
bers of molecules of A, B, and C change over time, but the

214106-3 Equation-free probabilistic steady-state approximation J. Chem. Phys. 123, 214106 �2005�

probability distribution of the number of molecules areroughly constant for a large period of time. Just after eachoccurrence of reaction �3�, the species A, B, and C are nolonger at a probabilistic steady state, but may reach a newsteady state within a short period of time. If the occurrencesof reaction �3� are rare, then there may be a significant periodof time when the species A, B, and C are at a probabilisticsteady state, such as the time interval �t1 , t4� labeled in Fig. 1.In this region, about 300 000 uninteresting reaction eventsare executed by the stochastic simulation algorithm, wastingcomputational time. There are many of these regions in thissimulation and, by eliminating extraneous reaction occur-rences, we may reduce the computational cost significantly.Note that we are loosely using the term probabilistic steadystate when the distribution of some subset of species be-comes insensitive to time, making it more of an approximatesteady state. We will formalize the definition of this approxi-mation in Sec. III.

There are three important regions labeled in Fig. 1: therelaxation period, the sampling period, and the leapfrog pe-riod. Reaction �3� fires at time t1, causing any previous PSSAto be invalid. Between times t1 and t2, the marginal distribu-tion of species A, B, and C is both stable and quickly relax-ing to a probabilistic quasi-steady-state. The marginal distri-bution moves quickly from a time-dependent one to astationary one. At time t2, we start to sample the steady-stateprobability distribution. Once we have collected enoughsamples to accurately determine the time of the next firing ofreaction �3�, which is t4, we leap ahead to that time andignore any occurrences of reactions �1� and �2� betweentimes t3 and t4. By repeating this process, we can skip overmany occurrences of reactions �1� and �2� while still accu-rately resolving the occurrences of reaction �3�. The impor-tant questions are as follows: How do we identify when aprobability distribution is stable and converges to a quasi-steady-state? How many samples from the distribution arerequired to accurately compute the time of the next slow

FIG. 1. A stochastic simulation trajectory of the dynamics of reactions �1�–�3� for 1000 s. The light lines are the species labeled A, B, and C, while thedark lines are the species labeled E and D. The vertical dotted lines definethe times t1– t4.

reaction? How do we choose the state of the system at any


time in the leapfrog period and at the time just prior to theoccurrence of the next slow reaction? We will present theanswers to these questions in the next section.

III. THEORY

Consider a system with M chemical or biochemical re-actions, or other transitions, and N unique chemical species.The state of the system is an N�1 vector, X�t�, where Xi�t�is the positive integer number of molecules of the ith speciesat time t. Let X�t� be a jump Markov process with an initialcondition, X0=X�t0�. The reaction propensities are an M �1vector a consisting of M positive functions of the state of thesystem, where aj is the propensity of the jth reaction suchthat ajdt is the probability that the jth reaction occurs in dt.The stoichiometric matrix � is an M �N matrix, where � j isthe jth row consisting of the stoichiometry of the jth reac-tion. Each reaction has a putative reaction time � j and thenext reaction to occur is designated as the �th reaction. T k isthe time of the kth occurrence of any reaction. The �th re-action will then cause the state of the system to transitionfrom X�T k� to X�T k+1�+��, where T k+1=min��=��. Weneed to count the number of occurrences of specific reactionsand so we define Rj�t , t�� as the number of occurrences of thejth reaction in the time interval �t , t��.

In order to apply a valid PSSA, we first partition thesystem of reactions into fast/discrete and slow/discrete sub-sets and then evaluate two conditions that indicate whetherthe separation of time scales between the dynamics of thetwo subsets is large enough. The partitioning is based on areaction’s likelihood of causing its participating species toconverge to a quasi-steady-state marginal distribution. Bysimply counting the number of occurrences of each reaction,we dynamically measure the separation of time scales anddetermine when the PSSA is most appropriate. When theapproximation is valid, we then assume that the steady-statedistribution is ergodic and sample the state of the systemover time, using those samples to compute the state of thesystem at future times.

We casually use terms such as “quasi-steady-state mar-ginal distribution” and “probabilistic steady-state approxima-tion” as replacements for their deterministic analogs. Wenote that, while similar, these notions of stability and sepa-ration of time scales are different in a random dynamicalsystem sense. We use the definition of an attractor definedon a compact random set as given by Arnold andSchmalfuss37 and describe its stability as in the “pull back”sense, where the convergence of trajectories to the attractoroccurs in probability. For multidimensional systems, or sys-tems with multiple chemical species, the marginal distribu-tion of a subset of those species may quickly converge to aquasistable attractor, one which remains asymptoticallystable in a lesser-dimensional space, but still evolves overtime in the larger-dimensional space. We call this the quasi-steady-state marginal distribution, but occasionally omit the“quasi.” The probabilistic steady-state approximation usesthe existence of this steady-state marginal distribution to de-termine the distribution of the slow dynamics without simu-
lating the entirety of the fast dynamics.


Below, we present the criteria for partitioning, the twoconditions for detecting convergence and stability of asteady-state marginal distribution, and the methods of com-puting the state of the system at future times. For clarity, wepresent the results in terms of the times and regions identi-fied in Fig. 1. In general, t1 is the time of the previous slow/discrete reaction, t2 is the time when the probability distribu-tion converges to its approximate steady state, t3 is the timewhen the fast/discrete reactions are “turned off,” and t4 is thetime of the next slow/discrete reaction.

A. Dynamic partitioning

Because the objective of the method is to quickly simu-late reaction networks while retaining accuracy, we are mostinterested in identifying reactions which cause their partici-pating species to converge quickly to a quasistable marginaldistribution. The entire system will converge to steady-statedistribution in infinite time, but we are only interested inspecies which relax to a marginal one in a finite and shorttime. The species that are most likely to rapidly converge toa quasistable marginal distribution are the ones which arefrequently affected by reaction events and are initially nearbytheir steady state. The sample space of the distributions ofthese species must also be sufficiently small so that it ispossible to accurately reconstruct the distributions with arelatively small number of samples. By restricting our inter-est to these distributions, we minimize the time required todetect stability and convergence and reduce the number ofnecessary samples.

We select the marginal distribution that meets these re-quirements by partitioning the system into fast/discrete andslow/discrete reaction subsets. The criteria for classifyingeach reaction as fast or slow and discrete or continuous arethe rate, or frequency, of the reaction occurrences and thenumbers of molecules of the participating species. A fasterreaction will cause its participating species to more rapidlyconverge to a steady state. Similarly, a reaction affecting adiscrete species, one with a few number of molecules, causesa greater effect in a shorter period of time, resulting in fasterconvergence. Consequently, we are most interested in apply-ing the PSSA to dilute species which are affected by fastreactions.

All of the reactions in the system are dynamically reclas-sified throughout the simulation time and these definitionsalways apply over a specific time interval, here labeled as�t1 , t4�. If a fast/discrete reaction fails to meet the criteria atany time within the interval it is no longer considered in thefast/discrete subset. Let the jth reaction be classified as be-longing to the fast/discrete subset, FD�t1 , t4�, on the timeinterval �t1 , t4� if the following are both true:

aj�t� � � ∀ t � �t1,t4� �4�

and

∃i:�ij � 0, Xi�t� � � ∀ t � �t1,t4� , �5�

where � �molecules/s� quantifies the minimum rate of thereaction in order to be fast and � �molecules� is the upper
bound of the sample space in which we choose to apply the

steady-state approximation. The fast/discrete reactions occurfrequently and have at least one participating reactant orproduct species with a dynamic trajectory always numberingless than � molecules for the entire time interval. If a reactionhas multiple participating species, only one needs to be dis-cretely valued so that the reaction extent is limited within� / ��ij�. If a reaction does meet the criteria in Eqs. �4� and �5�it is considered a slow/discrete one.

Similar to Cao et al.,33 species are partitioned into sets offast and slow species, Xf and Xs, respectively, on the timeinterval �t1 , t4�. The ith species is classified as a fast one if itis affected by any fast/discrete reaction and is otherwise clas-sified as a slow species if it is not, or

i � Xf iff ∃ j � FD�t1,t4�:�ij � 0,

�6�i � Xs otherwise.

If a species is affected by both fast/discrete and slow/discretereactions it is classified as a fast one.

It is possible that a reaction is both fast and has partici-pating species whose number of molecules all exceed �. Inthis case, we refer to them as fast/continuous reactions, cre-ate another subset for them, and use a hybrid jump/continuous Markov process algorithm to describe theirdynamics.23 For this article, we focus on systems with onlyslow/discrete and fast/discrete reactions.

B. Stability and convergence of an unknowndistribution

The complete joint probability distribution P�Xf ,Xs�may require a long period of time to converge to its stabledistribution. However, we can break up the joint probabilitydistribution into its conditional and marginal componentsand study the behavior of these distributions over the timeinterval of interest, �t1 , t4�. Following Rao and Arkin,32 webreak up the joint probability distribution using

P�Xf,Xs;t�X�t1�,t1� = P�Xs;t�Xf�P�Xf ;t�X�t1�,t1� , �7�

where the first right-hand-side �rhs� term is the conditionalprobability distribution of the slow species and the secondrhs term is the marginal probability distribution of the fastspecies. The two distributions may now be treated separately.

The conditional probability distribution of the slow spe-cies changes slowly over time due to the infrequency of theslow/discrete reaction occurrences. Because the slow/discretereactions are rare, we can make the following trivial state-ment and use it with great effect: If no slow reaction occursin the time interval �t1 , t4�, then the trajectories of the slowspecies do not change from their value at time t1. Conse-quently, the conditional probability distribution of the slowspecies, conditioned on the state of the slow species at time
t1, is simply


P�Xs;t�Xf,Xs�t1�� = �Xs�t1�� ∀ t � �t1,t4� ,

�8�iff ∀ j � FD�t1,t4�, Rj�t1,t4� = 0,

where �x� is the delta function evaluated at x. If the condi-tion in Eq. �8� is true, then the conditional probability distri-bution changes so slowly in time that it is constant for rela-tively large time intervals. We can make no statement onhow the conditional probability distribution is converging toits steady state. Instead, we only state that its distribution isexactly the delta function for the time interval �t1 , t4�.

We now focus on the marginal distribution, its stability,and the time when it converges to a steady-state distribution,which we label t2. The stable marginal distribution is definedto be either a time-independent stationary distribution, suchas

P�Xf,t�X�t1�,t1� = PSS�Xf�X�t1�� ∀ t � �t2,t4� , �9�

or a time-invariant distribution, as in

P�Xf,t�X�t1�,t1� = PSS�Xf,t + 2n/f �X�t1�� ∀ t � �t2,t4� ,

�10�

where the distribution oscillates with some constant fre-quency f . Time-invariant distributions may arise from oscil-latory chemical or biochemical reaction systems.

By the enforced definition of the fast/discrete reactionsand the resulting subset of fast species, the marginal prob-ability distribution of the fast species must always be, atleast, asymptotically stable. The limiting fast species mustalways have trajectories that remain less than � molecules. Ifthe fast/discrete reactions occur numerous times, any of theaffected fast species may either be fully consumed to zeromolecules, exceed � molecules, or achieve a probabilisticsteady state somewhere in between. If a fast/discrete reactioncauses one of its reactant or product species to exceed �molecules, then the reaction is no longer classified as a fast/discrete reaction. Additionally, because the number of mol-ecules of a fast species may be changed by a slow/discretereaction, the occurrence of a slow/discrete reaction will dis-rupt the convergence of the marginal distribution of the fastspecies to a steady state. Using these definitions, one canwrite down a limit on the separation of time scales and showthat the marginal distribution exactly converges to its stablesteady state under the following conditions: If, as time goesto infinity, the number of occurrences of all the fast/discreteand slow/discrete reactions go to infinity and zero, respec-tively, or

∀ j � FD�t1, � �, Rj�t1, � � → � , �11�

∀ j � FD�t1, � �, Rj�t1, � � = 0, �12�

then the following becomes exactly true:

0 � E�Xif�� and P�Xi

f�� = 0, �13�

where the Xi are the limiting reactants of the fast/discretereactions.

If Eqs. �11� and �12� are true, then the marginal distribu-tion of the fast species has been squeezed into a finite do-
main for an infinite amount of time. In that time, it will

frequently visit its entire sample space with the probability offinding the system at any particular state converging to eithera time-independent or time-invariant constant, referred to asconvergence in probability. The sample space of the limitingreactants is described as a hypercube with radius � centeredat the origin and facing the positive quadrant, along with theappropriate measure and probability space. The sample spaceof the other fast species is a linear translation, rotation, orstretching of this hypercube, dependent on the stoichiometryof the fast/discrete reactions. The state of the fast speciesperforms a biased random walk within the sample space. Wedo not make any assumptions on the form of the marginaldistribution of the fast species, such as a Gaussian one, butonly that it can be fully described using a generalized steady-state distribution such as either Eqs. �9� or �10�.

Of course, the separation between the fast/discrete andslow/discrete reactions will never be so wide as to allow theconditions in Eqs. �11� and �12� to be nontrivially true. In-stead, we propose an approximation that allows one to detectthe convergence of the marginal distribution of the fast spe-cies to a steady state. The marginal distribution of the fastspecies has approximately converged to a steady-state distri-bution when each fast/discrete reaction has occurred at least j �pronounced “pomega”� times, or

∀ j � FD�t1,t2�, Rj�t1,t2� � j , �14�

and no slow reaction occurs, such that

∀ j � FD�t1,t4�, Rj�t1,t4� = 0. �15�

The time t2 is the time in which the jth fast/discrete reactionhas occurred at least j times. For simplicity, we will col-lapse the Mfast j parameters into a single one, . Then,each fast/discrete reaction must occur at least times. Notethat the condition in Eq. �15� applies for a longer time inter-val �t1 , t4�, where t4 is greater than t2. When Eq. �14� isevaluated as true, we then tentatively assume that Eq. �15�will also be true until some time t4. As the parameter increases towards infinity the approximation becomes morevalid. In Sec. IV, the parameter is varied to determine theaccuracy of the convergence approximation.

The conditions in Eqs. �14� and �15� are an approximatemeans of detecting the convergence of a marginal distribu-tion to its quasistable steady state. It is possible that thetrajectories of the fast species are slowly exiting the hyper-cube, but do not exit before time t4. By making � smaller,one can more accurately detect the convergence of the mar-ginal distribution to a quasistable distribution by shrinkingthe hypercube. However, by reducing �, less species will beincluded in the subset of fast species, making the probabilis-tic steady-state approximation less effectively applied. Wewill show that, for a typically small �, the approximation isreasonably accurate.

As the simulation of the system progresses forward intime, the number of occurrences of each reaction is counted.If, for some time interval �t1 , t2�, the conditions in Eqs. �14�and �15� are evaluated as true, then the conditional distribu-tion of the slow species is tentatively assumed to be a deltafunction and the marginal distribution of the fast species has
approximately converged to a steady-state distribution. The


joint probability distribution of all of the species is then theproduct of the two distributions, which we begin to sample.Because the samples are taken from the joint probability dis-tribution and not its components, the numerical method, incontrast to the theory, need not distinguish between fast andslow species. Consequently, for the remainder of the paper,we discard the fast and slow superscripts on the state variableX.

C. Sampling the unknown steady-state distribution

Starting at time t2, the joint probability distribution ofthe species is sampled until one of two events occurs: eitherenough samples have been collected to accurately recon-struct the joint probability distribution, which we label timet3, or some slow/discrete reaction has occurred. If a slow/discrete reaction occurs, the condition in Eq. �15� is false andwe may no longer validly approximate the distribution ashaving converged to some steady state. At that point, we areforced to discard any collected samples and restart the count-ing of fast/discrete reactions. If no slow reactions occur be-fore we have finished sampling, then the tentative assump-tion of convergence is true and we may use those samples tocompute the next slow reaction time, which is t4.

By sampling over time, we assume that the steady-statedistribution is ergodic. Birkhoff’s theorem on ergodicity38

states that the ensemble space average of a steady-state dis-tribution is equal to its time average or

��

f�X�PSS�X�dX =1

T�t

t+T

f�t��dt�. �16�

The time average of a jump Markov process may be com-puted by periodically saving the state of the system to atwo-dimensional state matrix, Xsave, such that

Xsavek= X�T pk� , �17�

where p is the periodicity of sampling and by also saving thelifetimes of each state, where the lifetime for the �pk�th savedstate is the difference between the times of the �pk�+1th and�pk�th reaction occurrence or

Lk = T pk+1 − T pk. �18�

The space average in Eq. �16� can then be written as a dis-crete sum over states, each weighted by their lifetimes, or

��

f�X�PSS�X�dX =1

�k

Lk

�k

f�Xsavek�Lk. �19�

If the samples are highly correlated, then the periodicity ofsampling should be increased. One may also compute thecorrelation function of the occurring states and only savestates with a correlation coefficient near zero.

To perfectly reconstruct the joint probability distributionone would need an infinite number of samples. We introducea parameter �, which is the minimum number of samplesneeded to approximate the underlying distribution with a cer-
tain accuracy, and label t3 the time in which � samples are

collected. The probability of finding the state of the system atany time between times t3 and t4 is then proportional to thelifetime of that state or

P̃SS�X = Xksave� � Lk. �20�

By normalizing Eq. �20�, one can obtain a relative steady-state probability distribution of the system for any time be-tween t3 and t4. As the number of samples � goes to infinity,the steady-state distribution may be exactly computed. Theerror between the exact and relative steady-state distributionfollows the common inverse-square-root law, such that

�PSS�X� − P̃SS�X��2 �1

�. �21�

D. Computing the leapfrog state and time

After applying the steady-state approximation and col-lecting enough samples of the underlying distribution, thenext step is to compute the time of the next slow/discretereaction, labeled t4, and determine the state of the system justprior to its execution. The simulation time then leaps forwardto time t4, ignoring or “turning off ” any fast/discrete reac-tion occurrences in between times t3 and t4. The putative, orpossible, times for the occurrence of each slow/discrete re-action are computed using the previously collected samplesof the state of the system. The minimum of these times is theone corresponding to the slow/discrete reaction that occursnext. The probability distribution of the state of the systemjust prior to this reaction’s occurrence is then derived, usingthe evaluation of the reaction propensity at each state as aweighting factor.

Putative reaction times � j occurring with an exponentialdistribution and with a time-dependent propensity aj�t� obeythe constraint

�t

t+�j

aj�t��dt� + log�rj� = 0, �22�

where rj is a uniform random number between zero and 1.One may rewrite the reaction propensity function in terms ofthe state variable X and the time-dependent probability dis-tribution P�X ; t� so that

��

t

t+�j

aj�X�P�X;t�dtdX + log�rj� = 0, �23�

where the integrating variable dX is an abbreviation for dX1,dX2 , . . . ,dXN and � is an abbreviation for the limits of inte-gration for all X. The probabilistic steady-state approxima-tion is applied and the time integral in Eq. �23� is simplifiedto yield

� j��

aj�X�PSS�X�dX + log�rj� = 0, �24�

where PSS�X� may be either the time-independent or time-invariant steady-state distributions, found in Eqs. �9� and

SS
�10�, respectively. If P �X� is time invariant, we use the


time average instead of removing the time dependence of theintegral. Equation �24� is similar to the slow-scale propensityfunctions previously derived,33 but we use states directlysampled from the steady-state distribution instead of comput-ing its moments. By substituting the approximate steady-state probability distribution found in Eq. �20� and using Eq.�19� to replace the space average with a discretized timeaverage, Eq. �24� may be converted to an algebraic expres-sion for the reaction times,

� j =− log�rj�

�k=1

�

aj�Xsavek�Lk/�

i=1

�

Li� , �25�

which is the putative time of the jth reaction, given thesampled states Xsave and lifetimes L. The slow/discrete reac-tion to occur next is the one corresponding to the minimumof the reaction times of the slow/discrete reactions or

t4 = �� = min�� j� + t3, j � FD�t1,t4� , �26�

where the putative reaction times � j are computed using Eq.�25�.

The state of the system at any time between t3 and t4

may be sampled from the distribution in Eq. �20�. However,the state of the system at t4

−, or the time just prior to theexecution of the next slow/discrete reaction, is skewed indistribution towards states that result in a larger reaction pro-pensity. The probability of the system being at any particularstate at time t4

−, given that the �th reaction will occur at timet4, is the product of the probabilities of two different events:that the system exists at a particular state and that the �threaction occurs at this state, stated as

P�X = Xsavek;t = t4

−� = PSS�X = Xsavek�PSS�Xsavek

→ Xsavek

+ ��X = Xsavek;t = t4

−� . �27�

The probability of the �th reaction occurring at a state Xwithin the differential time interval �t+�� , t+��+d�� is aa��X�d��.6 By computing the reaction propensity of the �threaction at each sampled state, stored in Xsave, we can expressthe probability of finding the state of the system just prior tothe execution of the �th reaction as simply proportional tothe reaction propensity at that state or

PSS�Xsavek→ Xsavek

+ ��X = Xsavek� � a��Xsavek

� . �28�

Equations �20� and �28� are then substituted into Eq. �27�and, after normalization, yield the relative probability distri-bution of finding the state of the system at any one of thesample states at time t4

−, which is

P�X = Xsavek;t = t4

−� =a��Xsavek

�Lk

�i=1

�

a��Xsavei�Li

. �29�

The state of the system at time t4 is determined by first sam-pling the probability distribution in Eq. �29� and then updat-ing it according to the execution of the next slow/discrete
reaction.

The previous section details the approximations utilizedin developing a set of criteria for the stability, convergence,and sampling of a steady-state probability distribution. First,we use the parameters � and � to identify the fast/discreteand slow/discrete reactions and then apply the PSSA to onlythe fast/discrete subset. The PSSA utilizes three assumptions:when no slow/discrete reactions occur within the interval�t1 , t4� and when all fast/discrete reactions occur at least times within the interval �t1 , t2�, then the distribution hasconverged to a stable steady state; the steady-state distribu-tion is ergodic; and the steady-state distribution is adequatelyapproximated with only � samples. Using these approxima-tions, we may ignore all of the fast/discrete reaction occur-rences within the interval �t3 , t4� and still compute the time ofthe next slow/discrete reaction, t4, and the state of the systemat that time. After we detail the numerical implementation,we rigorously measure the accuracy of the approximationswith respect to the parameters �, , and �.

IV. ACCURACY AND SPEED

We present four examples to illustrate the advantagesand limitations of the probabilistic steady-state approxima-tion and to analyze how the separation of time scales in asimple system affects the accuracy and efficiency of the pro-posed method. In the examples, we vary the approximation’sparameters �� , ,�� to determine their effect on both theaccuracy of the solution and the speed up of the method. Forsimplicity, we keep the parameter � constant at a value wherethe examples contain only discrete reactions. All reportedcomputational times are from single realizations generatedon a single Itanium2 1.5 GHz processor.

A. Definitions of error, speed, and networkcharacteristics

To objectively assess the accuracy and efficiency of theequation-free PSSA, or any other simulation method, quan-titative measures must be derived that effectively distinguishthe exact from the approximate solution. Since multiple defi-nitions of error exist in the field of stochastic processes, wedefine and use the weak mean and variance errors as well asthe L2 distance between the probability distributions. Theweak mean and variance errors describe the inaccuracies inthe first two moments of the solution, while the L2 distancemeasures the composite error in the distribution itself. Char-acteristics of the reaction network also affect the method’sperformance. Therefore, we define and use two measurablequantities that describe the separation of time scales of astochastic chemical kinetic system. These characteristics arethe stiffness in event execution and the size of the gap intime scales. By showing that the proposed method’s perfor-mance has a relationship to these network characteristics, weseek to not only demonstrate the method’s ability, but also topredict its performance on any arbitrary reaction network.

An essentially exact solution of each example reactionnetwork is generated by running numerous independent real-izations using the original stochastic simulation algorithm,computing the mean, variance, and probability distribution of
all chemical species over time. We then simulate the reaction


networks using the PSSA with the same initial conditionsand compute the error quantities. The weak mean error of theith chemical species is computed using

�meani �t� = �E�Xi

SSA�t�� − E�XiPSSA�t�� 30�

and, for simplicity, the maximum weak mean error is definedas

�meanmax �t� = max��mean

i �t�� with respect to i . �31�

The weak variance error is similarly computed using

�vari �t� = �var�Xi

SSA�t�� − var�XiPSSA�t�� 32�

and the maximum weak variance error is also defined as

�varmax�t� = max��var

i �t�� with respect to i . �33�

The L2 distance between probability distributions is definedas

��Pi�2�t� = �x��i

�PSSA�Xi = x;t� − PPSSA�Xi = x;t��2,

�34�

where the sum is taken over all sampled values of the num-ber of molecules of the ith species or �i. Both probabilitydistributions are conditioned on the same initial conditionsand time. Of the three measurable quantities of error, the L2

distance is the most stringent.The stiffness in event execution, or simply stiffness, is a

measure of the difficulty of accurately executing individualor “bundles” of reactions when stochastically simulating ajump Markov process. The stiffness of a reaction network isdefined as the ratio between the maximum and minimumreaction propensities at a certain time, or

Sf�t� =max�a�t��min�a�t��

, �35�

which, if normalized, is the highest reaction propensity in thereaction network. The reaction propensities are in units ofreactions per second, making Sf the number of reaction oc-currences of the fastest reaction for every one occurrence ofthe slowest reaction. We also define the size of the gap intime scales, or the size of the largest separation between thefast and slow time scales at a certain time, as

G�t� =min�af�t��

�j

ajs�t�

, �36�

where af and as represent the fast/discrete and slow/discretereaction propensities, respectively. We show that the quantityG is a good estimate of the efficiency of the probabilisticsteady-state approximation. We also define the speed up ofthe proposed method as

Speed up =TSSA

TPSSA , �37�

where T is the average computational time of a single trial,calculated from numerous trials. Using these quantities, we
rigorously measure the accuracy and efficiency of the pro-

posed method using reaction networks with specific andquantitatively measured characteristics.

B. The illustrative example revisited

The illustrative example described in Sec. II has an av-erage stiffness �Sf of 9.54E4 and an average separation oftime scales �G of 6.52E4. Using the stochastic simulationalgorithm and the probabilistic steady-state approximationwith a parameter set �� , ,�� of �10, 10, 10�, we run 10 000independent trials of the reaction network. At this �, reac-tions �1� and �2� are always classified as fast/discrete andreaction �3� is always slow/discrete. The probabilistic steady-state approximation is applied over 250 000 times with only897 executed reaction events and a computational time of4.78�10−3 s per trial. The time evolution of the mean andvariance of each chemical species for both the exact andapproximate solution is shown in Fig. 2. The mean of theapproximate solution is extremely accurate, while there is asmall systematic error in the variance. The probability distri-butions of species A and E over multiple increasing times areshown in Fig. 3, where, for convenience, we include the L2

distance of the difference between the probability distribu-tions. We would like to note that the L2 distance is a goodmeasure of the accuracy of the distributions, capturing verysmall, visually imperceptible differences.

By increasing the parameters and � we sought todecrease the weak mean and variance errors as well as the L2

distance. For all parameter sets, we run 10 000 independenttrials using the probabilistic steady-state approximation andcompute the error quantities. In Fig. 4, we show the timeevolution of the maximum weak mean and variance errorswhen increasing either or � or both. For this simple reac-tion network, we find that increasing the minimum conver-gence criteria, , by 250-fold only decreased the weak meanerror by about 25% and did not significantly affect the weakvariance error. Increasing the minimum number of samples,�, by 250-fold decreased both the weak mean and variance

FIG. 2. The time evolution of the �top� mean and �bottom� variance ofspecies A, E, and C of the illustrative example reaction network, using eitherthe �solid/blue� stochastic simulation algorithm or the �circles/red� probabi-listic steady-state approximation with parameter set �� , ,��= �10,10,10�.

errors by 700%. It is possible that the convergence of the



marginal distributions of species A, B, and C occurs so rap-idly that the minimum value of tested here is sufficient.We next varied and � logarithmically between 10 and2500 for a total of 625 parameter sets while keeping � con-stant at 10. The resulting average L2 distance between theprobability distributions is shown in Fig. 5, where the aver-aging of the distance is taken over both species and timepoints. With increasing �, the L2 distance initially decreaseslike �−0.35 and then remains roughly constant at a � equal to200. Similar to the weak mean and variance, with increasing , the L2 distance does not significantly decrease.

As the parameters and � increase, the PSSA is applied

FIG. 3. The probability distributions of species A and E of the illustrativeexample reaction network at multiple time points, using either the �solid/blue� stochastic simulation algorithm or the �dashed/red� probabilisticsteady-state approximation with parameter set �� , ,��= �10,10,10�. �In-set� The time evolution of the L2 distance between the probability distribu-tions for species A, C, and E.

FIG. 4. The time evolution of the �top� maximum weak mean error and the�bottom� maximum weak variance error of the illustrative example reactionnetwork, using the probabilistic steady-state approximation with �=10 and
varying and �.

less often and results in fewer leaps in simulation time, caus-ing the computational time to increase with either parameter.Table I summarizes the effects of and � on the speed up.For this example, the speed up linearly decreases with eitherparameter. Because the speed up linearly decreases and theerror in the solution exponentially decreases, there is an op-timum parameter set that yields good accuracy while maxi-mizing the speed up. For this reaction network, however,using the fastest parameter set still results in a reasonablyaccurate solution.

C. A previously examined example

Consider the following reaction network used to studythe implicit and explicit tau-leap methods:24

S1→c1

� ,

2S1�c3

c2

S2,

S2→c4

S3,

with kinetic parameters c1=1 s−1, c2=10 �molecules s�−1,c3=1000 s−1, and c4=0.1 s−1 and initial conditions X1�0�=400, X2�0�=798, and X3�0�=0 molecules. This reaction

FIG. 5. The average L2 distance between exact and PSSA-enabled probabil-ity distributions of the illustrative example reaction network as a function ofthe parameters and � with �=10.

TABLE I. The effect of and � on the probabilistic steady-state approxi-mation’s speed up when simulating the illustrative example reaction net-work. Parameter � is constant at 10.

/� 10 20 100 1000 2500

10 1279 858 267 31.3 13.120 820 648 243 30.8 13.1

100 233 217 140 28.4 12.61000 26.0 25.6 24.3 14.7 9.12500 10.8 10.8 10.5 8.4 6.4



network has an �Sf of 2.89E4 and a �G of 2.24E3. Rathi-nam et al. report a speed up of 7.8 and 380 for the explicitand implicit tau-leap methods, respectively.24 Here, we per-form a similar analysis as in the first example, using both theoriginal stochastic simulation algorithm and the PSSA meth-ods with specific parameter sets to simulate the system onthe time interval �0,0.2� s. Varying and �, we computethe probability distribution of all species at t=0.2 s, the L2

distance between the exact and approximated distributions atthis time, and the maximum weak mean and variance errorsover time. For this example, we set � to 30 000, keeping itwithin the wide gap in the time scales. Using the first param-eter set of �10, 10�, the probability distributions of all threespecies are very accurately captured �Figs. 6–8� with a speedup of 76.37. By increasing or � or both by a factor of 10,

FIG. 6. The probability distribution of species S1 in the second example att=0.2 s comparing the usage of either the �solid/blue� stochastic simulationalgorithm, the �circles/red� probabilistic steady-state approximation with� ,��= �10, 10� or the �triangles/black� PSSA with � ,��= �100, 100�.

FIG. 7. The probability distribution of species S2 in the second example att=0.2 s comparing the usage of the stochastic simulation algorithm and theprobabilistic steady-state approximation. The lines and markers are the same
as in Fig. 6.

we obtain reductions in the L2 distance, but also in the speedup �Table II�.

We note that there is an initial spike in the weak meanand variance errors followed by a relaxation to very smallvalues �Fig. 9�. In the beginning of this simulation, the mar-ginal distribution of the fast species requires a long time torelax to a quasi-steady-state distribution. Before it has re-laxed, however, the criterion for relaxation is met and thePSSA is dynamically applied. When the distribution of thefast species is far from its steady-state distribution, but isassumed to be at steady state, the result is the loss of accu-racy. Note, however, that the method has good convergenceproperties, reproducing the correct solution within a smallperiod of time even though its initial solution is highly inac-curate. The initial inaccuracy is one reason why the methodshould normally be used on reactions with at least one dis-cretely valued species, limiting the extent of the reaction.

D. Nonlinear protein-protein interactionnetwork

In the next example, we demonstrate how the equation-free probabilistic steady-state approximation can quickly andaccurately simulate a system of nonlinear biochemical reac-tions. The system is a network of protein-protein interac-tions, consisting of four small proteins �S1–S4� each capableof binding to a large scaffold protein, P. Small proteins S1

and S3 competitively bind to the same binding site on P,while S2 and S4 also compete for a second binding site on P.The scaffold protein forms nine complexes, consisting of thefree scaffold and all permissible combinations of boundsmall proteins. There are 24 binding and unbinding fast/discrete reactions of a small protein to a complexed or freescaffold. The single slow/discrete reaction is the conversionof S1 to S3+A by the enzymatically active complex S1 :S2 : P.The reactions, kinetic constants, and initial conditions arelisted in Table III. The average stiffness �Sf of this reactionnetwork is 1.10E6, while the average size of the gap in time

FIG. 8. The probability distribution of species S3 in the second example att=0.2 s comparing the usage of the stochastic simulation algorithm and theprobabilistic steady-state approximation. The lines and markers are the sameas in Fig. 6.

scales, �G , is 1.36E5.



Initially, the fast/discrete reactions will cause the proteinspecies to converge to a steady-state distribution. With eachslow/discrete reaction occurrence, the steady-state distribu-tion will be disrupted and the fast dynamics will force thesystem to relax to a different steady-state distribution. Be-cause the small proteins compete for the same binding sitesin a series of bimolecular reactions, the relaxation process ishighly nonlinear. We chose this system because both its fast/discrete and slow/discrete dynamics are nonlinear and aredescribed by a chemical master equation whose solution is,at this time, unsolvable. Unlike other methods, instead ofsolving the master equation for the dynamics of the fast/discrete reactions, we obtain the steady-state distribution oftheir effects by direct sampling. This allows the equation-freeprobabilistic steady-state approximation to simulate highlycomplex nonlinear systems without the hassle of derivingequations and then developing approximations to make themsolvable.

For the interval �0,100� seconds, we run 2200 indepen-dent realizations of the protein-protein interaction networkusing the stochastic simulation algorithm and 10 000 inde-pendent realizations using the PSSA with multiple differentparameter sets. The stochastic simulation algorithm requires5035 s per trial to execute 4.436E8 reaction events. With�� , ,��= �2000,10,10�, the probabilistic steady-state

TABLE II. Accuracy and speed up of the probabilreaction network. Parameter � is constant at 30 000.

Method Exec� ,�� Speed up react

SSA 1.00 332PSSA �10, 10� 76.37 2PSSA �100, 10� 17.75 16PSSA �10, 100� 14.79 9PSSA �100, 100� 9.14 22

FIG. 9. The maximum weak mean and variance errors for the second ex-ample comparing the probabilistic steady-state approximation with different
parameter sets.

approximation requires only 2.329 s per trial to execute191 995 reaction events. In Figs. 10 and 11, we show theprobability distributions of selected species from the nonlin-ear protein-protein interaction network at a time of 100 s.Using a small value of � yields a reasonably accurate prob-ability distribution for most species. However, there is no-ticeable error, especially in the S1 : P complex, which has anL2 distance of 0.6742 at this time. Keeping � constant at2000, we sought to decrease the error by increasing � by afactor of 100. In Table IV, we report the effect of increasing� on the speed up, average number of executed reactionevents, and average L2 distance of the probability distribu-tions.

The species S1 : P has a very peaked distribution near theorigin with a mode of three molecules. The proposed methodinaccurately captures the proportional magnitude of the dis-tribution because the reactions which consume S1 : P mol-ecules have a very sharp threshold between being classifiedas fast/discrete or slow/discrete. With zero S1 : P molecules,the rate of consumption of S1 : P molecules is zero and thereaction is classified as slow/discrete. However, with a singleS1 : P molecule, the reaction has a large rate and it is classi-fied as fast/discrete. Because the distribution is so close tothe origin, the constant switching of the reactions affectingS1 : P inadvertently causes the sampling to bias the distribu-tion towards zero molecules, resulting in the observed error.Increasing � fivefold decreases the L2 distance of S1 : Pthreefold, but the effect is not linear and further increases donot reduce the error. Adding memory to the partitioning ofreactions, such as a hysteresis loop, may also decrease the

steady-state approximation for the second example

L2 distanceS1 S2 S3

0 0 00.0202 0.0195 0.01470.0128 0.0138 0.01350.0160 0.0130 0.01210.0135 0.0136 0.0142

TABLE III. The reactions, kinetic constants, and initial conditions of theprotein-protein interaction network.

S1+ P�c2

c1

S1 : P S1+S2 : P�c2

c1

S1 :S2 : P S1+S4 : P�c2

c1

S1 :S4 : P

S2+ P�c2

c1

S2 : P S2+S1 : P�c2

c1

S1 :S2 : P S4+S1 : P�c2

c1

S1 :S4 : P

S3+ P�c2

c1

S3 : P S3+S4 : P�c2

c1

S3 :S4 : P S2+S3 : P�c2

c1

S2 :S3 : P

S4+ P�c2

c1

S4 : P S4+S3 : P�c2

c1

S3 :S4 : P S3+S2 : P�c2

c1

S2 :S3 : P

S1→ae

S3+Ac1=100 �molecules s�−1 , c2=1E4 s−1, ae=0.1�S1��S1:S2: P� /1000+ �S1�S10

=S20=S30

=S40= P0=150 molecules,V0=1E−15 L

istic

utedions

475780230397575



observed error. However, other species whose peaked distri-butions have modes of six molecules are accurately captured;making this source of error restricted to species whose num-ber of molecules are very close to the origin.

The probabilistic steady-state approximation is able toaccurately produce the distribution of the other species, evenwith the least accurate parameter set. As � increases, theaverage L2 distance of the other species steadily decreases,while also reducing the speed up. For this example, the num-ber of executed reaction events did not linearly increase withincreasing �. Instead, a 100-fold increase in � results in onlya fivefold increase in the reaction events and a threefold de-crease in the speed up.

E. A diagnostic multiple time scale reaction network

In the last example, we systematically analyze how thesize of the gap in the time scales G of a reaction networkaffects the speed up of the probabilistic steady-state approxi-mation. We construct a simple diagnostic reaction network,consisting of eight uncoupled pairs of synthesis and degra-dation reactions, each producing or consuming a singlechemical species �reactions are listed in Table V�. The pairsof reactions are labeled RA–RH. The system is initialized atand fluctuates around the average steady-state values, mak-ing convergence to a steady-state distribution extremely fast.The average rates of reactions of the eight pairs are logarith-mically decreased from a high value of 1E7 molecules/ s forreaction pair RA to 1 molecules/ s for reaction pair RH. Theaverage stiffness �Sf of the reaction network is 1E7. Wesystematically increase the G by successively turning offpairs of reactions with intermediate rates. With all pairs ofreactions turned on, G is O�101�. To increase the size of thegap, we turn off one or more pairs of reactions. By turning

7

FIG. 10. The probability distributions of species S1 : P, S2 : P, S3 : P, S4 : P,and S1 :S2 : P from the nonlinear protein-protein interaction reaction networkat a time of 100 s obtained by using either the �solid/blue� original stochas-tic simulation algorithm, the �circles/red� probabilistic steady-state approxi-mation with � ,��= �10,10�, and the �triangles/black� PSSA with � ,��= �10,1000�.

off all pairs of reactions except RA and RH, G is O�10 �.


We then use the probabilistic steady-state approximationto run 100 independent simulations of the diagnostic reactionnetworks for 1 s of simulated time, varying G from O�10� toO�107�. We keep the parameters � ,�� constant at�25,1000�. While the accuracy of the solution is unaffectedby the choice of �, if it is incorrectly chosen, the PSSA israrely applied and the computational time of the simulationis similar to the original method. Here, we choose � to sitsomewhere in the middle of the gap. On average, the sto-chastic simulation algorithm will execute 2.222�107 reac-tion events in 1 s of simulated time. With �G =10, theprobabilistic steady-state approximation is rarely, if ever, ap-plied. As a result, the method executes a similar number ofreaction events compared to the stochastic simulation algo-rithm. However, the computational time is about 14.7% morethan the stochastic simulation algorithm, due to the overheadof dynamically partitioning reactions and counting reactionoccurrences to detect possible convergence. As pairs of reac-tions are turned off, the PSSA is more frequently applied,resulting in fewer executed reaction events and a greaterspeed up. The speed up as a function of the size of the gap inthe time scales is summarized in Fig. 12. The approximatemethod becomes faster when �G is roughly 20 and increasesproportionally to �G 0.6815 until leveling off at about 107.

V. DISCUSSION

The probabilistic steady-state approximation is anequation-free method that dynamically determines when theunknown probability distribution of species in a system hasconverged to a quasistable steady state and then, by samplingthat steady-state distribution, calculates the time of the nextrare reaction event and the state of the system at that time.The method speeds up the simulation of the stochastic dy-namics of a jump Markov process featuring any system offast/discrete and slow/discrete reactions, including nontrivial,nonlinear ones whose chemical master equation is currently

FIG. 11. The probability distributions of species P, A, and S1 :S4 : P from thenonlinear protein-protein interaction reaction network at a time of 100 susing the same lines and markers as in Fig. 10.

unsolvable. The approximation is most frequently and effec-



tively applied when the separation between time scales of thejump Markov process is large. With a large gap in the timescales, the application of this method results in speed ups ofmany orders of magnitude, compared to the original stochas-tic simulation algorithm. Unlike other approximations, it isimportant to note that PSSA retains accuracy in not only themean and variance of the solution, but also the distributionitself. Systems with interesting behavior such as multimodeldistributions39 require a method that reproduces the probabil-ity distribution of the system and not just the first two mo-ments. Along with the advantages of this method, there arealso some notable limitations, although these limitations maybe transient obstacles to a better algorithm. We will addressboth topics below.

The theory developed in this paper may be applied tomany other types of kinetic Monte Carlo simulations and isnot dependent on the behavior of the forward propagation ofthe system. Besides studying biochemical or chemical reac-tion networks, one may also utilize the proposed theory insimulations of ecological, population, evolutionary, or trafficflow dynamics. In this paper, we use the next reaction variantof the stochastic simulation algorithm to propagate the dy-namics of the well-stirred system. However, one may easilysubstitute a tau or binomial leap method17–19 to generatestates of the system. One may also apply the equation-freeprobabilistic steady-state approximation to heterogeneousstochastic numerical methods, such as the spatial stochasticsimulation5 and Green’s-function reaction dynamics40 meth-ods. It is even more difficult to find analytical solutions of amaster equation describing nonlinear heterogeneous systemsand our equation-free approach is an ideal method for deter-mining the quasi-steady-state distribution. By dynamically

TABLE IV. The effect of increasing � on the accnonlinear protein-protein interaction network.

Method� ,�� Speed up

Reactionevents

SSA 1.0 4.436E8PSS �10,10� 2162 1.92E5PSS �10,50� 2150 2.07E5PSS �10,100� 2090 2.14E5PSS �10,500� 1227 3.54E5PSS �10,1000� 810 5.30E5

TABLE V. A diagnostic reaction network with multiple time scales.

Pairs of reactions Kinetic constants

RA : →A→ ks=1E7 molecules/ s, kd=1E5 s−1

RB : →B→ ks=1E6 molecules/ s, kd=1E4 s−1

RC : →C→ ks=1E5 molecules/ s, kd=1E3 s−1

RD : →D→ ks=1E4 molecules/ s, kd=1E2 s−1

RE : →E→ ks=1E3 molecules/ s, kd=1E1 s−1

RF : →F→ ks=1E2 molecules/ s, kd=1E0 s−1

RG : →G→ ks=10 molecules/ s, kd=1E−1 s−1

RH : →H→ ks=1 molecules/ s, kd=1E−2 s−1

Initial conditions are 100 molecules for all speciesVolume=1E−15 L


detecting the relaxation of the system to a quasistationarydistribution in a three-dimensional space and then extractingsamples from that distribution, the method bypasses all ofthe difficulties in working with these highly intractable mas-ter equations and also reduces the computation and memoryrequirements by only sampling states that have high prob-abilities, which become important when dealing with high-dimensional joint probability distributions. Similar to thecurrent procedure, one then uses the samples to compute thetime and state of the system of the next important �slow�event. For heterogeneous systems, however, the state of thesystem now includes both the position and the identity ofindividual molecules. We anticipate further development inthis area.

The advantage of the equation-free probabilistic steady-state approximation is its ability to separate the time scalesof an arbitrary reaction network into fast and slow ones,detect the relaxation of the marginal distribution of the fastspecies to some quasistable marginal distribution, correctlysample the noise of the marginal distribution of the fast spe-cies, and use those samples to accurately predict the state ofthe system at future times. The proposed method does notapproximate the stable marginal distribution of the fast

and efficiency of the stochastic simulation of the

Average L2 distancewithout S1 : P

Average L2

distance of S1 : P

0 00.0304 0.67420.0273 0.22780.0256 0.26370.0250 0.37450.0254 0.3879

FIG. 12. �Circles� The effect of the average size of the gap in time scales,�G , in a diagnostic reaction network on the speed up of the probabilisticsteady-state approximation with � ,��= �25,1000�. �Dashed� A straight-line fit to the log-log transformation of the data with the algorithm perfor-mance scaling proportionally to �G 0.6815. �Dotted vertical� The breakevenpoint in terms of speed up for using the probabilistic steady-state approxi-

uracy

mation, including the computational overhead.



species as a Gaussian distribution nor any other form of dis-tribution. The equation-free probabilistic steady-state ap-proximation is also not limited to carefully selected or linearchemical reaction networks. In addition, we show that, byassuming a distribution is ergodic and directly sampling itmidsimulation, one can use those samples to correctly pre-dict the future.

With each occurrence of a slow/discrete reaction, thequasistable marginal distribution of the fast species maychange. We take the conservative approach and recalculatethe marginal distribution of the fast species after each slow/discrete reaction occurrence. If a slow/discrete reaction oc-curs before the fast dynamics have relaxed or enoughsamples have been extracted to accurately compute a distri-bution, we ignore the partially obtained results and startanew. If the distribution of the fast species relaxes to a stablesteady state and we sufficiently sample it, we only simulatethe occurrence of a single slow reaction and thereafter as-sume that the fast species are no longer at a probabilisticsteady state. Even with such a conservative and possiblywasteful approach, the computational time of the simulationis still reduced by orders of magnitude. Further improve-ments or approximations may additionally increase the speedup. We will address some of the possible improvements be-low.

The limitations of the probabilistic steady-state approxi-mation, as it is currently formulated, are that it requires asufficiently large gap in the time scales to speed up the simu-lation and that its accuracy and efficiency are dependent onthe values of its parameters. While it may be possible toselect, perhaps even automatically, the optimal values of pa-rameters for a given reaction network, it is more difficult toapply this theoretical approach to systems with highly mixedtime scales with little separation. We address these limita-tions below. We also suggest additional ways to detect therelaxation of a system to a quasistationary distribution thatdoes not utilize the counting of reaction occurrences.

A. Optimal parameter selection

The assumptions in the equation-free probabilisticsteady-state approximation are parametrized by �� ,� , ,��.The parameters � and � dictate the partitioning of reactionsinto fast/discrete and slow/discrete subsets, where � controlsthe maximum number of molecules of some reactant speciesin order for the reaction to be considered discrete and �controls the minimum rate of the reaction in order for it to beconsidered fast. The parameter � describes the minimumnumber of fast/discrete reaction occurrences before their ef-fects cause convergence to a quasi-steady-state distribution,while the parameter determines the number of samplestaken from the distribution. The values of these parametersdetermine the efficiency and accuracy of the method and,unlike the � and � in a previous hybrid stochastic method,23

the optimal values depend on the reaction network and ontime. Parameter selection then becomes an important task.

Altering the parameters and �, but not �, will affectthe error of the solution. As one increases , the method
more stringently detects the relaxation of the fast species to a

quasi-steady-state distribution. As goes to infinity, the de-tection of this distribution becomes exactly accurate. Practi-cally, the parameter should be the same order of magni-tude as �, although there is no guarantee that this issufficiently restrictive. For example, if the combined occur-rences of two or more fast/discrete reactions cause the num-ber of molecules of a particular species to change veryslowly over time, then after numerous occurrences of thesereactions the species may still not have relaxed to its steady-state distribution. However, the error in this case is typicallysmall and is not appreciably propagated forward in time.Similarly, as one increases �, the method more accuratelysamples the quasi-steady-state distribution and better pre-dicts the time and state of the next slow/discrete reaction.The theory suggests that the error contributed from approxi-mate sampling is proportional to �−0.5. However, in the firstexample, the error of the solution is found to scale propor-tionally to �−0.35. The increased error may arise from othercontributions or the theoretical prediction may be an upper-bound limit.

Because the optimal parameter selection depends on thestate of the system, the optimal values will change over thecourse of a simulation. In order to minimize the error of thesolution and maximize the speed up, it would be best todynamically select values of the parameters based on somecriteria. These values may be initially computed at the begin-ning of the simulation and periodically after each occurrenceof a slow/discrete reaction. A large variety of criteria may bederived. For example, selecting a value of ��t� that maxi-mizes the quantity G�t� is a straightforward process. Optimalvalues for j�t� and � j�t� will depend on the current state ofthe system and the reaction propensities of each reaction,where we note the addition of the j index to indicate thateach reaction may have its own criteria. An optimal value of� j�t� should be large enough so that every fast/discrete reac-tion has occurred and changed the state of the system. If� j�t� is too small then the sampled states will consist of thoseonly changed by the fastest fast/discrete reactions. As men-tioned before, states may also be periodically sampled. Inorder to sample states resulting from the occurrences of boththe fastest and the slowest fast/discrete reactions, the periodof sampling p�t� should be about �max�af�t�� /min�af�t��. Inthe end, parameter selection is always a trade off betweenaccuracy and speed. As one increases the parameters and�, the accuracy of the solution will increase, but the speed upof the method will also decrease.

B. Additional methods of detecting quasi-steady-statedistributions

In the current formulation of the method, we count thenumber of occurrences of the fast/discrete reactions and as-sume that the system has reached a quasi-steady-state distri-bution when each fast/discrete reaction occurs a minimalnumber of times. While this procedure works well in manycases, there are alternative methods that may be substituted.We will briefly describe an alternate method.

The method is a semiempirical one that uses the auto-
correlation functions of the fast species to determine the sys-


tem’s relaxation to a quasi-steady-state distribution. The au-tocorrelation coefficient of a species whose marginaldistribution has relaxed to a time-independent function willbe zero. By computing the autocorrelation function of thefast species and monitoring the time when all of the autocor-relation functions have decayed to near zero, one can com-pute the time in which the system has relaxed to a quasi-steady-state. The autocorrelation function of a species isdefined as

Cxx�t� =��X�t0� − �x��X�t� − �x�

��X�t0� − �x� 2 , �38�

where �x is the mean of X between times t0 and t and theautocorrelation coefficient Cxx�t� varies between full correla-tion �1� and fully anticorrelated �−1�.

In addition, to ensure proper sampling of the distribu-tion, one should save states of the system for between fiveand ten times the maximum characteristic relaxation time ofthe autocorrelation functions. However, because these auto-correlation functions are computed and used while the simu-lation is running, it is difficult to compute their ensembleaverage. Not surprisingly, the autocorrelation function of asingle trajectory is extremely noisy �Fig. 13�a�� and it is dif-ficult to calculate their relaxation times. Instead, we may use

FIG. 13. The autocorrelation functions of single trajectories of the S1, S1 : P,and S1 :S2 : P species from the protein-protein interaction network over thetime interval �0,0.015�. �a� No filtering has been applied. �b� A 20-pointbackward moving average has been applied to the raw data. �c� A 20-pointforward/backward zero-phase-distortion filter has been applied to the rawdata. The black arrows show the times at which the system is assumed tohave relaxed to a quasi-steady-state distribution.

signal-processing filters to reduce the noise. By running a


20-point backward moving average �Fig. 13�b�� or a 20-pointzero-phase-distortion moving average �Fig. 13�c�� on the sig-nals, the noise in the autocorrelation functions of single tra-jectories is largely reduced. When each of the autocorrelationfunctions has decayed towards zero �Figs. 13�b� and 13�c�,black arrows�, we may then more accurately assume that thesystem has relaxed to a quasi-steady-state distribution andbegin the rest of the procedure.

C. The effect of the separation in time scales onperformance

The size of the gap in time scales, G, greatly affects theefficiency of the equation-free probabilistic steady-state ap-proximation. At G� �20, application of the PSSA results ina speed up of between 0.85 and 0.9 due to the overhead costsof the method. However, as G increases, the speed up dra-matically improves, up to the value of Sf itself. Whereas theoriginal stochastic simulation method requires the most timeto simulate a system with a large G, the probabilistic steady-state approximation effectively simulates such systems. It isimportant to note that the quantity G may be measured at anytime during the simulation, making conditional usage of theprobabilistic state approximation possible.

The stiffness parameter Sf is also a good measure of thecomputational requirement of simulating a reaction networkusing any simulator that seeks to execute individual orbundles of reaction events over time. However, a reactionnetwork with a high Sf does not guarantee that there exists alarge gap in time scales. It is possible that the occurrences ofsome slow/discrete reaction may be as frequent as the occur-rences of any single fast/discrete reaction. If all of the reac-tions are highly coupled, then the evolution of the fast dy-namics may be inseparable from the slow dynamics withouta great loss of accuracy. In this case, the quantity G would berelatively small and the conditions in Eqs. �14� and �15� willrarely evaluate as true, causing the PSSA to be infrequentlyapplied.

There may also be multiple separations between timescales, allowing for the partitioning into multiple subsets ofreactions. A modified PSSA may then be applied to detectwhen the fastest subset of reactions has caused the state ofthe system to achieve a quasi-steady-state distribution andthen determine what the state of the system will be when areaction in the next slowest subset occurs. This process maythen be repeated, resetting a given reaction’s counter onlywhen a reaction in a slower subset has occurred. It is alsopossible to transform the system of reactions into a reducedone consisting of separated fast and slow variables.41 Amodel reduction method may increase the separation of timescales and the quantity G, further increasing the efficiency ofan equation-free probabilistic steady-state approximation.

VI. CONCLUSION

The equation-free probabilistic steady-state approxima-tion is an accurate and efficient method for stochasticallysimulating arbitrary reaction networks consisting of fast/discrete and slow/discrete reactions. The method may be
implemented in conjunction with existing kinetic Monte


Carlo simulators5,6,18,19,40 and may be used on systems ofreactions whose dynamics are highly nonlinear. The methoddoes not assume any approximation on the shape or form ofthe steady-state distribution and its sampling reproduces notonly the mean and variance of the solution, but also theprobability distribution itself. The probabilistic steady-stateapproximation may also be integrated into existing hybridstochastic methods21–23 to effectively treat reaction networksof fast/continuous, fast/discrete, and slow/discrete reactions,with or without high degrees of stiffness. A FORTRAN95 nu-merical implementation of the equation-free probabilisticsteady-state approximation is available upon request.

ACKNOWLEDGMENTS

We would like to thank David C. Morse and Marie N.Contou-Carrere for thoughtful suggestions and valuable dis-cussion. This work was supported by grants from the Na-tional Science Foundation �BES-0425882� and the NationalInstitutes of Health �NIH training Grant No. GM08347�.Computational support from the Minnesota SupercomputingInstitute �MSI� is gratefully acknowledged. This work wasalso supported by the National Computational Science Alli-ance under TG-MCA04N033.

1 M. Samoilov, S. Plyasunov, and A. P. Arkin, Proc. Natl. Acad. Sci.U.S.A. 102, 2310 �2005�.

2 E. M. Ozbudak, M. Thattai, I. Kurtser, A. D. Grossman, and A. vanOudenaarden, Nat. Genet. 31, 69 �2002�.

3 H. Salis and Y. Kaznessis, Comput. Chem. Eng. 29, 577 �2005�.4 S. Krishna, B. Banerjee, T. V. Ramakrishnan, and G. V. Shivashankar,Proc. Natl. Acad. Sci. U.S.A. 102, 4771 �2005�.

5 A. B. Stundzia and C. J. Lumsden, J. Comput. Phys. 127, 196 �1996�.6 D. T. Gillespie, J. Comput. Phys. 22, 403 �1976�.7 M. A. Gibson and J. Bruck, J. Phys. Chem. A 104, 1876 �2000�.8 Y. Cao, H. Li, and L. Petzold, J. Chem. Phys. 121, 4059 �2004�.9 R. Gunawan, Y. Cao, L. Petzold, and F. J. Doyle III, Biophys. J. 88, 2530�2005�.

10 C. G. Moles, P. Mendes, and J. R. Banga, Genome Res. 13, 2467 �2003�.11 C. N. Chen, C. I. Chou, C. R. Hwang, J. Kang, T. K. Lee, and S. P. Li,

Phys. Rev. E 60, 2388 �1999�.12 K. S. Brown and J. P. Sethna, Phys. Rev. E 68, 021904 �2003�.


13 X. J. Feng, S. Hooshangi, D. Chen, G. Li, R. Weiss, and H. Rabitz,Biophys. J. 87, 2195 �2004�.

14 A. G. Makeev, D. Maroudas, A. Z. Panagiotopoulos, and I. G. Kevreki-dis, J. Chem. Phys. 117, 8229 �2002�.

15 A. G. Makeev, D. Maroudas, and I. G. Kevrekidis, J. Chem. Phys. 116,10083 �2002�.

16 D. T. Gillespie, J. Chem. Phys. 115, 1716 �2001�.17 D. T. Gillespie and L. R. Petzold, J. Chem. Phys. 119, 8229 �2003�.18 T. Tian and K. Burrage, J. Chem. Phys. 121, 10356 �2004�.19 A. Chatterjee, D. G. Vlachos, and M. A. Katsoulakis, J. Chem. Phys.

122, 024112 �2005�.20 D. T. Gillespie, J. Chem. Phys. 113, 297 �2000�.21 E. L. Haseltine and J. B. Rawlings, J. Chem. Phys. 117, 6959 �2002�.22 J. Puchalka and A. M. Kierzek, Biophys. J. 86, 1357 �2004�.23 H. Salis and Y. Kaznessis, J. Chem. Phys. 122, 054103 �2005�.24 M. Rathinam, L. R. Petzold, Y. Cao, and D. T. Gillespie, J. Chem. Phys.

119, 12784 �2003�.25 Y. Cao, L. R. Petzold, M. Rathinam, and D. T. Gillespie, J. Chem. Phys.

121, 12169 �2004�.26 J. G. Gaines and T. J. Lyons, SIAM J. Appl. Math. 57, 1455 �1997�.27 P. M. Burrage, R. Herdiana, and K. Burrage, J. Comput. Appl. Math.

171, 317 �2004�.28 P. E. Kloeden and E. Platen, Numerical Solution of Stochastic Differential

Equations �Springer-Verlag, Berlin, 1992�.29 G. N. Milstein, E. Platen, and H. Schurz, SIAM �Soc. Ind. Appl. Math.�

J. Numer. Anal. 35, 1010 �1998�.30 E. Hausenblas, SIAM �Soc. Ind. Appl. Math.� J. Numer. Anal. 40, 87

�2002�.31 H. S. Fogler, Elements of Chemical Reaction Engineering �Prentice-Hall,

Englewood Cliffs, NJ, 1999�.32 C. V. Rao and A. P. Arkin, J. Chem. Phys. 118, 4999 �2003�.33 Y. Cao, D. T. Gillespie, and L. R. Petzold, J. Chem. Phys. 122, 014116

�2005�.34 J. Goutsias, J. Chem. Phys. 122, 184102 �2005�.35 C. Gadgil, C. H. Lee, and H. G. Othmer, Bull. Math. Biol. 67, 901

�2005�.36 L. Chen, P. G. Debenedetti, C. W. Gear, and I. G. Kevrekidis, J. Non-

Newtonian Fluid Mech. 120, 215 �2004�.37 L. Arnold and B. Schmalfuss, J. Diff. Eqns. 177, 235 �2001�.38 G. Mircea, Stochastic Calculus: Applications in Science and Engineering

�Birkhauser, Boston, 2002�.39 T. S. Gardner, C. R. Cantor, and J. J. Collins, Nature �London� 403, 339

�2000�.40 J. S. van Zon and P. R. ten Wolde, Phys. Rev. Lett. 94, 128103 �2005�.41 R. Bundschuh, F. Hayot, and C. Jayaprakash, Biophys. J. 84, 1606

�2003�.


an equation-free probabilistic steady-state approximation ... · of chemical or biochemical...

Documents