BIO

PHYS

ICS

AN

D

COM

PUTA

TIO

NA

LBI

OLO

GY

The effect of network topology on the stabilityof discrete state models of genetic controlAndrew Pomerance1, Edward Ott, Michelle Girvan, and Wolfgang Losert

Department of Physics, University of Maryland, College Park, MD 20742b;

Edited by Peter J. Bickel, University of California, Berkeley, CA, and approved February 26, 2009 (received for review January 7, 2009)

Boolean networks have been proposed as potentially useful mod-els for genetic control. An important aspect of these networks is thestability of their dynamics in response to small perturbations. Pre-vious approaches to stability have assumed uncorrelated randomnetwork structure. Real gene networks typically have nontrivialtopology significantly different from the random network para-digm. To address such situations, we present a general method fordetermining the stability of large Boolean networks of any speci-fied network topology and predicting their steady-state behaviorin response to small perturbations. Additionally, we generalize tothe case where individual genes have a distribution of “expres-sion biases,” and we consider a nonsynchronous update, as wellas extension of our method to non-Boolean models in which thereare more than two possible gene states. We find that stability isgoverned by the maximum eigenvalue of a modified adjacencymatrix, and we test this result by comparison with numerical sim-ulations. We also discuss the possible application of our work toexperimentally inferred gene networks.

B oolean networks have been extensively investigated as amodel for genetic control of cells (1, 2). In this model, each

gene is represented by a node of a network, and each node hasone of two states: on—i.e., producing (“expressing”) its targetprotein—or off. Directed links between genes indicate that onegene influences the expression of another, either through theexpressed protein directly binding to DNA or to other signalingpathways that modulate transcription of a gene. In the standardBoolean network model, the system evolves in discrete time steps(t = 0, 1, 2, . . .), and at each step the state of every node is simul-taneously updated according to some function of its inputs. Thisfunction approximates the action of activators (proteins that actto increase expression of a given gene) or inhibitors (proteins thatact to reduce expression). Although this model might seem to bean oversimplification considering the complex kinetics involved inall steps of a transcription pathway, experimental evidence sug-gests that real biological systems are, in some cases, reasonablywell-approximated by Boolean networks (3).

In 1969, S. A. Kauffman (1) introduced a type of Boolean net-work known as an N −K network. In this model, there are N nodeseach having exactly K input links, and the nodes from which theseinput links originate are chosen randomly with uniform probabil-ity. We refer to the number of input (output) links to (from) a nodeas the in-degree (out-degree) of that node. At any given time t,the system state can be represented as an N-vector whose ith com-ponent σi(t) is either zero or one, where zero or one correspondsto off or on and i = 1, 2, . . . , N . There are 2N possible states. Thefunction determining the time evolution at each node is defined bya random, time-independent, 2K -entry truth table. Because this is afinite, deterministic system, there is always an attractor: eventually,the system must return to a previously visited state (finiteness),after which the subsequent dynamics will be the same as for theprevious visit (determinism). These attractors can be fixed pointsor periodic orbits. By using the Hamming distance between twostates [i.e., the number of nodes for which the σi(t) disagree] asthe distance measure, the system exhibits both what is termed a“chaotic” (or unstable) regime, where the distance between typicalinitially close states on average grows exponentially in time, as wellas a stable regime, where the distance decreases exponentially.

Between the two there is a “critical” regime. (Here, by “close” wemean that the Hamming distance is small compared with N .)

As a model of genetic control, these attractors have been pos-tulated to represent a specific pattern of protein expression thatdefines the cell’s character (1). In single-celled organisms, theseattractors might be taken to correspond to different cell states(growing, dividing, starving, heat- or pH-shocked). In multicellu-lar organisms, different cell types (muscle, nerve, liver, etc.) havedifferent expression patterns, and, within each type, a cell couldbe in a variety of states (resting, “activated,” dividing, etc.) thateach correspond to different expression patterns. Boolean net-work approximations have been successful in predicting the geneexpression time sequence of the segment polarity gene networkin Drosophilia, a model for embryonic development where indi-vidual cells turn specific proteins on and off in patterns that guidethe growth of certain organs and structures (3). Because the pro-tein expression pattern of the cell is modeled from the state ofthe corresponding Boolean network, the question of the stabilityof the network then becomes important: do small perturbationsin the expression pattern, due perhaps to chemical fluctuations,die out quickly, returning the cell to its original state, or do theyquickly grow, pushing the cell into another state? The purpose ofthis article is to examine the stability of network dynamics in thecontext of discrete state models of gene networks.

One motivation for the consideration of dynamical stability isits possible relevance to cancer. Specifically, we hypothesize thatdynamical instability of a gene network might be a causal mecha-nism contributing to the occurence of some cancers. We empha-size that this hypothesis is distinct from the previous hypothesisof “genomic instability” as a cause of cancer (4). In particular,genomic instability has been defined (as in the glossary of NatureGenetics) as “the failure to transmit an accurate copy of the entiregenome from one cell to its two daughter cells.” In contrast, theinstability we refer to is that of the dynamics of a given gene net-work, and we use the term “dynamical network instability” (DNI)to distinguish this condition. We speculate that DNI might arisefrom mutations and that, once established, as cells divide, DNIcould lead to widely varying gene expression patterns from cell tocell. We emphasize that DNI implies that this variation wouldarise even in the absence of further mutation. That is, similarto the concept of chaos in continuous-state dynamical systems(e.g., ref. 5), DNI causes exponential sensitivity of typical sys-tem trajectories to small changes, which we speculate may leadto many different outcomes in the course of cell division. Recentmicrodissection results indicate wide variations in gene expressionpatterns, even for nearby cells within the same cancerous tissue(6). This variability provides a basis for understanding why cancercan adapt and evade treatment (7).

Author contributions: A.P. and E.O. designed research; A.P. and E.O. performed research;A.P. and E.O. analyzed data; and A.P., E.O., M.G., and W.L. wrote the paper.

The authors declare no conflict of interest.

This article is a PNAS Direct Submission.1To whom correspondence should be addressed. E-mail: pomeranc@umd.edu.

This article contains supporting information online at www.pnas.org/cgi/content/full/0900142106/DCSupplemental.

www.pnas.org / cgi / doi / 10.1073 / pnas.0900142106 PNAS May 19, 2009 vol. 106 no. 20 8209–8214

Dow

nloa

ded

by g

uest

on

Aug

ust 2

3, 2

020

Another motivation for our study is the argument, put forwardby Kauffman (2), that evolution favors gene networks that are onthe border between stability and instability (8–11). Whether ornot our cancer hypothesis or Kauffman’s stability-border hypoth-esis holds, the question of dynamical stability of such networks iscrucial to their understanding and use as models.

Although previous works have addressed the question ofdynamical network stability in simple, specific types of randomnetworks (e.g., N − K nets), in this article we address the ques-tion of dynamical network stability for general network topologyand node attributes. We also consider nonsynchronous updateand extend the considerations to non-Boolean models allowingfor the possibility of nodes having more than two states. Thus, ourwork provides a potentially enhanced framework for modeling andusing the discrete state network paradigm. In particular, we con-sider how our network stability considerations can be employedon experimentally derived gene networks.

In the original N − K nets as proposed by Kauffman, the truthtable output governing node dynamics was randomly chosen withon and off having equal probability. Subsequently, it was shownthat if the truth table output was biased such that p denotes theprobability of randomly assigning an off output, the transitionbetween the stable and chaotic regimes depends on p (12). Weterm p the “expression bias.” Additionally, networks with a distri-bution of in-degrees, but no in-/out-degree correlation, have beenconsidered in refs. 13–16, and it has been shown that the nodal in-degree average, 〈K in〉, suffices to determine the stability. (Here,〈·〉 indicates average of a nodal quantity over all nodes.) Specif-ically, the critical average number of connections, Kc, governingthis transition is

Kc = 1/[2p(1 − p)], [1]

where the network is stable for 〈K in〉 < Kc, unstable for 〈K in〉 > Kc,and critical for 〈K in〉 = Kc. Aldana and Cluzel (16) consideredthe consequences of Eq. 1 in the case of networks with scale-freetopology (17), i.e., the in-degree (out-degree) distribution P(K in)[P(Kout)] is a power-law: P(K) ∝ K−γ . (Since every out-link froma node is an in-link to another, 〈K in〉 = 〈Kout〉; thus, the result isunchanged whether it is the in- or out-degree that has power-lawscaling.)

Recently, some authors have noted, but not numerically tested,a generalization of Eq. 1 that takes into account nodal correlationsbetween the in-degree and out-degree characterized by the jointK in − Kout degree distribution function P(K in, Kout). In this case,the critical transition occurs at (18)

〈K inKout〉〈K〉 = 1

2p(1 − p). [2]

We emphasize that Eq. 1 was derived in the annealed approx-imation (see later discussion) for networks with a given in- orout-degree distribution P(K) and with the complimentary linkscompletely random, and that Eq. 2 uses only the additional infor-mation contained in the nodal in-/out-degree correlation. Fur-thermore, all nodes (“genes”) were taken to have the same pvalue. However, gene networks, in common with real networksoccurring across a broad range of applications, can be expected todeviate substantially from the above simple network model. Exam-ples of network properties that could make previous analyses ofnetwork stability inapplicable are assortativity/disassortativity (19)(the tendency for highly connected nodes to prefer or avoid link-ing to other highly connected nodes) and community structure(20) (the existence of highly connected, sparsely interconnectedsubgraphs), two properties that are not captured in the degreedistributions. Additionally, these properties may have biologicalimplications. For example, a recent article (21) examined geneinteraction networks from cancerous tissue and found significantcommunity structure, as well as positive correlation between the

in-degree and out-degree of nodes; additionally, protein interac-tion networks have been shown to exhibit significant disassorta-tivity (19, 22). Furthermore, for modeling purposes, it might beimportant to allow the expression bias p to vary from node tonode [as an extreme example, so-called housekeeping genes (23)have a predominant tendency to be on, corresponding to low p,unlike other genes]. In this article, we derive and test the stabilitycriterion for large networks with arbitrary network topology andheterogeneous expression biases. In particular, our theory eval-uates the stability of any given network with its specific topology(i.e., its adjacency matrix A defined subsequently) and its node-specific expression biases. We show that stability is determinedby the largest eigenvalue of a modified adjacency matrix, and wenumerically test this criterion.

With respect to real gene networks, the synchronous update atinteger times (t = 0, 1, 2, . . .) used in the above models representsan additional deviation from the real situation, where chemicalkinetics and transport processes can be expected to introducenon-trivial dynamics. As a partial step toward remedying this (andto make Boolean approximations suitable for atmospheric andgeophysical processes), Ghil and Mullhaupt (24) consider a gen-eralization in which t is a continuous variable and σi(t) dependson σj(t−τij), where τij is a delay time that can be different for eachlink from j to i. The original formulation (e.g., in refs. 1, 12–16)corresponds to τij = 1 for all i, j. We will argue and numericallyconfirm that the criterion determining the stability/instability bor-der of this generalization of the Boolean network model is thesame as that for the synchronous update models.

In addition to nonsynchronous update, another generalizationof Boolean networks that we will examine is models in which eachnode i is allowed to have one of Si possible discrete states (e.g.,for Si = 3, we label the states σi ∈ {0, 1, 2}, and for Boolean net-works Si = 2 for all i). This model may be closer to the behavior ofactual cells, and models with multiple states can be related to cer-tain piecewise ODE models of transcription (25, 26). The generalmodel using arbitrary, multivalued truth tables has been previouslytreated in the special case of N −K networks with all nodes havingthe same number of possible states S. In the case where each pos-sible state is equally likely, the critical number of inputs has beenshown by Sole et al. (see http://arxiv.org/abs/adap-org/9907011)to be

Kc = 11 − 1/S

. [3]

The applicability of our work to any specific network and set ofnodewise expression biases may be of particular interest in situa-tions where experimental data provide the possibility of estimatinga gene network and expression biases. Such information could beused as input to our method which could give an indication of thestability of a given experimentally derived network. The possibilitythat such analyses may be feasible becomes more and more likelywith the rapid technological advances in obtaining new types ofhigh-quality, quantitative data useful for deducing gene networks.For example, such analyses could be used to address the hypoth-esis that dynamical instability of gene networks is connected withthe occurence of cancer.

ModelDeterministic Boolean networks are formally defined by a statevector �(t) = [σ1(t)σ2(t) . . . σN (t)]T , where σi ∈ {0, 1}, and a setof update functions fi, such that

σi(t) = fi(σj(i,1)(t − 1), σj(i,2)(t − 1), . . .), [4]

where j(i, 1), j(i, 2), . . . , j(i, K ini ) denote the indices of the K in

inodes that input to node i; we denote this set of nodes by Ki ={j(i, k)|k = 1, 2, . . . , K in

i }. The update function fi is defined at each

8210 www.pnas.org / cgi / doi / 10.1073 / pnas.0900142106 Pomerance et al.

Dow

nloa

ded

by g

uest

on

Aug

ust 2

3, 2

020

BIO

PHYS

ICS

AN

D

COM

PUTA

TIO

NA

LBI

OLO

GY

node i by specifying a truth table whose outputs are randomly pop-ulated. Previous analytic results assumed a constant expressionbias for all nodes; however, we allow that, in the truth table fornode i, output entries are randomly assigned zero with probabilitypi or one with probability 1 − pi. In the case of uniform expressionbias, we drop the subscript and use the notation p ≡ pi.

We consider the interaction structure of this system as a graphwhere the nodes represent individual elements of the state vector,and a directed edge is drawn from node j to node i if j ∈ Ki. Anadjacency matrix A is defined in the usual way: Aij is one if thereis a directed edge from node j to node i and zero otherwise.

The stability of a large Boolean network is defined by consid-ering the trajectories resulting from two close initial states, �(t)and �(t). To quantify their divergence, the Hamming distance ofcoding theory is used: h(t) = ∑N

i=1 |σi(t) − σi(t)|. If the networkis stable, on average h(t) → 0 as t → ∞. In unstable networks,h(t) quickly increases to O(N), while a “critical” network is at theborder separating stability and chaos.

To study the stability of an N −K Boolean network, Derrida andPomeau (12) considered an annealed situation and calculated theprobability that, after t steps, a node state is the same on twotrajectories that originated from initially close conditions. [Thiscalculation was later generalized to variable in-degree (13–15)and joint degree distribution (18).] In Derrida and Pomeau’s“annealed” situation, at each time step t the truth table outputsand the network of connections are randomly chosen. The actualsituation of interest, however, is the case of “frozen-in” networks,where the truth table and network of connections are fixed in time.It has been commonly assumed that analytical results obtained forthe annealed case are a good approximation to the frozen-in case(e.g., refs. 12–15). We also adopt this view in a modified form, andwe will test its predictions with numerical simulations.

The randomization of the network of connections at each timestep while keeping the degree distribution fixed carries the implicitassumption that there is no additional dynamically relevant struc-ture in the frozen network other than that contained in the jointdegree distribution P(K in, Kout). To avoid this assumption, weobtain theoretical results for a different annealing protocol, whichwe term “semiannealed.” In this semiannealed procedure, we keepthe network fixed (i.e., the adjacency matrix A does not changewith time), and we envision randomly assigning the output entriesof the truth table of each node i at every time t according to thetime-independent expression bias pi assigned to node i. We thenimagine tracking the probability that individual node states σi(t)and σi(t) differ over time with an N-dimensional difference vec-tor, whose components are yi(t) = 〈〈|σi(t) − σi(t)|〉〉, where 〈〈·〉〉denotes an average over every possible small initial perturbation.Here by “every possible small initial perturbation” we mean allperturbations for which a small fraction ε of the states are flipped.Additionally, we define the “sensitivity” qi as the probability thatthe output of fi changes when given two different input strings, sim-ilar to the “average sensitivity” of ref. 27. In the case of completelyrandom Boolean functions,

qi = 1 − (p2

i + (1 − pi)2) = 2pi(1 − pi). [5]

Thus, similar to ref. 12, we can write the update equation for yi as

yi(t) = qi

⎛⎝1 −

∏j∈Ki

(1 − yj(t − 1))

⎞⎠ . [6]

Eq. 6 follows from noting that the probability that σj and σj areequal is (1 − yj) and thus the probability that all inputs to node i areequal is the above product. Note that this equation uses topologi-cal information contained in the Ki. However, we have treated yj,yj′ , and qi as if they were probabilities of statistically independentrandom events. We hypothesize that this semiannealed protocol

might be expected to yield good results for frozen-in cases whenthe network is large and the fraction of network nodes on shortloops is small [the network is locally tree-like (28)]. To see theproblem posed by short loops, consider a node with two inputsthat themselves have inputs both coming from a common node;in this case, the elements of y(t) in Eq. 6 are no longer statisticallyindependent and multiplying the probabilities is no longer cor-rect. Our numerical tests of frozen networks indeed yield resultsthat agree very well with our semiannealed hypothesis on large,locally tree-like networks as well as networks with a large numberof feedforward motifs, a non-tree-like three-node subgraph thathas been found to be prevalent in real gene networks (30).

The case where both network states are exactly the same corre-sponds to yi(t) = 0, which is a fixed point of Eq. 6. To determinethe stability of this fixed point, we linearize Eq. 6 around y(t) = 0for small perturbations:

yi(t + 1) ≈ qi

N∑j=1

Aijyj, [7]

where Aij are the elements of the adjacency matrix A. Eq. 7 canbe written in matrix form as y(t + 1) = Qy(t) where

Qij = qiAij. [8]

Thus the largest eigenvalue of this matrix λQ governs stability:

λQ > 1, y = 0 is unstable;λQ = 1, y = 0 is critical; [9]λQ < 1, y = 0 is stable.

Since Qij ≥ 0, the Perron–Frobenius theorem (31) guarantees thatλQ is real and positive. We also note that, for any given adjacencymatrix A and assignment of qi’s to nodes, Eq. 6 can be iteratednumerically to predict the expected time-asymptotic saturationvalue of the difference in two initially nearby states when evolvedto steady state. We numerically test this prediction, as well as thestability criterion in Eq. 9 in the next section.∗

As a special case of interest, if the qi are uniform, qi ≡ q, thenλQ = qλ, where λ is the maximum eigenvalue of the adjacencymatrix. This yields the critical condition,

λ = 1/q. [10]

Furthermore, for the case of a large network whose links arerandomly assigned subject to a joint probability distributionP(K in, Kout) at each node (with no assortativity), the mean-fieldapproximation for the largest eigenvalue is (32)

λ ≈ 〈K inKout〉〈K〉 , [11]

where, since 〈K in〉 = 〈Kout〉, we use the notation 〈K〉 ≡ 〈K in〉 =〈Kout〉. Eqs. 10 and 11 yield the same criterion as in Eq. 2. Inthe case where K in and Kout are uncorrelated, P(K in, Kout) =Pin(K in)Pout(Kout) and 〈K inKout〉 = 〈K〉2, yielding Eq. 1.

The eigenvalue of random network adjacency matrices withassortativity has been considered in ref. 32, which defines anassortativity measure ρ as

ρ = 〈K ini Kout

j 〉e

〈K inKout〉 , [12]

∗ We note that Eq. 8 and the condition λQ = 1 also occurs in the treatment (28) of sitepercolation on directed networks where different sites have different removal prop-erties. A similar condition involving 〈K2〉/〈K〉 also arises in percolation on undirectednetworks (29).

Pomerance et al. PNAS May 19, 2009 vol. 106 no. 20 8211

Dow

nloa

ded

by g

uest

on

Aug

ust 2

3, 2

020

where 〈K ini Kout

j 〉e denotes an average over all links (i, j) from nodei to node j. The network is assortative (disassortative) if ρ > 1(ρ < 1). For ρ near one, the largest eigenvalue λ is approximatelygiven by (32)

λ ≈ 〈K inKout〉〈K〉 ρ. [13]

Thus, it is predicted that, for uniform q, assortativity (disassorta-tivity) decreases (increases) the critical q value.

In the case of nonuniform qi, we have recently generalizedEq. 11 to obtain an analogous mean-field approximation to λQwithout assortativity or community structure,

λQ ≈ 〈qK inKout〉〈K〉 . [14]

Our derivation of 14 will be published elsewhere. From Eq. 14,we see that correlation (anticorrelation) between q and K inKout

decreases (increases) network stability and that, in the absenceof correlation, the result is similar to that for a uniform q, λQ ≈〈q〉〈K inKout〉/〈K〉, with 〈q〉 replacing the uniform q (Eqs. 9 and 10).

We now consider the generalization to allow any number of dis-crete node states. We denote the number of possible states of nodei by Si, and we label the possible states 0, 1, 2, . . . , Si −1. The num-ber of possible inputs to i from the set Ki of nodes that influence itis

∏j∈Ki

Sj. For each of these possible inputs, the truth-table func-tion fi in Eq. 4 assigns one of the Si possible states to node i. Similarto the Boolean case, we take the assignment to be random and tohave an “expression bias” pi,s for each of the s = 0, 1, 2, . . . Si − 1node states, where pi,s denotes the probability that fi, for a given setof inputs, assigns the state s to node i, and

∑s pi,s = 1 for all nodes

i. As in the Boolean case, we can then introduce the sensitivity qigiving the probability that two different sets of inputs result in adifferent updated state of node i, which, in the random truth tablecase,

qi = 1 −∑

s

p2i,s. [15]

With this definition, we see that all our previous reasoning stillapplies, and Eqs. 6–9 hold with this generalized expression forthe node sensitivities and with yi(t) interpreted as the probabilityof disagreement between σi(t) and σi(t). In the case of uniformnumber of node states, Si ≡ S, and equal expression biases,pi,s ≡ ps = 1/S, among these states, Eq. 15 becomes q = 1 − 1/S,which, when combined with Eq. 10 yields the previous resultin Eq. 3.

Finally, we note that our criticality criterion, λQ = 1, isunchanged by the presence of delays, as in the models of refs. 24,and only a slight modification is required of Eq. 6 [i.e., yj(t − τij)replaces yj(t − 1)]. The condition λQ = 1 implies that the com-ponents of y in Eq. 6 are time-independent. Thus we predict thatthe delays τij do not influence the result, and the criticality condi-tion in Eq. 9 is independent of the synchronous update structureof the most commonly used random Boolean network models.Similarly, the time-asymptotic steady state obtained by repeatediteration of Eq. 6 is, by definition, time-independent and thusalso does not depend on the τij (although the τij will influencethe time-dependent approach to the asymptotic steady state; seesupporting information (SI) Appendix).

Statistical MethodsWe numerically test the above predictions on several classes ofBoolean networks with uniform sensitivity (i.e., qi = q is the samefor all nodes):

(a) Random networks with K in = Kout;(b) Random networks with imperfect correlation between K in

and Kout;

(c) Networks with assortativity or disassortativity; and(d) Networks constructed as in (a) but with a substantial

number of feedforward loops.

We additionally test our predictions on two classes of networkswith nonuniform sensitivities:

(e) Networks constructed as in (a) but where nodes havedifferent sensitivities correlated with the degrees of thenodes; and

(f) Networks with significant community structure, where thetwo communities have different, uniform sensitivities.

Finally, we test our generalization to more than two node states onnetworks of type (a) but with Si = 4 for all nodes. For types (a)–(c)and (e), we use networks with truncated power-law degree distri-butions. (Evidence for the presence of this type of distribution ingene networks has been seen in ref. 33.)

The algorithms for constructing the networks of types (a)–(c)are as follows. (i) Establish the in- and out-degrees for each node,which are drawn from a distribution,

P(K) ∝{

K−γ , K ≤ Kmax,0, K > Kmax,

[16]

where γ = 2.1 and Kmax = 15 (Boolean case) or Kmax = 8 (Si = 4case). The out-degree is initially set to the in-degree. (ii) Randomlyswap the out-degrees between pairs of nodes. If maximal corre-lation between in- and out-degrees is desired, as in (a), this stepis skipped so that K in = Kout and 〈K inKout〉 is maximal. A com-pletely uncorrelated network has every nodal out-degree swappedexactly once, yielding 〈K inKout〉 = 〈K〉2. The quantity 〈K inKout〉,which approximately determines λ by Eq. 11, can thus be tunedby the number of nodes that have their out-degrees swapped.(iii) Place links randomly between nodes subject to the constraintsof the specified in- and out-degrees assigned at each node by the“configuration model” (34). (iv) If networks with assortativity (dis-assortativity) are desired, as in (c), perform a given number oflink swaps, as in ref. 32, that increase (decrease) the assortativityρ in Eq. 12. In all cases we employ networks with N = 104 andtwo initial conditions separated by a Hamming distance of 100.In the SI Appendix we discuss finite size effects that can occur forsmaller N .

We emphasize that, although we determine our networks ran-domly, in our numerical experiments we do not average over thisrandomness. Rather, we generate one random network for eachexperiment and examine the resulting behavior of that specificnetwork.

ResultsWe test the steady-state predictions of Eq. 6 and the criticalitycondition of Eq. 9 in Fig. 1, and compare the calculated criti-cal parameters to the mean-field-type approximations of Eqs. 11,12, and 14 in SI Appendix. To compare the theory (solid curvesin Fig. 1) to experimental measurements of the Hamming dis-tance from numerical evolution of true frozen Boolean dynamicalsystems (markers in Fig. 1), we simulate Eq. 6 to steady-stateand calulate the expected fractional Hamming distance from thedistance vector with

y = limt→∞

1N

∑i

yi(t). [17]

When simulating test Boolean networks, we calculate the steady-state fractional Hamming distance as the average from t = 90 tot = 100 when all delays are the same ([τ ]ij = 1) or from t = 490to t = 500 and normalize by the network size N . These timesare well after the steady-state value is reached (see SI Appendix).

8212 www.pnas.org / cgi / doi / 10.1073 / pnas.0900142106 Pomerance et al.

Dow

nloa

ded

by g

uest

on

Aug

ust 2

3, 2

020

BIO

PHYS

ICS

AN

D

COM

PUTA

TIO

NA

LBI

OLO

GY

Fig. 1. Simulation results. (A) y vs. q for three networks with different largest eigenvalues (λ ≈ 4.4, 2.9, 2.3), both with uniform delay on all links τij = 1(closed markers) and with half of the links having increased delay of τij = 10 (open markers). The solid curves correspond to the prediction y (defined in Eq.17) obtained by simulating Eq. 6. The downward vertical arrows correspond to qcrit = 1/λ for each of the three networks. (B) y vs. q for three networks withdifferent assortativities. (C) y vs. q for networks with added feedforward motifs, with an illustration of the feedforward motif (Inset). (D) y vs. q0 for networkswith maximum correlation (circles) and minimum correlation (squares) between K in

i Kouti and qi , where qi is drawn from a uniform distribution centered at q0

with width 0.1. (E) y vs. θ∩/(θ∪ + θ∩) for networks with community structure, where the two communities have qa = 0.5 and qb = 0.1. (F) y vs. p0 for a networkwhere each node can take one of Si = 4 possible states. p0 is the probability that a zero appears in the truth table output; the remaining three symbols appearwith equal probability.

Each experimental data point in Fig. 1 corresponds to a single real-ization of interconnections averaged over 100 realizations of thetime-independent truth table with specified sensitivity as before.

In-/Out-degree Correlations and Heterogeneous Time Delay. Fig. 1Ashows the steady-state Hamming distance as a function of the sen-sitivity for one network of type (a) (λ = 4.4) and two of type (b)(λ = 2.9, 2.3). The closed markers in the figure represent exper-iments with uniform delay τij = 1 on all links, while the openmarkers correspond to experiments where half the links, randomlychosen, have τij = 10 and the remainder have τij = 1. Importantly,the degree distributions are the same for all three networks, andwe attain different λ values by varying the correlation between thein-degree and the out-degrees. We see from Fig. 1A that there isclose agreement between the theoretical prediction and the exper-imental results and our prediction that the presence of delays doesnot change the stability is confirmed. Additionally, the measuredsteady-state Hamming distance is essentially zero below the crit-ical value of the sensitivity, qcrit = 1/λ (this point is indicatedby vertical downward arrows in Fig. 1A). We emphasize that thedegree distributions (and hence 〈K〉) are the same for the net-works in Fig. 1A, and thus, if the in-/out-degree correlation wereignored, the observed difference between the stability conditionsfor these networks would not be predicted.

Assortativity/Disassortativity. Fig. 1B shows results obtained whensignificant assortativity or disassortativity is present [type (c) net-works]. In this experiment, as well as all those reported below,the delays are all uniform. The networks under considerationhave the same joint degree distribution with K in = Kout. How-ever, each of the networks have very different assortativities(ρ = 0.52, 1.0, 1.7, defined in Eq. 12), which yield different largesteigenvalues (λ = 3.0, 4.4, 9.9). Since the joint degree distributionsare the same, Eq. 2 would predict that the three networks havethe same stability characteristics. However, since their eigenval-ues are very different, we predict that, as observed, the transitionsof the three networks occur at different values of q. Again thetheoretical predictions of qcrit are indicated by vertical arrows.

Motifs. Random construction of networks, as used in the net-works above, is expected to yield networks that are locally tree-like(32). However, we note that biological and other types of net-works often have motifs (small subgraphs) that occur with higherfrequency than in randomly constructed networks (30). For genenetworks of Escherichia coli and Saccharomyces cerevisiae, it wasfound that the number of feedforward loop motifs (see Fig. 1CInset) is significantly enhanced compared with the expected num-ber in a randomly constructed network. In these networks, thenumber of feedforward loops per node c is roughly 0.1. Thus, weconsider a network of type (a) (λ ≈ 2.9 and N = 104) after adding1,000 (c = 0.1) and, in an extreme case, 2,000 (c = 0.2) feedfowardloops. To add a feedforward loop, we randomly choose a node A,follow a random output to node B, and follow a random outputof B to node C. We then add a link from node A to node C. Wedo this a given number of times, avoiding nodes that already par-ticipate in an added feedforward loop. In Fig. 1C, we see that thesemiannealed theory of Eq. 6 (solid curve) again agrees well withour numerical experiments (solid markers). Based on such results,we believe that the locally tree-like network requirement does notinvalidate application of our method to real gene networks. Wealso note that the critical point is essentially unchanged by theaddition of loops (adding links only slightly increases the largesteigenvalue), however, more feedforward loops tend to increasethe steady-state Hamming distance for q > qcrit.

Application to S. cerevisiae. As a real biological example, weinclude in the SI Appendix a graph similar to those in Figs. 1A–Cusing a published network for the yeast S. cerevisiae (35).

Heterogeneous Correlated Sensitivities. Fig. 1D demonstrates theeffect of a distribution of qi’s on the stability of a network withK in

i = Kouti = Ki and with correlation between the nodal values of

qi and Ki, i.e., type (d) networks. We consider two situations, onewhere 〈qK2〉/(〈q〉〈K2〉) is maximal, and one where it is minimal.The qi’s are drawn from a uniform distribution centered at q0 (theabscissa in the figure), with width q = 0.1. Maximal (minimal)〈qK2〉 is attained by assigning the largest qi to the node with the

Pomerance et al. PNAS May 19, 2009 vol. 106 no. 20 8213

Dow

nloa

ded

by g

uest

on

Aug

ust 2

3, 2

020

largest (smallest) Ki, the second largest qi to the node with the sec-ond largest (second smallest) Ki, and so on. As can be seen fromthe figure, there is good agreement between the semiannealed the-ory and the numerical experiments, and the two networks becomeunstable at different values of q0. (Vertical arrows again indicatethe points where λQ = 1.)

Community Structure. Fig. 1E shows our results for a case wherethere is community structure and community-dependent sensitiv-ity. To construct the networks in Fig. 1E, consider the case wherethere are two communities, and we assign a link from node i incommunity a to node j in community b with probability θab. Weimpose the additional constraints that θaa = θbb ≡ θ∪ and thatθab = θba ≡ θ∩, and the sizes of the two communities are thesame, N/2. We take 〈K in〉 = 〈Kout〉 = 〈K〉 = (θ∪ + θ∩)N to be thesame for both communities, and we also assume that communitiesa and b have different sensitivities qa and qb, respectively. As θ∩is increased from zero to θ∩ = θ∪, λQ changes from the case oftwo completely separated communities to one of a single randomnetwork. Communities a and b have equal sizes of 5,000 nodes,community a has qa = 0.5, and community b has qb = 0.1. To varyλQ, we vary θ∪ and θ∩, keeping their sum constant to maintainconstant 〈K〉. As with the curves in Fig. 1A–C, the transition tochaos is governed by λQ (λQ = 1 at the vertical arrow), and Eq. 6(solid curve) accurately predicts the numerically observed (solidcircles) steady-state Hamming distance.

Non-Boolean Models. Fig. 1F illustrates an application to a casein which there are more than two possible states at each node. Inparticular, we consider S ≡ Si = 4 possible states at each node.(Since the number of possible inputs to the truth table for node i

in this case is 4K ini , we take Kmax = 8 due to memory constraints.)

Labeling the possible node states σi ∈ {0, 1, 2, 3}, we take nodesto have uniform expression biases for occurrence of state-label 0,p0 ≡ pi,0, from 0 to 1. The three remaining labels (S = 1, 2, 3)also have uniform biases for all nodes i, ps ≡ pi,s = (1 − pi,0)/3.From Eq. 15, q ≡ qi = 1 − [p2

0 + (1 − p0)2/3], which has a maxi-mum qmax = 0.75 at p0 = 0.25. As can be seen in the figure, thepredicted fraction of nodes with differing states y (solid curve)also has a maximum there. It is again seen that the measurements(markers) are well predicted by the theory.

DiscussionIn this article, we have presented theoretical results (Eqs. 6 and 9)which predict the steady-state Hamming distance between states

evolved from two nearby initial conditions and the stability of agiven network. These results are derived using the hypothesis thata theory derived in the semiannealed case approximates the truesituation, where by semiannealed we mean that the network ofconnections is frozen, but the truth table at each node is randomlyreassigned at each time step. For large networks, this approxi-mation was found to give excellent agreement with the true caseof frozen connections and frozen truth tables. Our semiannealedhypothesis does not rely on gross statistical properties of thenetwork, but instead uses the specific network topology, as charac-terized by the network adjacency matrix, and the individual nodesensitivities, to make predictions.

We tested our theoretical predictions with numerical experi-ments. Previously unaddressed issues that we considered includethe effects of assortativity, nonuniform time delay, nonuniformsensitivity, motifs, and community structure. In all cases tested wefound good agreement with our theory.

The theory that we have presented and tested above may rep-resent a step forward in facilitating the application of discretestate dynamical network models to biological systems. Given aspecific genetic interaction network and an estimate of the nodesensitivities, Eq. 9 predicts the stability of that particular net-work directly from the adjacency matrix. Curated networks alreadyexist in the literature for model single-cellular systems, and newalgorithms continue to be developed for inferring interaction net-works from a wide range of data sources (microarray experiments,genome sequencing, etc.). We note that such a procedure hasthe advantage that, because the actual experimentally determinednetwork is employed, topological aspects such as nodal in-/out-degree correlation, assortativity, community structure, etc., donot first have to be determined and then statistically modeled.Thus, by use of our stability criterion (Eq. 9), there is the potentialthat future analysis may be able to evaluate a supposed relation-ship between the stability characteristics of various networks andtheir functioning. For example, one might test whether cancergene networks are less stable than those in healthy tissue. Thiscould lead to the strong variations in gene expression observedin cancerous tissue (9), even when the underlying gene net-work is unchanged. We are currently pursuing research along thisline.

ACKNOWLEDGMENTS. We thank L. Staudt for discussions. The work wassupported by the National Science Foundation (Physics) and by Office ofNaval Research Contract N000014-1-0734. A. P. was supported in part by theNational Cancer Institute intramural program.

1. Kauffman S (1969) Metabolic stability and epigenesis in randomly constructed geneticnets. J Theor Biol 22:437–467.

2. Kauffman S (1993) The Origins of Order (Oxford Univ Press, New York).3. Albert R, Othmer HG (2003) The topology of the regulatory interactions predicts the

expression pattern of the segment polarity genes in Drosophila melanogaster. J TheorBiol 223:1–18.

4. Coleman W, Tsongalis G (2001) Molecular Basis of Human Cancer (Humana Press,Clifton, NJ).

5. Ott E (2002) Chaos in Dynamical Systems (Cambridge Univ Press, New York), 2nd Ed.6. Gonzalez-Garcia I, Sole RV, Costa J (2002) Metapopulation dynamics and spatial

heterogeneity in cancer. Proc Natl Acad Sci USA 99:13085–13089.7. Bignolda L (2007) Variation, ‘evolution’, immortality and genetic instabilities in

tumour cells. Cancer Lett 253:155–169.8. Rämö P, Kesselia J, Yli-Harjaa O (2006) Perturbation avalanches and criticality in gene

regulatory networks. J Theor Biol 242:164–170.9. Serra R, Villani M, Semeria A (2004) Genetic network models and statistical

properties of gene expression data in knock-out experiments. J Theor Biol 227:149–157.

10. Nykter M, et al. (2003) Gene expression dynamics in the macrophage exhibit criticality.Proc Natl Acad Sci USA 105:1897–1900.

11. Balleza A, et al. (2008) Critical dynamics in genetic regulatory networks: Examplesfrom four kingdoms. PLoS ONE 3(6):e2456.

12. Derrida B, Pomeau Y (1986) Random networks of automata: A simple annealedapproximation. Europhys Lett 1:45–49.

13. Ricard V. Solé, RV, Luque B (1995) Phase transitions and antichaos in generalizedKauffman networks. Phys Lett A 196:331–334.

14. Luque B, Solé, RV (1997) Phase transitions in random networks: Simple analyticdetermination of critical points. Phys Rev E 55:257–260.

15. Fox JJ, Hill CC (2001) From topology to dynamics in biochemical networks. Chaos11:809–815.

16. Aldana M, Cluzel P (2003) A natural class of robust networks. Proc Natl Acad Sci USA100:8710–8714.

17. Barabasi AL, Albert R (1999) Emergence of scaling in random networks. Science286:509–512.

18. Lee DS, Rieger H (2008) Broad edge of chaos in strongly heterogeneous Booleannetworks. J Phys A 41:415001.

19. Newman MEJ (2002) Assortative mixing in networks. Phys Rev Lett 89:208701.20. Newman MEJ, Girvan M (2004) Finding and evaluating community structure in

networks. Phys Rev E 69:026113.21. Cui Q, et al. (2007) A map of human cancer signaling. Mol Sys Biol 3:152.22. Maslov S, Sneppen K (2002) Specificity and stability in topology of protein networks.

Science 269:910–913.23. Eisenberg E, Levanon EY (2003) Human housekeeping genes are compact. Trends

Genet 19:362–365.24. Ghil M, Mullhaupt A (1985) Boolean delay equations II: Periodic and aperiodic

solutions. J Stat Phys 41:127–173.25. Kappler K, Edwards R, Glass L (2003) Dynamics in high-dimensional model gene

networks. Signal Process 83:789–798.26. Luque B, Ballesteros FJ (2004) Random walk networks. Physica A 342:207–213.27. Shmulevich I, Kauffman SA (2004) Activities and sensitivities in boolean network

models. Phys Rev Lett 93:048701.28. Restrepo JG, Ott E, Hunt BR (2008) Weighted percolation on directed networks. Phys

Rev Lett 100:058701.29. Cohen R, Erez K, ben-Avraham D, Havlin S (2000) Resilience of the Internet to random

breakdowns. Phys Rev Lett 85:4626–4630.30. Milo R, et al. (2002) Network motifs: Simple building blocks of complex networks.

Science 298:824–827.31. MacCluer CR (2000) The many proofs and applications of Perron’s theorem. SIAM Rev

42:487–498.32. Restrepo JG, Ott E, Hunt BR (2007) Approximating the largest eigenvalue of network

adjacency matrices. Phys Rev E 76:056119.33. Farkas I, Jeong H, Vicsek T, Barabási AL, Oltvai ZN (2003) The topology of the

transcription regulatory network in the yeast, Saccharomyces cerevisiae. Physica A318:601–612.

34. Newman MEJ (2003) The structure and function of complex networks. SIAM Rev45:167.

35. Luscombe NM, et al. (2004) Genomic analysis of regulatory network dynamics revealslarge topological changes. Nature 431:308–312.

8214 www.pnas.org / cgi / doi / 10.1073 / pnas.0900142106 Pomerance et al.

Dow

nloa

ded

by g

uest

on

Aug

ust 2

3, 2

020