Physica A 389 (2010) 4265–4298

Contents lists available at ScienceDirect

Physica A

journal homepage: www.elsevier.com/locate/physa

Evolutionary game theory: Theoretical concepts and applications tomicrobial communitiesErwin Frey ∗Ludwig-Maximilians-Universität München, Arnold Sommerfeld Center for Theoretical Physics and Center for NanoScience, Theresienstrasse 37,D-80333 Munich, Germany

a r t i c l e i n f o

Article history:Received 30 January 2010Available online 4 March 2010

Keywords:Population dynamicsGame theoryEvolutionStochastic processesNonlinear dynamicsPattern formation

a b s t r a c t

Ecological systems are complex assemblies of large numbers of individuals, interactingcompetitively under multifaceted environmental conditions. Recent studies usingmicrobial laboratory communities have revealed some of the self-organization principlesunderneath the complexity of these systems. Amajor role of the inherent stochasticity of itsdynamics and the spatial segregation of different interacting species into distinct patternshas thereby been established. It ensures the viability of microbial colonies by allowing forspecies diversity, cooperative behavior and other kinds of ‘‘social’’ behavior.A synthesis of evolutionary game theory, nonlinear dynamics, and the theory of

stochastic processes provides the mathematical tools and a conceptual framework for adeeper understanding of these ecological systems.Wegive an introduction into themodernformulation of these theories and illustrate their effectiveness focussing on selectedexamples of microbial systems. Intrinsic fluctuations, stemming from the discretenessof individuals, are ubiquitous, and can have an important impact on the stability ofecosystems. In the absence of speciation, extinction of species is unavoidable. It may,however, take very long times. We provide a general concept for defining survival andextinction on ecological time-scales. Spatial degrees of freedom come with a certainmobility of individuals.When the latter is sufficiently high, bacterial community structurescan be understood through mapping individual-based models, in a continuum approach,onto stochastic partial differential equations. These allow progress using methods ofnonlinear dynamics such as bifurcation analysis and invariantmanifolds.We concludewitha perspective on the current challenges in quantifying bacterial pattern formation, and howthis might have an impact on fundamental research in non-equilibrium physics.

© 2010 Elsevier B.V. All rights reserved.

1. Introduction

This article is intended as an introduction into the concepts and the mathematical framework of evolutionary gametheory, seen with the eyes of a theoretical physicist. It grew out of a series of lectures on this topic given at various summerschools in the year 2009.1 Therefore, it is not meant to be a complete review of the field but rather a personal, and hopefullypedagogical, collection of ideas and concepts intended for an audience of advanced students.

∗ Tel.: +49 0 89 2180 4537.E-mail address: frey@lmu.de.

1 International Summer School Fundamental Problems in Statistical Physics XII, August 31–September 11, 2009, Leuven, Belgium; Boulder School forCondensed Matter and Materials Physics, ‘‘Nonequilibrium Statistical Mechanics: Fundamental Problems and Applications’’, July 6–31, 2009, Boulder,Colorado; WE-Heraeus Summerschool 2009, ‘‘Steps in Evolution: Perspectives from Physics, Biochemistry and Cell Biology 150 Years after Darwin’’,June 28–July 5, 2009, Jacobs University Bremen.

0378-4371/$ – see front matter© 2010 Elsevier B.V. All rights reserved.doi:10.1016/j.physa.2010.02.047

4266 E. Frey / Physica A 389 (2010) 4265–4298

The first chapter gives an introduction into the theory of games.Wewill start with ‘‘strategic games’’, mainly to introducesome basic terminology, and then quickly move to ‘‘evolutionary game theory’’. The latter is the natural framework for theevolutionary dynamics of populations consisting of multiple interacting species, where the success of a given individualdepends on the behavior of the surrounding ones. It is most naturally formulated in the language of nonlinear dynamics,where the game theory terms ‘‘Nash equilibrium’’ or ‘‘evolutionary stable strategy’’ map onto ‘‘fixed points’’ of ordinarynonlinear differential equations. Illustrations of these concepts are given in terms of two-strategy games and the cyclicLotka–Volterra model, also known as the ‘‘rock–paper–scissors’’ game. We conclude this first chapter by an (incomplete)overview of game-theoretical problems in biology, mainly taken from the field of microbiology.A deterministic description of populations of interacting individuals in terms of nonlinear differential equations,

however, misses some important features of actual ecological systems. The molecular processes underlying the interactionbetween individuals are often inherently stochastic and the number of individuals is always discrete. As a consequence,there are random fluctuations in the composition of the population which can have an important impact on the stability ofecosystems. In the absence of speciation, extinction of species is unavoidable. It may, however, take very long times. Oursecond chapter starts with some elementary, but very important, notes on extinction times. These ideas are then illustratedfor two-strategy games, whose stochastic dynamics is still amenable to an exact solution. Finally, we provide a generalconcept for defining survival and extinction on an ecological time scale which should be generally applicable.Section 3, introducing the May–Leonard model, serves two purposes. On one hand, it shows how a two-step cyclic

competition, split into a selection and reproduction step, modifies the nonlinear dynamics from neutral orbits, as observedin the cyclic Lotka–Volterra model, to an unstable spiral. This teaches that the details of the molecular interactions betweenindividuals may matter and periodic orbits are, in general, non-generic in population dynamics. On the other hand, theMay–Leonard model serves to introduce somemore advanced concepts from non-linear dynamics: linear stability analysis,invariant manifolds, and normal forms. In addition, the analysis will also serve as a necessary prerequisite for the analysisof spatial games in the subsequent chapter. Section 3 is mainly technical and may be skipped in a first reading.Cyclic competition of species, as metaphorically described by the children’s game ‘‘rock–paper–scissors’’, is an intriguing

motif of species interactions. Laboratory experiments on populations consisting of different bacterial strains of Escherichiacoli have shown that bacteria can coexist if a low mobility enables the segregation of the different strains and thereby theformation of patterns [1]. In Section 4 we analyze the impact of stochasticity and an individual’s mobility on the stability ofdiversity as well as the emerging patterns. Within a spatially-extended version of the May–Leonard model we demonstratethe existence of a sharp mobility threshold, such that diversity is maintained below, but jeopardized above that value.Computer simulations of the ensuing stochastic cellular automaton show that entangled rotating spiral waves accompanybiodiversity. These findings are rationalized using stochastic partial differential equations (SPDE), which are reduced to acomplex Ginzburg–Landau equation (CGLE) upon mapping the SPDE onto the reactive manifold of the nonlinear dynamics.In Section 5, we conclude with a perspective on the current challenges in quantifying bacterial pattern formation and

how this might have an impact on fundamental research in non-equilibrium physics.

1.1. Strategic games

Classical Game Theory [2] describes the behavior of rational players. It attempts to mathematically capture behaviors instrategic situations, in which an individual’s success inmaking choices depends on the choices of others. A classical exampleof a strategic game is the prisoner’s dilemma. In its classical form, it is presented as follows2:‘‘Two suspects of a crime are arrested by the police. The police have insufficient evidence for a conviction, and, having

separated both prisoners, visit each of them to offer the same deal. If one testifies (defects from the other) for the prosecutionagainst the other and the other remains silent (cooperates with the other), the betrayer goes free and the silent accomplicereceives the full 10-year sentence. If both remain silent, both prisoners are sentenced to only 1 year in jail for a minorcharge. If each betrays the other, each receives a five-year sentence. Each prisoner must choose to betray the other or toremain silent. Each one is assured that the other would not know about the betrayal before the end of the investigation.How should the prisoners act?’’The situation is best illustrated in what is called a ‘‘payoff matrix’’ which in the classical formulation is rather a ‘‘cost

matrix’’:P Cooperator (C) Defector (D)C 1 year 10 yearsD 0 years 5 years

Here rows and columns correspond to player (suspect) 1 and 2, respectively. The entries give the prison sentence for player1; this is sufficient information since the game is symmetric. Imagine you are player 1, and that player 2 is playing thestrategy ‘‘cooperate’’. Then you are obviously better off to play ‘‘defect’’ since you can get free. Now imagine player 2 isplaying ‘‘defect’’. Then you are still better off to defect since 5 years in prison is better than 10 years in prison. If both playersare rational players a dilemma arises since both will analyze the situation in the same way and come to the conclusion that

2 This description is taken fromWikipedia: http://en.wikipedia.org/wiki/Prisoner’s_dilemma.

E. Frey / Physica A 389 (2010) 4265–4298 4267

it is always better to play ‘‘defect’’ irrespective of what the other suspect is playing. This outcome of the game, both playing‘‘defect’’, is called a Nash equilibrium [3]. The hallmark of a Nash equilibrium is that none of the players has an advantage ofdeviating from his strategy unilaterally. This rational choice, where each player maximizes his own payoff, is not the bestoutcome! If both defect, they will both be sentenced to prison for 5 years. Each player’s individual reward would be greaterif they both played cooperatively; they would both only be sentenced to prison for 1 year.We can also reformulate the prisoner’s dilemma game as a kind of a public good game. A cooperator provides a benefit b to

another individual, at a cost c to itself (with b− c > 0). In contrast, a defector refuses to provide any benefit and hence doesnot pay any costs. For the selfish individual, irrespective of whether the partner cooperates or defects, defection is favorable,as it avoids the cost of cooperation, exploits cooperators, and ensures not to become exploited. However, if all individualsact rationally and defect, everybody is, with a gain of 0, worse off compared to universal cooperation, where a net gain ofb− c > 0 would be achieved. The prisoner’s dilemma therefore describes, in its most basic form, the fundamental problemof establishing cooperation.

P Cooperator (C) Defector (D)C b− c −cD b 0

This scheme can be generalized to include other basic types of social dilemmas [4,5]. Namely, two cooperators that meetare both rewarded with a payoffR, while two defectors obtain a punishment P . When a defector encounters a cooperator,the first exploits the second, gaining the temptation T , while the cooperator only gets the sucker’s payoff S. Social dilemmasoccur when R > P , such that cooperation is favorable in principle, while temptation to defect is large: T > S, T > P .These interactions may be summarized by the payoff matrix:

P Cooperator (C) Defector (D)C R SD T P

Variation of the parameters T ,P ,R andS yields four principally different types of games. The prisoner’s dilemma arises if thetemptation T to defect is larger than the rewardR, and if the punishmentP is larger than the suckers payoff S. As we havealready seen above, in this case, defection is the best strategy for the selfish player. Within the three other types of games,defectors are not always better off. For the snowdrift game the temptation T is still higher than the rewardR but the sucker’spayoff S is larger than the punishment P . Therefore, now actually cooperation is favorable when meeting a defector, butdefection pays offwhen encountering a cooperator, and a rational strategy consists of amixture of cooperation anddefection.The snowdrift game derives its name from the potentially cooperative interaction present when two drivers are trappedbehind a large pile of snow, and each driver must decide whether to clear a path. Obviously, then the optimal strategy is theopposite of the opponent’s (cooperate when your opponent defects and defect when your opponent cooperates). Anotherscenario is the coordination game, where mutual agreement is preferred: either all individuals cooperate or defect as therewardR is higher than the temptation T and the punishment P is higher than sucker’s payoff S. Finally, the scenario ofby-product mutualism (also called harmony) yields cooperators fully dominating defectors since the rewardR is higher thanthe temptation T and the sucker’s payoff S is higher than the punishment P .

1.2. Evolutionary game theory

Strategic games, as discussed in the previous section, are thought to be a useful concept in economic and social settings.In order to analyze the behavior of biological systems, the concept of rationality is not meaningful. Evolutionary GameTheory (EGT) as developed mainly by Maynard Smith and Price [6,7] does not rely on rationality assumptions but on theidea that evolutionary forces like natural selection andmutation are the driving forces of change. The interpretation of gamemodels in biology is fundamentally different from strategic games in economics or social sciences. In biology, strategies areconsidered to be inherited programs which control the individual’s behavior. Typically one looks at a population composedof individuals with different strategies who interact generation after generation in game situations of the same type. Theinteractions may be described by deterministic rules or stochastic processes, depending on the particular system understudy. The ensuing dynamic process can then be viewed as an iterative (nonlinear) map or a stochastic process (eitherwith discrete or continuous time). This naturally puts evolutionary game theory in the context of nonlinear dynamics andthe theory of stochastic processes. We will see how a synthesis of both approaches helps to understand the emergence ofcomplex spatio-temporal dynamics.In this section, we focus on a deterministic description of well-mixed populations. The term ‘‘well-mixed’’ signifies systems

where the individual’s mobility (or diffusion) is so large that one may neglect any spatial degrees of freedom and assumethat every individual is interacting with everyone at the same time. This is a mean-field picture where the interactions aregiven in terms of the average number of individuals playing a particular strategy. Frequently, this situation is visualized asan ‘‘urnmodel’’, where two (ormore) individuals from a population are randomly selected to playwith each other accordingto some specified game theoretical scheme. The term ‘‘deterministic’’ means that we are seeking a description of populationswhere the number of individuals Ni(t) playing a particular strategy Ai is macroscopically large such that stochastic effectscan be neglected.

4268 E. Frey / Physica A 389 (2010) 4265–4298

Fig. 1. The urn model describes the evolution of well-mixed finite populations. Here, as an example, we show three species as yellow (A), red (B), andblue (C) spheres. At each time step, two randomly selected individuals are chosen (indicated by arrows in the left picture) and interact with each otheraccording to the rules of the game resulting in an updated composition of the population (right picture). (For interpretation of the references to colour inthis figure legend, the reader is referred to the web version of this article.)

Fig. 2. Illustration of cyclic dominance of three states A, B, and C: A invades B, B outperforms C , and C in turn dominates over A.

Pairwise reactions and rate equationsIn the simplest setup the interaction between individuals playing different strategies can be represented as a reaction

process characterized by some set of rate constants. For example, consider a gamewhere three strategies A, B, C cyclicallydominate each other, as in the famous rock–paper–scissors game: A invades B, B outperforms C , and C in turn dominatesover A, schematically drawn in Fig. 2:In an evolutionary setting, the game may be played according to an urn model as illustrated in Fig. 1: at a given time t

two individuals from a population with constant size N are randomly selected to play with each other (react) according tothe scheme

A+ BkA−→ A+ A,

B+ CkB−→ B+ B, (1)

C + AkC−→ C + C,

where ki are rate constants, i.e. probabilities per unit time. This interaction scheme is termed a cyclic Lotka–Volterra model.3It is equivalent to a set of chemical reactions, and in the deterministic limit of a well-mixed population one obtains rateequations for the frequencies (a, b, c) = (NA,NB,NC )/N:

∂ta = a(kAb− kCc),∂tb = b(kBc − kAa), (2)∂tc = c(kCa− kBb).

Here the right hand sides gives the balance of ‘‘gain’’ and ‘‘loss’’ processes. The phase space of the model is the simplex S3,where the species’ densities are constrained by a+ b+ c = 1. There is a constant of motion for the rate equations, Eq. (2),namely the quantity ρ := akBbkC ckA does not evolve in time [10]. As a consequence, the phase portrait of the dynamics,shown in Fig. 3, yields cycles around the reactive fixed point.

The concept of fitness and replicator equationsAnother route of setting up an evolutionary dynamics, often taken in the mathematical literature of evolutionary game

theory [11,10], introduces the concept of fitness and then assumes that the per-capita growth rate of a strategy Ai is given

3 The two-species Lotka–Volterra equations describe a predator–prey system where the per-capita growth rate of prey decreases linearly with theamount of predators present. In the absence of prey, predators die, but there is a positive contribution to their growth which increases linearly with theamount of prey present [8,9].

E. Frey / Physica A 389 (2010) 4265–4298 4269

Fig. 3. The three-species simplex for reaction rates kA = 1, kB = 1.5, kC = 2. Since there is a conserved quantity, the rate equations predict cyclic orbits.

by the surplus in its fitness with respect to the average fitness of the population. We will illustrate this reasoning for two-strategy games with a payoff matrix P:

P A BA R SB T P

(3)

Let NA and NB be the number of individuals playing strategy A and B in a population of size N = NA + NB. Then the relativeabundances of strategies A and B are given by

a =NAN, b =

NBN= (1− a). (4)

The ‘‘fitness’’ of a particular strategy A or B is defined as a constant background fitness, set to 1, plus the average payoffobtained from playing the game:

fA(a) := 1+Ra+ S(1− a), (5)fB(a) := 1+ T a+ P (1− a). (6)

In order tomimic an evolutionary process one is seeking a dynamics which guarantees that individuals using strategies witha fitness larger than the average fitness increasewhile those using strategieswith a fitness below average decline in number.This is, for example, achieved by choosing the per-capita growth rate, ∂ta/a, of individuals playing strategy A proportionalto their surplus in fitness with respect to the average fitness of the population,

f (a) := afA(a)+ (1− a)fB(a). (7)

The ensuing ordinary differential equation is known as the standard replicator equation [11,10]

∂ta =[fA(a)− f (a)

]a. (8)

Lacking a detailed knowledge of the actual ‘‘interactions’’ of individuals in a population, there is, of course, plenty of freedomin how to write down a differential equation describing the evolutionary dynamics of a population. Indeed, there is anotherset of equations frequently used in EGT, called adjusted replicator equations, which reads

∂ta =fA(a)− f (a)f (a)

a. (9)

Here we will not bother to argue why one or the other is a better description. As we will see later, these equations emergequite naturally from a full stochastic description in the limit of large populations.

1.3. Nonlinear dynamics of two-player games

This section is intended to give a concise introduction into elementary concepts of nonlinear dynamics [12].We illustratethose for the evolutionary dynamics of two-player games characterized in terms of the payoffmatrix, Eq. (3), and the ensuingreplicator dynamics

∂ta = a(fA − f ) = a(1− a)(fA − fB). (10)

4270 E. Frey / Physica A 389 (2010) 4265–4298

Table 1Classification of two player games according to their payoff matrices and stable fixed point values of the replicator equation.

Game Control parameters Stable fixed points

Prisoner’s dilemma µA < 0; µB > 0 0Snowdrift µA > 0; µB > 0 µA/(µA + µB)

Coordination µA < 0; µB < 0 0, 1Harmony µA > 0; µB < 0 1

Fig. 4. Classification of two-player games. Left: the black arrows in the control parameter plane (µA, µB) indicate the flow behavior of the four differenttypes of two-player games. Right: graphically the solution of a one-dimensional nonlinear dynamics equation, ∂ta = F(a), is simply read off from the signsof the function F(a); illustration for the snowdrift game.

This equation has a simple interpretation: the first factor, a(1 − a), is the probability for A and B to meet and the secondfactor, fA − fB, is the fitness advantage of A over B. Inserting the explicit expressions for the fitness values one finds

∂ta = a(1− a)[µA(1− a)− µBa

]=: F(a), (11)

where µA is the relative benefit of A playing against B and µB is the relative benefit of B playing against A:µA := S − P , µB := T −R. (12)

Hence, as far as the replicator dynamics is concerned, we may replace the payoff matrix byP A BA 0 µAB µB 0

Eq. (11) is a one-dimensional nonlinear first-order differential equation for the fraction a of players A in the population,whose dynamics is most easily analyzed graphically. The sign of F(a) determines the increase or decrease of the dynamicvariable a; compare the right half of Fig. 4. The intersections of F(a) with the a-axis (zeros) are fixed points, a∗. Generically,these intersections are with a finite slope F ′(a∗) 6= 0; a negative slope indicates a stable fixed point while a positive slopean unstable fixed point. Depending on some control parameters, here µA and µB, the first or higher order derivatives of Fat the fixed points may vanish. These special parameter values mark ‘‘threshold values’’ for changes in the flow behavior(bifurcations) of the nonlinear dynamics. We may now classify two-player games as summarized in Table 1.For the prisoner’s dilemmaµA = −c < 0 andµB = c > 0 and hence playerswith strategy B (defectors) are always better

off (compare the payoff matrix). Both players playing strategy B is a Nash equilibrium. In terms of the replicator equationsthis situation corresponds to F(a) < 0 for a 6= 0 and F(a) = 0 at a = 0, 1 such that a∗ = 0 is the only stable fixed point.Hence the term ‘‘Nash equilibrium’’ translates into the ‘‘stable fixed point’’ of the replicator dynamics (nonlinear dynamics).For the snowdrift game bothµA > 0 andµB > 0 such that F(a) can change sign for a ∈ [0, 1]. In fact, a∗int = µA/(µA+µB)

is a stable fixed point while a∗ = 0, 1 are unstable fixed points; see the right panel of Fig. 4. Inspection of the payoffmatrix tells us that it is always better to play the opposite strategy of your opponent. Hence there is no Nash equilibriumin terms of the pure strategies A or B. This corresponds to the fact that the boundary fixed points a∗ = 0, 1 are unstable.There is, however, a Nash equilibrium with amixed strategywhere a rational player would play strategy Awith probabilitypA = µA/(µA + µB) and strategy Bwith probability pB = 1− pA. Hence, again, the term ‘‘Nash equilibrium’’ translates intothe ‘‘stable fixed point’’ of the replicator dynamics.For the coordination game, there is also an interior fixed point at a∗int = µA/(µA + µB), but now it is unstable, while the

fixed points at the boundaries a∗ = 0, 1 are stable. Hence we have bistability: for initial values a < a∗int the flow is towardsa = 0 while it is towards a = 1 otherwise. In the terminology of strategic games there are two Nash equilibria. The gameharmony corresponds to the prisoner’s dilemma with the roles of A and B interchanged.

1.4. Bacterial games

Bacteria often grow in complex, dynamical communities, pervading the earth’s ecological systems, from hot springsto rivers and the human body [13]. As an example, in the latter case, they can cause a number of infectious diseases,

E. Frey / Physica A 389 (2010) 4265–4298 4271

such as lung infection by Pseudomonas aeruginosa. Bacterial communities, quite generically, form biofilms [13,14], i.e., theyarrange into a quasi-multi-cellular entity where they interact highly. These interactions include competition for nutrients,cooperation by providing various kinds of public goods essential for the formation and maintenance of the biofilm [15],communication through the secretion and detection of extracellular substances [16,17], and last but not least physical forces.The ensuing complexity of bacterial communities has conveyed the idea that they constitute a kind of ‘‘social groups’’ wherethe coordinated action of individuals leads to various kinds of system-level functionalities. Since additionally microbialinteractions can be manipulated in a multitude of ways, many researchers have turned to microbes as the organisms ofchoice to explore fundamental problems in ecology and evolutionary dynamics.Two of the most fundamental questions that challenge our understanding of evolution and ecology are the origin

of cooperation [18–20,16,17,21–23] and biodiversity [24–26,1,27]. Both are ubiquitous phenomena yet are conspicuouslydifficult to explain since the fitness of an individual or the whole community depends in an intricate way on a plethoraof factors, such as spatial distribution and mobility of individuals, secretion and detection of signaling molecules, toxinsecretion leading to inter-strain competition and changes in environmental conditions. It is fair to say that we are still a longway off from a full understanding, but the versatility of microbial communities makes their study a worthwhile endeavorwith exciting discoveries still ahead of us.

CooperationUnderstanding the conditions that promote the emergence and maintenance of cooperation is a classic problem in

evolutionary biology [28,29,7]. It can be stated in the language of the prisoner’s dilemma. By providing a public good,cooperative behavior would be beneficial for all individuals in the whole population. However, since cooperation is costly,the population is at risk from invasion by ‘‘selfish’’ individuals (cheaters), who save the cost of cooperation but can still obtainthe benefit of cooperation from others. In evolutionary theory many principles were proposed to overcome this dilemma ofcooperation: repeated interaction [20,28], punishment [20,30], or kin discrimination [31,32]. All of these principles share onefundamental feature: They are based on some kind of selectionmechanism. Similar to the old debate between ‘‘selectionists’’and ‘‘neutralists’’ in evolutionary theory [33], there is an alternative. Due to random fluctuations, a population initiallycomposed of both cooperators and defectors may (with some probability) become fixed in a state of cooperators only [34].We will come back to this point later in Section 2.2.There has been an increasing number of experiments using microorganisms to shed new light on the problem of

cooperation [18,19,22,23]. Here, we will shortly discuss a recent experiment on ‘‘cheating in yeast’’ [22]. Budding yeastprefers to use the monosaccharides glucose and fructose as carbon sources. If they have to grow on sucrose instead, thedisaccharide must first be hydrolyzed by the enzyme invertase. Since a fraction of approximately 1 − ε = 99% of theproduced monosaccharides diffuses away and is shared with neighboring cells, it constitutes a public good available to thewhole microbial community. This makes the population susceptible to invasion by mutant strains that save the metaboliccost of producing invertase. One is now tempted to conclude from what we have discussed in the previous sections thatyeast is playing the prisoner’s dilemma game. The cheater strains should take over the population and the wild type strainshould become extinct. But, this is not the case. Gore and collaborators [22] show that the dynamics is rather described bya snowdrift game, in which cheating can be profitable, but is not necessarily the best strategy if others are cheating too.The explanation given is that the growth rate as a function of glucose is highly concave and, as a consequence, the fitnessfunction is non-linear in the payoffs4

fC (x) :=[ε + x(1− ε)

]α− c, (13)

fD(a) :=[x(1− ε)

]α, (14)

with α ≈ 0.15 determined experimentally. The ensuing phase diagram, Fig. 5, as a function of capture efficiency ε andmetabolic cost c shows an altered intermediate regime with a bistable phase portrait, i.e. the hallmark of a snowdriftgame as discussed in the previous section. This explains the experimental observations. The lesson to be learned fromthis investigation is that defining a payoff function is not a trivial matter, and a naive replicator dynamics fails to describebiological reality. It is, in general, necessary to have a detailed look on the nature of the biochemical processes responsiblefor the growth rates of the competing microbes.

Pattern formationInvestigations of microbial pattern formation have often focussed on one bacterial strain [35–37]. In this respect, it

has been found that bacterial colonies on substrates with a high nutrient level and intermediate agar concentrations,representing ‘‘friendly’’ conditions, grow in simple compact patterns [38]. When instead the level of nutrient is lowered,when the surface on which bacteria grow possesses heterogeneities, or when the bacteria are exposed to antibiotics,complex, fractal patterns are observed [35,39,40]. Other factors that affect the self-organizing patterns includemotility [41],

4 Note that ε is the fraction of the carbon source kept by cooperators solely for themselves and x(1− ε) is the amount of the carbon source shared withthe whole community. Hence, the linear growth rate of cooperators and defectors would be ε + x(1 − ε) − c and x(1 − ε), respectively, where c is themetabolic cost for invertase production.

4272 E. Frey / Physica A 389 (2010) 4265–4298

a b

Fig. 5. Game theorymodels of cooperation in sucrosemetabolism of yeast. (a) Phase diagram resulting from fitness functions fC and fD linear in the payoffs.This model leads to a fixation of cooperators (x = 1) at a low cost and/or a high efficiency of capture (ε > c , implying that the game is mutually beneficial(MB)) but leads to a fixation of defectors (x = 0) for a high cost and/or a low efficiency of capture (ε < c , implying that the game is the prisoners dilemma(PD)). (b) A model of cooperation with experimentally measured concave benefits yields a central region of parameter space that is a snowdrift game (SG),thus explaining the coexistence that is observed experimentally (α = 0.15).Source: Adapted from Ref. [22].

the kind of bacterial movement, e.g., swimming [42], swarming, or gliding [43,44], as well as chemotaxis and externalheterogeneities [45]. Another line of research has investigated patterns of multiple co-evolving bacterial strains. As anexample, recent studies looked at growth patterns of two functionally equivalent strains of E. coli and showed that, dueto fluctuations alone, they segregate into well-defined, sector-like regions [36,46].

The Escherichia Col E2 systemSeveral Colibacteria such as E. coli are able to produce and secrete specific toxins called Colicines that inhibit the growth

of other bacteria. Kerr and coworkers [1] have studied three strains of E. coli, amongst which one is able to produce the toxinCol E2 that acts as an DNA endonuclease. This poison producing strain (C) kills a sensitive strain (S), which outgrows thethird, resistant one (R), as resistance bears certain costs. The resistant bacteria grow faster than the poisonous ones, as thelatter are resistant and produce a poison, which is yet an extra cost. Consequently, the three strains of E. coli display a cycliccompetition, similar to the children’s game rock–paper–scissors.When placed on a Petri-dish, all three strains coexist, arranging in time-dependent spatial clusters dominated by one

strain. In Fig. 6, snapshots of these patterns monitored over several days are shown. Sharp boundaries between differentdomains emerge, and all three strains co-evolve at comparable densities. The patterns are dynamic: Due to the non-equilibrium character of the species’ interactions, clusters dominated by one bacterial strain cyclically invade each other,resulting in an endless hunt of the three species on the Petri-dish. The situation changes considerably when putting thebacteria in a flask with additional stirring. Then, only the resistant strain survives, while the two others die out after a shorttransient time.These laboratory experiments thus provide intriguing experimental evidence for the importance of spatial patterns

for the maintenance of biodiversity. In this respect, many further questions regarding the spatio-temporal interactionsof competing organisms under different environmental conditions lie ahead. Spontaneous mutagenesis of single cells canlead to enhanced fitness under specific environmental conditions or due to interactions with other species. Moreover,interactions with other species may allow unfit, but potentially pathogenic bacteria to colonize certain tissues. Additionally,high concentrations of harmless bacteria may help pathogenic ones to nest on tissues exposed to extremely unfriendlyconditions. Information about bacterial pattern formation arising from bacterial interaction may therefore allow one todevelop mechanisms to avoid pathogenic infection.

2. Stochastic dynamics in well-mixed populations

The machinery of biological cells consists of networks of molecules interacting with each other in a highly complexmanner.Many of these interactions can be described as chemical reactions,where the intricate processeswhich occur duringthe encounter of two molecules are reduced to reaction rates, i.e. probabilities per unit time. This notion of stochasticitycarries over to the scale of microbes in a manifold way. First, there is phenotypic noise. Due to fluctuations in transcriptionand translation, phenotypes vary even in the absence of genetic differences between individuals and despite constantenvironmental conditions [47,48]. In addition, phenotypic variability may arise due to various external factors like celldensity, nutrient availability and other stress conditions. A general discussion of phenotypic variability in bacteria maybe found in recent reviews [49–52]. Then, there is interaction noise. Interactions between individuals in a given population,as well as cell division and cell death, occur at random points in time (following some probability distribution) and lead todiscrete steps in the number of the different species. Then, as noted long ago by Delbrück [53], a deterministic description,

E. Frey / Physica A 389 (2010) 4265–4298 4273

Fig. 6. The three strains of the Escherichia Col E2 system evolve into spatial patterns on a Petri-dish. The competition of the three strains is cyclic (ofrock–paper–scissors type) and therefore non-equilibrium in nature, leading to dynamic patterns.Source: The picture has been modified from Ref. [1].

as discussed in the previous section, breaks down for small copy numbers. Finally, there is external noise due to spatialheterogeneities or temporal fluctuations in the environment.In this section we will focus on interaction noise, whose role for extinction processes in ecology has recently been

recognized to be very important, especially when the deterministic dynamics exhibits neutral stability [54–56] or weakstability [57,34]. After a brief and elementary discussion of extinction times wewill analyze the stochastic dynamics of two-strategy games [34] and the cyclic Lotka–Volterra model [55]. We conclude with a general concept for defining survival andextinction on ecological time-scales which should be generally applicable.

2.1. Some elementary notes on extinction times

The analysis in the previous section is fully deterministic. Given an initial condition, the outcome of the evolutionarydynamics is certain. However, processes encountered in biological systems are often stochastic. For example, consider thedegradation of a protein or the death of an individual bacterium in a population. To a good approximation it can be describedas a stochastic event which occurs at a probability per unit time (rate) λ, known as a stochastic linear death process. Then thepopulation size N(t) at time t becomes a random variable, and its time evolution becomes a set of integers Nα changingfrom Nα to Nα − 1 at particular times tα; this is also called a realization of the stochastic process. Now it is no longermeaningful to ask for the time evolution of a particular population, as one would do in a deterministic description in termsof a rate equation, N = −λN . Instead one studies the time evolution of an ensemble of systems or tries to understand thedistribution of times tα. A central quantity in this endeavor is the probability P(N, t) to find a population of size N giventhat at some time t = 0 there was some initial ensemble of populations. Assuming that the stochastic process is Markovian,its dynamics is given by the following master equation:

∂tP(N, t) = λ(N + 1)P(N + 1, t)− λNP(N, t). (15)

A master equation is a ‘‘balance equation’’ for probabilities. The right hand side simply states that there is an increase inP(N, t) if in a population of size N + 1 an individual dies with a rate of λ, and a decrease in P(N, t) if in a population of sizeN an individual dies with a rate of λ.5Master equations can be analyzed by standard tools from the theory of stochastic processes. In particular, they can (for

some elementary cases) be solved exactly using generating functions. In this section we are only interested in the averageextinction time T , i.e. the expected time for the population to reach the state N = 0. This state is also called an absorbingstate since (for the linear death process considered here) there are only processes leading into but not out of this state. Theexpected extinction time T can be obtained rather easily by considering the probability Q (t) that a given individual is stillalive at time t given that it was alive at time t = 0. We immediately obtain

Q (t + dt) = Q (t)(1− λt) with Q (0) = 1 (16)

5 For an elementary introduction into the theory of stochastic processes see the textbooks by van Kampen [58] and Gardiner [59].

4274 E. Frey / Physica A 389 (2010) 4265–4298

since an individual will be alive at time t + dt if it was alive at time t and did not die within the time interval [t, t + dt].The ensuing differential equation (in the limit dt → 0), Q = −λQ is solved by Q (t) = e−λt . This identifies τ = 1/λ as theexpected waiting time for a particular individual to die. We conclude that the waiting time for the population to change byone individual is distributed exponentially and its expected value is τN = τ/N for a population of size N; note that eachindividual in a population has the same chance to die. Hence we can write for the expected extinction time for a populationwith initial size N0

T = τN0 + τN0−1 + · · · + τ1 =N0∑N=1

τ

N≈ τ

∫ N0

1

1NdN = τ lnN0. (17)

We have learned that for a system with a ‘‘drift’’ towards the absorbing boundary of the state space the expected timeto reach this boundary scales, quite generically, logarithmically in the initial population size, T ∼ lnN0.6 Note thatwithin a deterministic description, N = −λN , the population size would exponentially decay to zero but never reach it,N(t) = N0e−t/τ . This is, of course, flawed in twoways. First, the process is not deterministic and, second, the population sizeis not a real number. Both features are essential to understand the actual dynamics of a population at low copy numbers ofindividuals.Now we would like to contrast the linear death process with a ‘‘neutral process’’ where death and birth events balance

each other, i.e. where the birth rate µ exactly equals the death rate λ. In a deterministic description one would write∂tN(t) = −(λ− µ)N(t) = 0 (18)

and conclude that the population size remains constant at its initial value. In a stochastic description, one starts from themaster equation

∂tP(N, t) = λ(N + 1)P(N + 1, t)+ λ(N − 1)P(N − 1, t)− 2λNP(N, t). (19)Though this could again be solved exactly using generating functions it is instructive to derive an approximation valid in thelimit of a large population size, i.e. N 1. This is most easily done by simply performing a second order Taylor expansionwithout worrying too much about the mathematical validity of such an expansion. With[

(N ± 1)P(N ± 1, t)]≈ NP(N, t)± ∂N

[NP(N, t)

]+12∂2N[NP(N, t)

](20)

we obtain

∂tP(N, t) = λ∂2N[NP(N, t)

]. (21)

Measuring the population size in units of the initial population size at time t = 0 and defining x = N/N0, this becomes

∂tP(x, t) = D∂2x[xP(x, t)

](22)

with the ‘‘diffusion constant’’ D = λ/N0.7 This implies that all time scales in the problem scale as t ∼ D−1 ∼ N0; this iseasily seen by introducing a dimensionless time τ = Dt resulting in a rescaled equation

∂τP(x, τ ) = ∂2x[xP(x, τ )

]. (23)

Hence for a (deterministically) ‘‘neutral dynamics’’ the extinction time, i.e. the time for reaching the absorbing state N = 0,scales, also quite generically, is linear in the initial system size T ∼ N0.Finally, there are processes like the snowdrift game where the deterministic dynamics drives the population towards

an interior fixed point well separated from the absorbing boundaries, x = 0 and x = 1. In such a case, starting from aninitial state in the vicinity of the interior fixed point, the stochastic dynamics has to overcome a finite barrier in order toreach the absorbing state. This is reminiscent of a chemical reaction with an activation barrier which is described by anArrhenius law. Hence we expect that the extinction time scales exponentially in the initial population size T ∼ eN0 . Thiswill be corroborated later by explicit calculations for the snowdrift game.To summarize, themean extinction time T can be used to classify evolutionary dynamics into a few fundamental regimes.

For systems with a deterministic drift towards the absorbing boundaries of states space, as frequently encountered innonlinear dynamic systems with unstable interior fixed points, typical extinction times are expected to scale as T ∼ lnN .We refer to such a system as an ‘‘unstable’’ system. If, in contrast, the deterministic dynamics is characterized by a stablefixed point with some domain of attraction, we expect extinction times to scale as T ∼ eN . We will refer to these systemsas ‘‘stable’’. The case of neutral dynamics yields T ∼ N and will be referred to as ‘‘neutral’’ (or marginally stable). There mayalso be intermediate scenarios with extinction times scaling as a power law in the population size, T ∼ Nγ . Transitionsbetween these regimes can occur and are manifest as crossovers in the functional relation T (N).

6 Naively, one may estimate the extinction time from the deterministic dynamics by asking for the time T where N(T ) = 1, and indeed this givesT = τ lnN0 .7 If instead of a linear birth–death process one would consider a symmetric random walk with hopping rate ε on a one-dimensional lattice with sitesxi = ia, the resulting equation would be a diffusion equation ∂tP(x, t) = D∂2x P(x, t) with diffusion constant D = εa2 . Restricting the random walk to afinite lattice with absorbing boundaries at x0 = 0 and xN = Na := 1 this would result in a diffusion constant scaling as the inverse square of the systemsize, D ∼ 1/N2 . In the linear birth–death process, the corresponding amplitude D, measuring the magnitude of stochastic effects, scales as D ∼ 1/N sincethe rates are proportional to the size of the system; each individual has the same probability of undergoing a reaction.

E. Frey / Physica A 389 (2010) 4265–4298 4275

Fig. 7. The discrete state space S2 for the two-strategy game. The stochastic process is a one-step process where transitions are possible only betweenneighboring states.

Fig. 8. The evolutionary dynamics consists of a fitness-dominated, directed part caused by selection of defectors (D) against cooperators (C), and a neutral,undirected part due to fluctuations. Eventually, after some characteristic time T , only one species survives and the population reaches a state of overalldefection or cooperation.Source: Adapted from Ref. [34].

2.2. Two-strategy games

In this section we will exemplify how to set up a stochastic dynamics in evolutionary game theory by studying two-strategy games. We restrict ourselves to standard urn models with a fixed population size N where the composition of thepopulation is updated by aMoran process [60]: At every time step an individual with a given strategy Ai is randomly selectedfor reproduction, with a probability proportional to its relative abundance in the population, Ni/N , and proportional to itsrelative fitness, fi/f . In order to keep the constant population size fixed this replacement happens at the expense of thealternative strategy, i.e. an individual with the alternative strategy is randomly selected and deleted from the population.8Taken together, these update rules amount to a one-step stochastic process in the number of playerswith one given strategy,say n := NA. Transitions are possible only between neighboring states. If a player with strategy Awins the game, the numberof players with this strategy increases by one, n→ n+ 1 (see Fig. 7), with a transition rate given by

w+n (x) = N x(1− x)fA(x)f (x)

, (24)

where x := n/N is defined as the relative abundance of strategy A. Here we have chosen units of time such that N games areplayed per unit time, or, in other words, each player plays one game on average per unit time; ∆t = 1/N . If a player withstrategy Bwins the game, we have n→ n− 1 with the corresponding transition rate

w−n (x) = N x(1− x)fB(x)f (x)

. (25)

The ensuing Master equation for this one-step stochastic process reads

∂tP(n, t) = w−n+1P(n+ 1, t)+ w+

n−1P(n− 1, t)−[w−n + w

+

n

]P(n, t). (26)

It is straightforward to show that the first moment obeys the equation

∂t〈n〉 = 〈w+n 〉 − 〈w−

n 〉 (27)

which upon neglecting correlations reduces to the deterministic (adjusted) replicator equation for the mean relativeabundance x = 〈n〉/N:

∂t x = x(1− x)fA(x)− fB(x)f (x)

. (28)

However, these rate equations miss some important features of the evolutionary dynamics, which is intrinsically stochastic.As illustrated in Fig. 8 for the prisoner’s dilemma, there is an interplay between deterministic drift and stochastic fluc-tuations. While selection-dominated dynamics drives the population to overall defection, random fluctuations induce anundirected randomwalk between complete defection and complete cooperation.When selection by fitness differences dom-inates the dynamics, the resulting outcome (most likely) equals the one of rational agents described in the previous chapter.

8 Note that such a Moran process is – by far – neither the most general nor the typical scenario for microbial populations. One should think of such adynamics as an effective and crude way to account for a system with limited resources where the one’s gain is the other’s loss.

4276 E. Frey / Physica A 389 (2010) 4265–4298

When selection is weak and stochastic fluctuations dominate, both overall defection and overall cooperation are possibleoutcomes. Note that upon reaching overall defection or cooperation, the temporal development comes to an end. Hence,even if defection is the dominant strategy, there is always a finite chance that a population reaches overall cooperation.In the following we are going to quantify these observations employing methods from the theory of stochastic

processes. Exact solutions are, in general, not feasible. In many instances, however, a low-noise approximation employinga Kramers–Moyal expansion gives excellent results. The Fokker–Planck equation resulting from the Master equation reads

∂tP(x, t) = −∂x(α(x)P(x, t)

)+12∂2x

(β(x)P(x, t)

)(29)

with

α(x) =1N

(w+n − w

−

n

)= x(1− x)

fA(x)− fB(x)f (x)

, (30)

β(x) =1N2(w+n + w

−

n

)=1Nx(1− x)

fA(x)+ fB(x)f (x)

. (31)

The most important conclusions to be drawn from these expressions are: (i) The diffusion term proportional to β(x) scalesas 1/N and hence vanishes for large population size. (ii) The diffusion term is always positive and vanishes at the boundariesof the phase space. (iii) The drift term is identical to the right hand side of the adjusted replicator equation, α(x) = F(x).Hence the Moran process reduces to the adjusted replicator dynamics in the limit of large populations.Specializing to the prisoner’s dilemma in the limit of weak selection, c 1, we have

α(x) = −c x(1− x),

β(x) =2Nx(1− x). (32)

At this level of description the model closely resembles the ‘‘diffusion theory’’ for population genetics as introduced byWright [61,62]. There, x denotes the relative frequency for a certain allele at one genetic locus. The drift term correspondsto all directional evolutionary forces such as selection and mutation. The diffusion term in population genetics is actuallycalled ‘‘genetic drift’’ and incorporates all non-directional processes such as random variation in survivorship and randomgamete success; see e.g. the textbook by Ewens [63]. Diffusion theory played a central role in the development of the neutraltheory of molecular evolution [64,65]; here, the neutral limit corresponds to c → 0.With the expressions for the drift and diffusion term, Eq. (32), one can calculate the fixation probability Pfix,C to end up

with only cooperators if starting with an equal fraction of cooperators and defectors. It is given by Refs. [65,63]

Pfix,C =e−

12Nc − e−Nc

1− e−Nc. (33)

The probability for fixation of defectors follows by Pfix,D = 1 − Pfix,C. Within the fitness-dominated regime (Nc → ∞)defectors fixate (Pfix,D = 1), whereas for the fluctuation-dominated neutral regime, Nc → 0, both strategies have the samechance of prevailing (Pfix,C = Pfix,D = 1

2 ).9

Insight into the maintenance of cooperation by fluctuation versus selection-dominated evolution is provided by themean extinction time T (x) of cooperators or defectors, i.e., the average time after which a population that initially displayscoexistence of cooperators and defectors consists only of one species. Within the above diffusion approximation the meanextinction time, T (x), for a population which initially contained a fraction x of cooperators, can be calculated employing thebackward Kolmogorov equation:[

α(x)∂

∂x+12β(x)

∂2

∂x2

]T (x) = −1. (34)

This equation can be solved exactly by iterative integration [34], where, in the following, we specialize to a case where theinitial population is composed of an equal number of cooperators and defectors, x = 1

2 . A main feature of the extinctiontime is that its dependence on population size N and fitness difference c can be cast into the form of a homogeneity relation

T (N, c) = TeG (N/Ne) , (35)

with a scaling function G. Te(c) andNe(c) are characteristic time scales and population sizes depending only on the selectionstrength c . Analyzing its properties, one finds that G increases linearly in N for small argument N/Ne 1, such that T ∼ N ,c.f. Fig. 9(b). This indicates [67,68] that for small system sizes, N Ne, evolution is neutral; compare the results in theprevious Section 2.1. Fluctuations dominate the evolutionary dynamics while the fitness advantage of defectors does not

9 If, instead of an equal fraction, x = 12 , one starts with a fraction x of cooperators, the standard gambler’s ruin problem tells us that in the neutral case

the fixation probability becomes Pfix,C = 1− x; see e.g. chapter 2 of Ref. [66].

E. Frey / Physica A 389 (2010) 4265–4298 4277

Fig. 9. Stochastic dynamics of the prisoner’s dilemma game. (a) Exemplary evolutionary trajectories. A high selection strength, i.e., a high fitness differencec = 0.1 (purple), leads to selection-dominated evolution and fast extinction of cooperators, while a small one, c = 0.001 (green), allows for dominanteffects of fluctuations andmaintenance of cooperation on long time-scales.We have usedN = 1000 in both cases. (b) The dependence of the correspondingmean extinction time T on the system size N . We show data from stochastic simulations as well as analytical results (solid lines) for T , starting from equalabundances of both species, for different values of c (see text): c1 = 0.1 (♦), c2 = 0.01 (), c3 = 0.001 (), and c4 = 0.0001 (4). The transition fromthe neutral to the selection-dominated regime occurs at population sizes N (1)e ,N

(2)e ,N

(3)e , and N

(4)e . They scale as 1/c: Ne ≈ 2.5/c , as is confirmed by the

rescaled plot where the data collapse onto the universal scaling function G, shown in the inset. (For interpretation of the references to colour in this figurelegend, the reader is referred to the web version of this article.)Source: Adapted from Ref. [34].

give them an edge. Indeed as shown above, in this regime, cooperators and defectors have an equal chance of surviving. For aT ∼ N behavior the extinction time considerably grows with increasing population size; a larger system size proportionallyextends the time cooperators and defectors coexist. Very different behavior emerges for large system sizes, N/Ne 1,where G increases only logarithmically in N , and therefore T ∼ lnN; compare the results in the previous Section 2.1. Thus,the extinction time remains small even for large system sizes, and coexistence of cooperators and defectors is unstable.Indeed, in this regime, selection dominates over fluctuations in the stochastic time evolution and quickly drives the systemto a state where only defectors remain, c.f. Fig. 9(b). The evolution is selection-dominated.As described above, the regimes of fluctuation and selection dominated evolution emerge for N/Ne 1 and N/Ne 1,

respectively. The cross-over population sizeNe delineates both scenarios. Further analyzing the universal scaling function G,as well as comparison with data from stochastic simulations, see Fig. 9(b), reveals that the transition at Ne is notably sharp.One may therefore refer to it as the edge of neutral evolution. The crossover time Te and the crossover population size Newhich define the edge of neutral evolution decrease as 1/c in increasing cost c [34]. This can be understood by recallingthat the cost c corresponds to the fitness advantage of defectors and can thus be viewed as the selection strength. Thelatter drives the selection-dominated dynamics which therefore intensifies when c grows, and the regime of fluctuation-dominated dynamics (neutral evolution) diminishes. On the other hand, when the cost of cooperation vanishes, evolutionbecomes neutral also for large populations. Indeed, in this case, defectors do not have a fitness advantage compared tocooperators; both do equallywell. The stochastic approach now yields information about how large the costmay be until theevolution changes from neutral to selection-dominated. From a numerical inspection of G one finds that neutral evolution ispresent for cN < 2.5, and selection-dominated evolution takes over for cN > 2.5 [34]. This resembles a condition previouslyderived by Kimura and others [65,64] for frequency independent fitness advantages. The edge of neutral evolution arises atNe = 2.5/c and Te = 2.5/c .Though selection pressure clearly disfavors cooperation, these results reveal that the ubiquitous presence of randomness

(stochasticity) in any population dynamics opens a window of opportunity where cooperation is facilitated. In the regimeof neutral evolution,

cN < 2.5 (36)

cooperators have a significant chance of taking over the whole population when initially present. Even if not, they remainon time-scales proportional to the system’s size, T ∼ N , and therefore considerably longer than in the regime of selection-dominated evolution, where they become extinct after a short transient time, T ∼ lnN .

2.3. Cyclic three-strategy games

As we have learned in the previous section, the coexistence of competing species is, due to unavoidable fluctuations,always transient. Here we illustrate this for yet another example, the cyclic Lotka–Volterra model (rock–scissors–papergame) introduced in the section on Evolutionary Game Theory 1.2 as a mathematical description for non-transitivedynamics. Like the original Lotka–Volterra model the deterministic dynamics of the rock–scissors–paper game yieldsoscillations along closed, periodic orbits around a coexistence fixed point. These orbits are neutrally stable due to theexistence of a conserved quantity ρ. Including noise in such a game it is clear that eventually only one of the three specieswill survive [55,69,70]. However, it is far from obvious which species will most likely win the contest. Intuitively, one might

4278 E. Frey / Physica A 389 (2010) 4265–4298

Fig. 10. The phase space S3 . We show the reactive fixed point F, the center Z, as well as a stochastic trajectory (red/light gray). It eventually deviatesfrom the ‘outermost’ deterministic orbit (black) and reaches the absorbing boundary. λA, λB and λC (blue/dark gray) denote distances of Parameters are(kA, kB, kC ) = (0.2, 0.4, 0.4) and N = 36. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version ofthis article.)

think, at a first glance, that it pays for a given strain to have the highest reaction rate and hence strongly dominate over itscompetitor. As it turns out, however, the exact opposite strategy is the best [71]. One finds what could be called a ‘‘law of theweakest’’: When the interactions between the three species are (generically) asymmetric, the ‘‘weakest’’ species (i.e., theone with the smallest reaction rate) survives at a probability that tends to one in the limit of a large population size, whilethe other two are guaranteed to go to extinction.The reason for this unexpected behavior is illustrated in Fig. 10, showing a deterministic orbit and a typical stochastic

trajectory. For asymmetric reaction rates, the fixed point is shifted from the center Z of the phase space (simplex) towardsone of the three edges. All deterministic orbits are changed in the same way, squeezing in the direction of one edge. InFig. 10 reaction rates are chosen such that the distance λA to the a-edge of the simplex, where Awouldwin the contest, is thesmallest. The important observation here is that because of simple geometric reasons λA is the smallest because the reactionrate kA is the smallest! Intuitively, the absorbing state which is reached from this edge has the highest probability of beinghit, as the distance λ from the deterministic orbit towards this edge is the shortest. Indeed, this behavior can be validatedby stochastic simulations complemented by a scaling argument [71]. One can go even beyond such a scaling analysis andanalytically calculate the probability distribution of extinction times [72].

2.4. A general concept for classifying stability

The dependence of the mean extinction time T of competing species on the system size N provides a universal andpowerful framework to distinguish neutral from selection-dominated evolution. Indeed, selection, as a result of someinteractions within a finite population, can either stabilize or destabilize a species’ coexistence with others as comparedto neutral interactions, thereby altering the mean time until extinction occurs.For the selection-dominated regime, instability leads to steady decay of a species, and therefore to fast

extinction [67,68,73]: The mean extinction time T increases only logarithmically in the population size N , T ∼ lnN , anda larger system size does not ensure much longer coexistence. This behavior can be understood by noting that a speciesdisfavored by selection decreases by a constant rate. Consequently, its population size decays exponentially in time, leadingto a logarithmic dependence of the extinction time on the initial population size. In contrast, the stable existence of a speciesinduces T ∼ expN , such that extinction takes an astronomically long time for large populations [67,68,34]. In this regime,extinction stems from large fluctuations that cause sufficient deviation from the (deterministically) stable coexistence. Theselarge deviations are exponentially suppressed and hence the time until a rare extinction event occurs scales exponentiallyin the system size N . Then coexistence is maintained on ecologically relevant time-scales which typically lie below T . Anintermediate situation, i.e., when T has a power-law dependence on N , T ∼ Nα , signals dominant influences of stochasticeffects and corresponds to neutral evolution. Here the extinction time grows considerably, though not exponentially, inincreasing population size. Large N therefore clearly prolongs coexistence of species but can still allow for extinction withinbiologically reasonable time-scales. A typical neutral regime, as found above in the prisoner’s dilemma, is characterized byα = 1, such that T scales linearly in the system size N . There, the dynamics yields an essentially unbiased random walk instate space; see Fig. 9(a). Themean-square displacement grows linearly in time, with a diffusion constant proportional to N .The absorbing boundary is thus reached after a timeproportional to the systemsizeN . Summarizing these considerations,wehave proposed a quantitative classification of coexistences stability in the presence of absorbing states, which is presentedin the following Box [73]:

E. Frey / Physica A 389 (2010) 4265–4298 4279

Classification of coexistence stability

Stability: If the mean extinction time T increases faster than any power of the system size N , meaning T/Nα →∞ in the asymptotic limitN →∞ and for any value ofα > 0, we refer to coexistence as stable. In this situation,typically, T increases exponentially in N .

Instability: If the mean extinction time T increases slower than any power in the system size N , meaningT/Nα → 0 in the asymptotic limit N →∞ and for any value of α > 0, we refer to coexistence as unstable. Inthis situation, typically, T increases logarithmically in N .

Neutral stability: Neutral stability lies in between stable and unstable coexistence. It emerges when the meanextinction time T increases proportionally to some power α > 0 of the system size N , meaning T/Nα → O(1)in the asymptotic limit N →∞.

The strength of the above classification lies in that it only involves quantities which are directly measurable (for examplethrough computer simulations), namely the mean extinction time and the system size. Therefore, it is generally applicableto stochastic processes, e.g. incorporating additional internal population structure like individuals’ age or sex, or whereindividuals interaction networks are more complex, such as lattices, scale-free- networks or fractal ones. In these situation,it is typically impossible to infer analytically, from the discussion of fixed points stability, whether the deterministicpopulation dynamics yields a stable or unstable coexistence. However, based on the scaling of extinction time T withsystem size N , differentiating stable from unstable diversity according to the above classification is feasible. In Section 4,we will follow this line of thought and fruitfully apply the above concept to the investigation of a rock–paper–scissorsgame on a two-dimensional lattice, where an individual’s mobility is found to mediate between stable and unstablecoexistence.

3. The May–Leonard model

In ecology, competition for resources has been classified [74] into two broad groups, scramble and contest. Contestcompetition involves direct interaction between individuals. In the language of evolutionary game theory the winner inthe competition replaces the loser in the population (Moran process). In contrast, scramble competition involves the rapiduse of limiting resources without direct interaction between the competitors.The May–Leonard model [75] of cyclic dominance between three subpopulations A, B and C dissects the non-transitive

competition between these into a contest and a scramble step. In the contest step an individual of subpopulation Aoutperforms a B through ‘‘killing’’ (or ‘‘consuming’’), symbolized by the (‘‘chemical’’) reaction AB→ A, where denotesan available empty space. In the same way, B outperforms C , and C beats A in turn, closing the cycle. We refer to thesecontest interactions as selection and denote the corresponding rate byσ . In the scramble step,whichmimics a finite carryingcapacity, each member of a subpopulation is allowed to reproduce only if an empty space is available, as described by thereaction A → AA and analogously for B and C . For all subpopulations, these reproduction events occur with rate µ, suchthat the three subpopulations equally compete for empty space. To summarize, the reactions that define the May–Leonardmodel (selection and reproduction) read

ABσ−→ A, A

µ−→ AA,

BCσ−→ B, B

µ−→ BB,

CAσ−→ C, C

µ−→ CC . (37)

Let a, b, c denote the densities of subpopulations A, B, and C , respectively. The overall densityρ then readsρ = a+b+c .As every lattice site is at most occupied by one individual, the overall density (as well as densities of each subpopulation)varies between 0 and 1, i.e. 0 ≤ ρ ≤ 1. With these notations, the rate equations for the reactions (37) are given by

∂ta = a [µ(1− ρ)− σ c],∂tb = b [µ(1− ρ)− σa],∂tc = c [µ(1− ρ)− σb], (38)

or in short (with Ea = (a, b, c))

∂tEa = EF(Ea). (39)

The goal of this section is to analyze this model with advanced tools from nonlinear dynamics [76]: linear stability analysis,invariant manifolds, and normal forms. In addition, the analysis will also serve as a necessary prerequisite for the analysis ofspatial games in the subsequent section. This section is mainly technical and may be skipped in a first reading.The phase space of the model is organized by fixed point and invariant manifolds. Eq. (38) possess four absorbing fixed

points. One of these (unstable) is associated with the extinction of all subpopulations, (a∗1, b∗

1, c∗

1 ) = (0, 0, 0). The others

4280 E. Frey / Physica A 389 (2010) 4265–4298

Fig. 11. The phase space of the May–Leonard model. It is spanned by the densities a, b, and c of species A, B, and C . On an invariant manifold (yellow), theflows obtained as solutions of the rate equation (38) (an example trajectory is shown in blue) initially in the vicinity of the reactive fixed point (red) spiraloutwards, approaching the heteroclinic cycle which connects three trivial fixed points (blue). In Section 3.2, we introduce the appropriate coordinates(yA, yB, yC )which reveal the mathematical structure of the manifold and reflect the cyclic symmetry of the system. (For interpretation of the references tocolour in this figure legend, the reader is referred to the web version of this article.)Source: Adapted from Ref. [77].

are heteroclinic points (i.e. saddle points underlying the heteroclinic orbits) and correspond to the survival of only onesubpopulation, (a∗2, b

∗

2, c∗

2 ) = (1, 0, 0), (a∗3, b∗

3, c∗

3 ) = (0, 1, 0) and (a∗4, b∗

4, c∗

4 ) = (1, 0, 0), shown in blue (dark gray) inFig. 11. In addition, there exists a reactive fixed point, indicated in red (gray) in Fig. 11, where all three subpopulationscoexist (at equal densities), namely (a∗, b∗, c∗) = µ

3µ+σ (1, 1, 1).For a non-vanishing selection rate, σ > 0, Leonard andMay [75] showed that the reactive fixed point is unstable, and the

system asymptotically approaches the boundary of the phase space (given by the planes a = 0, b = 0, and c = 0). There,they observed heteroclinic orbits: the system oscillates between states where nearly only one subpopulation is present, witha rapidly increasing cycle duration. While mathematically fascinating, this behavior was recognized to be unrealistic [75].For instance, as discussed in Section 2, the system will, due to finite-size fluctuations, always reach one of the absorbingfixed points in the vicinity of the heteroclinic orbit, and then only one population survives.

3.1. Linear stability analysis: Jordan normal form

Our goal here is to study the nonlinear dynamics close to the coexistence (reactive) fixed point Ea∗. We would like toknow its stability and the typical behavior of a trajectory in its vicinity. To this end we introduce a shifted reference frameby defining a displacement vector

Ex = Ea− Ea∗ = (a− a∗, b− b∗, c − c∗)T . (40)

Then

∂tEx = EF(Ex+ Ea) = DF |Ea∗ Ex+ EG(Ex) (41)

where A ≡ DF |Ea∗ is the Jacobian of EF at the reactive fixed point Ea∗, and EG(Ex) is the remaining nonlinear part of EF(Ex + Ea).From the structure of EF , we know that EG is quadratic in xA, xB, and xC . Explicitly one finds

A = −µ

3µ+ σ

(µ µ µ+ σ

µ+ σ µ µµ µ+ σ µ

). (42)

and

EG =

(µxA(xA + xB)+ xAxC (µ+ σ)µxB(xB + xC )+ xBxA(µ+ σ)µxC (xC + xA)+ xCxB(µ+ σ)

). (43)

E. Frey / Physica A 389 (2010) 4265–4298 4281

As the matrix A is circulant, its eigenvalues can be obtained from a particularly simple general formula (see e.g. Ref. [10]);they read:

λ0 = −µ,

λ± = c1 ± iω (44)with

c1 =12

µσ

3µ+ σ,

ω =

√32

µσ

3µ+ σ(45)

and corresponding (complex) eigenvectors ξ0, and ξ±; note that ξ0 = 13 (1, 1, 1)

T is easy to guess. From the sign of theeigenvalues we can read off that the reactive fixed point is linearly stable along the eigendirection of the first eigenvalueλ0. As elaborated below, there exists an invariant manifold [76] (including the reactive fixed point), that the system quicklyapproaches. To first order such a manifold is the plane normal to the eigendirection of λ0. On this invariant manifold, flowsspiral away from the reactive fixed point, which is an unstable spiral, as sketched in Fig. 11 (blue trajectory).10To complete the linear stability analysis, it is useful to transform to Jordan normal form by introducing suitable

coordinates (yA, yB, yC ) originating in the reactive fixed point. We choose the yC -axis to coincide with the eigenvector of λ0,and the coordinates yA and yB to span the plane normal to the axis yC , forming an orthogonal set. The coordinates (yA, yB, yC )are shown in Fig. 11. Such coordinates Ey = (yA, yB, yC ) are, e.g., obtained by the linear transformation Ey = SEx, with thematrixS given by

S =13

√3 0 −√3

−1 2 −11 1 1

. (48)

To linear order this gives

∂tEy = JEy (49)with

J = SAS−1 =

( c1 ω 0−ω c1 00 0 −µ

). (50)

With these results we may now rewrite the dynamics in the reference frame of the Jordan normal form which is theoptimized frame for the linear stability analysis:

∂tEy = JEy+ SEG(S−1Ey) ≡ JEy+ EH(Ey), (51)where one finds (with a straightforward calculation)

EH(Ey) =

√34σ[y2A − y

2B

]−σ

2yAyB −

12yC[(6µ+ σ)yA −

√3σyB

]−σ

4

[y2A − y

2B

]−

√32σyAyB −

12yC[√3σyA + (6µ+ σ)yB

]−(3µ+ σ)y2C +

σ

4

[y2A + y

2B

]

. (52)

As the rate equations, Eq. (38), have one real eigenvalue smaller than zero and a pair of complex conjugate eigenvalues,they fall into the class of the Poincaré–Andronov–Hopf bifurcation, well known in the mathematical literature [76]. Thetheory of invariant and center manifolds allows us to recast these equations into a normal form. The latter, as discussed inthe next section, will turn out to be instrumental in the derivation of the complex Ginzburg–Landau equation (CGLE).

10 The linear stability analysis only reveals the local stability of the fixed points. The global instability of the reactive fixed point is proven by the existenceof a Lyapunov functionL [10,75]:

L =abcρ3. (46)

In fact, using Eq. (38), the time derivative ofL is found to be always non-positive,

∂tL = −12σρ−4abc

[(a− b)2 + (b− c)2 + (c − a)2

]≤ 0. (47)

We note that ∂tL vanishes only at the boundaries (a = 0, b = 0 or c = 0) and along the line of equal densities, a = b = c. The latter coincides with theeigendirection of λ0 , along which the system approaches the reactive fixed point. However, on the invariant manifold we recover ∂tL < 0, correspondingto a globally unstable reactive fixed point, as exemplified by the trajectory shown in Fig. 11.

4282 E. Frey / Physica A 389 (2010) 4265–4298

3.2. Invariant manifold

An invariant manifold is a subspace, embedded in the phase space, which is left invariant by the deterministic dynamics,Eq. (38). In phase space, thismeans that flows starting on an invariantmanifold always lie and evolve on it. Here, we considera two-dimensional invariant manifoldM associated with the reactive fixed point of the rate equations, Eq. (38), onto whichall trajectories (initially away from the invariant manifold) decay exponentially quickly.We call this manifoldM the reactivemanifold. Upon restricting the dynamics to that reactive invariant manifold, the system’s degrees of freedom are reducedfrom three to two, which greatly simplifies the mathematical analysis.To determine this invariant manifold, we notice that the eigenvector of the eigenvalue λ0 < 0 at the reactive fixed

point is a stable (attractive) direction. Therefore, to lowest order around the reactive fixed point, the invariant manifold issimply the plane normal to the eigendirection of λ0. To parameterize the invariant manifold sketched in Fig. 11, we seek afunctionM(yA, yB), with yC = M(yA, yB). If all nonlinearities of the rate equations, Eq. (38), are taken into account, this is avery complicated problem. However, for our purpose it is sufficient to expandM to second order in yA, yB. As the invariantmanifold is left invariant by the dynamics, by definition,M must obey

∂tM (yA(t), yB(t)) =∂M∂yA

∂tyA +∂M∂yB

∂tyB = ∂tyC |yC=M . (53)

To linear order in yA and yB, we simply have M = 0 and recover yC = 0, corresponding to the plane normal to theyC -direction. We have anticipated this result above: to first order, the invariant manifold coincides with this plane, and istangential to it when higher orders are included. To second order, only linear terms of ∂tyA, ∂tyB contribute to Eq. (53). Thelatter are invariant under rotations in the (yA, yB)-plane, andM must obey the same symmetry. It is therefore proportionalto y2A + y

2B. After some simple calculations, one obtains:

yC = M(yA, yB) =σ

4µ3µ+ σ3µ+ 2σ

(y2A + y2B)+ o(Ey

2). (54)

The comparison of this expression for the invariant manifold, valid to second order, with the numerical solutions of therate equations, Eq. (38), (which should, up to an initial transient, lie on the invariant manifold) confirms that Eq. (54) is anaccurate approximation, with only minor deviations occurring near the boundaries of the phase space.

3.3. Normal form: the simplest nonlinear dynamics

Nonlinear systems are notably characterized by the bifurcations that they exhibit [76]. Normal forms are defined asthe simplest differential equations that capture the essential features of a system near a bifurcation point, and thereforeprovide an insight into the system’s universal behavior. Here, we derive the normal form associated with the rate equations,Eq. (38), of the May–Leonard model and show that they belong to the universality class of the Hopf bifurcation [76]. Below,we demonstrate that this property allows one to describe the system in terms of a well-defined complex Ginzburg–Landauequation.Restricting the (deterministic) dynamics onto the invariant manifold, given by Eq. (54), the system’s behavior can be

analyzed in terms of two variables. Here, we choose to express yC as a function of yA and yB, with the resulting rate equationsgiven up to third order by:

∂tyA =µσ

2(3µ+ σ)

[yA +√3yB]+

√34σ[y2A − y

2B

]−σ

2yAyB

−σ(3µ+ σ)8µ(3µ+ 2σ)

(y2A + y

2B

) [(6µ+ σ)yA −

√3σyB

]+ o(y3),

∂tyB =µσ

2(3µ+ σ)

[yB −√3yA]−σ

4

[y2A − y

2B

]−

√32σyAyB

−σ(3µ+ σ)8µ(3µ+ 2σ)

(y2A + y

2B

) [√3σyA + (6µ+ σ)yB

]+ o(y3). (55)

This set of nonlinear equations can be cast into a normal form (see Ref. [76, Chapter 2.2]) by performing a nonlinear variabletransformation Ey → Ez which eliminates the quadratic terms and preserves the linear ones (i.e. Ey and Ez coincide to linearorder). As an Ansatz for such a transformation, we choose the most general quadratic expression in Ey for the new variableEz.11 One finds for the normal form of the rate equations in the new variables:

∂tzA = c1zA + ωzB − c2 (zA + c3zB) (z2A + z2B )+ o(z

3),

∂tzB = c1zB − ωzA − c2 (zB − c3zA) (z2A + z2B )+ o(z

3). (58)

11 The equations of motion, Eq. (55), comprise quadratic and cubic terms. To recast them in their normal form, we seek a transformation allowing oneto eliminate the quadratic terms. We make the Ansatz of a quadratic transformation Ey → Ez and determine the coefficients by canceling the quadratic

E. Frey / Physica A 389 (2010) 4265–4298 4283

In these equations,

ω =

√32

µσ

3µ+ σ, (59)

is the (linear) frequency of oscillations around the reactive fixed point. The constant

c1 =12

µσ

3µ+ σ, (60)

gives the intensity of the linear drift away from the fixed point, while

c2 =σ(3µ+ σ)(48µ+ 11σ)

56µ(3µ+ 2σ), (61)

c3 =

√3(18µ+ 5σ)48µ+ 11σ

, (62)

are the coefficients of the cubic corrections. In complex notation, z = zA + izB, we have

∂tz = (c1 − iω)z − c2 (1+ ic3) |z|2z. (63)

To gain some insight into the dynamics in the normal form, it is useful to rewrite Eq. (58) in polar coordinates (r, φ),where zA = r cosφ, zB = r sinφ. This leads to

∂t r = r[c1 − c2r2],

∂tθ = −ω + c2c3r2. (64)

These equations only have a radial dependence, which clearly reveals a polar symmetry. They predict the emergence of alimit cycle of radius r =

√c1/c2 and therefore fall into the universality class of the (supercritical) Hopf bifurcation. It can

be shown that an asymmetry in the reaction rates, not taken into account in the above analysis, leads to the same type ofdynamics, merely with renormalized parameters [78].We close with a final remark on the approximate nonlinear dynamics obtained from the construction of a second order

normal form. We know from the original analysis by Leonard and May that when all nonlinearities are taken into account,the rate equations, Eq. (38), give rise to heteroclinic orbits instead of limit cycles. The heteroclinic orbits rapidly approach theboundaries of the phase space, and thus are in general well separated from the limit cycles predicted by (64). Surprisingly,however, one finds for the spatial system, discussed in the next section, that an analytical approach employing the abovecomplex Ginzburg Landau equation gives results which nicely agree with agent-based stochastic lattice simulations. As itturns out, the reason for this surprising agreement is that the interplay between noise and spatial degrees of freedom leadsto an effective space-induced ‘‘stochastic limit’’ cycle [72].

4. Spatial games with cyclic dominance

Spatial distribution of individuals, as well as their mobility, are common features of real ecosystems that often comepaired [79]. On all scales of living organisms, from bacteria residing in soil or on Petri dishes, to the largest animals living insavannas – like elephants – or in forests, populations’ habitats are spatially extended and individuals interact locally withintheir neighborhood. Field studies as well as experimental and theoretical investigations have shown that the locality of theinteractions leads to the self-formation of complex spatial patterns [79–85,1,86–93]. Another important property of mostindividuals is mobility. For example, bacteria swim and tumble, and animals migrate. As motile individuals are capable ofenlarging their district of residence, mobility may be viewed as a mixing or stirring mechanism which ‘‘counteracts’’ thelocality of spatial interactions.

contributions to the RE in the Ez variables. This leads to

zA = yA +3µ+ σ28µ

[√3y2A + 10yAyB −

√3y2B],

zB = yB +3µ+ σ28µ

[5y2A − 2√3yAyB − 5y2B]. (56)

To second order, this nonlinear transformation can be inverted:

yA = zA −3µ+ σ28µ

[√3z2A + 10zAzB −

√3z2B ] +

(3µ+ σ)2

14µ2[z3A + zAz

2B ] + o(z

3),

yB = zB −3µ+ σ28µ

[5z2A − 2√3zAzB − 5z2B ] +

(3µ+ σ)2

14µ2[z2AzB + z

3B ] + o(z

3). (57)

With these expressions, one can check that equations of motion, Eq. (55), are recast in the normal form, Eq. (58).

4284 E. Frey / Physica A 389 (2010) 4265–4298

A B

C

Fig. 12. Individuals on neighboring sites may react with each other according to the rules of cyclic dominance (selection; contest competition), orindividuals may give birth to new individuals if they happen to be next to an empty site (reproduction; scramble competition).

The role of mobility in ecosystems

The interplay between mobility and spatial separation on the spatio-temporal development of populations is one ofthe most interesting and complex problems in theoretical ecology [79–81,84,86,73]. If mobility is low, locally interactingpopulations can exhibit involved spatio-temporal patterns,12 and for example lead to the self-organization of individuals intospirals in myxobacteria aggregation [102] and insect host-parasitoid populations [83]. See also the discussion in Section 1.4.In contrast, high mobility results in well-mixed systems where the spatial distribution of the populations is irrelevant [73].In this situation, spatial patterns no longer form: The system adopts a spatially uniform state, which therefore drasticallydiffers from the low-mobility scenario.Pioneering work on the role of mobility in ecosystems was performed by Levin [103], who investigated the dynamics

of a population residing in two coupled patches: Within a deterministic description, he identified a critical value forthe individuals’ mobility between the patches. Below the critical threshold, all subpopulations coexisted, while only oneremained above that value. Later, more realistic models of many patches, partly spatially arranged, were also studied;see e.g. Refs. [83–85,104] as well as references therein. These works shed light on the formation of patterns, in particulartravelingwaves and spirals. However, patchmodels have been criticized for treating the space in an ‘‘implicit’’ manner (i.e. inthe form of coupled habitats without internal structure) [25]. In addition, the above investigations were often restricted todeterministic dynamics and thus did not address the spatio-temporal influence of noise. To overcome these limitations,Durrett and Levin [24] proposed to consider interacting particle systems, i.e. stochastic spatial models with populationsof discrete individuals distributed on lattices. In this realm, studies have mainly focused on numerical simulations and on(often heuristic) deterministic reaction-diffusion equations, or coupled maps [105,24,25,86,26,106–108,91].

Cyclic dominance in ecosystems

An intriguing motif of the complex competitions in a population, promoting species diversity, is constituted by threesubpopulations exhibiting cyclic dominance, also called non-transitive competition. This basic motif is metaphoricallydescribed by the rock–paper–scissors game,where rock crushes scissors, scissors cut paper, and paperwraps rock. Such non-hierarchical, cyclic competitions, where each species outperforms another, but is also itself outperformed by a remainingone, have been identified in different ecosystems like coral reef invertebrates [109], rodents in the high-Arctic tundra inGreenland [110], lizards in the inner Coast Range of California [111] andmicrobial populations of colicinogenic E. coli [1,112].As we have discussed in Section 1.4, in the latter situation it has been shown that spatial arrangement of quasi-immobilebacteria on a Petri-dish leads to the stable coexistence of all three competing bacterial strains, with the formation of irregularpatterns. In stark contrast, when the system is well-mixed, there is spatial homogeneity resulting in the take over of onesubpopulation and the extinction of the others after a short transient.

4.1. The spatially-extended May–Leonard model

In this chapter we analyze the stochastic spatially-extended version of the May–Leonard model [73], as illustrated inFig. 12 This is supposed to mimic the generic features found in ecosystems with species forming non-transitive interactionnetworks. We adopt an interacting particle description where individuals of all subpopulations are arranged on a lattice. LetL denote the linear size of a 2-dimensional square lattice (i.e. the number of sites along one edge), such that the total numberof sites reads N = L2. In this approach, each site of the grid is either occupied by one individual or is empty, meaning thatthe system has a finite carrying capacity, and the reactions are then only allowed between nearest neighbors.In addition, we endow the individuals with a certain form of mobility. Namely, at rate ε all individuals can exchange theirposition with a nearest neighbor. With that same rate ε, any individual can also hop on a neighboring empty site. These

12 The emergence of spatio-temporal noisy patterns is a feature shared across disciplines by many complex systems characterized by their out-of-equilibrium nature and nonlinear interactions. Examples range from the celebrated Belousov–Zhabotinsky reaction [94] (spiralling patterns) and manyother chemical reactions [95], to epidemic outbreaks (traveling waves) [96,97], excitable media [98,95], and calcium signaling within single cells [99–101].

E. Frey / Physica A 389 (2010) 4265–4298 4285

Fig. 13. The stochastic spatial system at different scales. Here, each of the colors yellow, red, and blue (level of gray) represents one species, and black dotsidentify empty spots. Left: Individuals are arranged on a spatial lattice and randomly interact with their nearest neighbors. Middle: At the scale of about1000 individuals, stochastic effects dominate the system’s appearance, although domains dominated by different subpopulations can already be detected.Right: About 50,000 mobile interacting individuals self-organize into surprisingly regular spiral waves. (For interpretation of the references to colour inthis figure legend, the reader is referred to the web version of this article.)Source: Figure adapted from Ref. [77].

‘‘microscopic’’ exchange processes lead to an effective diffusion of the individuals described by a macroscopic diffusionconstant D = ε/2L2.The goal of this chapter is to analyze the spatio-temporal dynamics at asymptotically long time scales as a function of

the hopping and reaction rates. We will learn that the mobility of the individuals, characterized in terms of their diffusionconstant, has a critical influence on species diversity. When mobility exceeds a certain threshold value, biodiversity isjeopardized and lost [73]. In contrast, below this critical threshold all subpopulations coexist and the spatial stochasticmodelof cyclically interacting subpopulations self-organizes into regular, geometric spiral waves [113,77]. The latter becomevisible on the scale of a large number of interacting individuals, see Fig. 13 (right). In contrast, stochastic effects solelydominate on the scale of a few individuals, see Fig. 13 (left), which interact locally with their nearest neighbors. Spatialseparation of subpopulations starts to form on an intermediate scale, Fig. 13 (middle), wheremobility leads to fuzzy domainboundaries, withmajor contributions of noise. On a larger scale, Fig. 13 (right), these fuzzy patterns adopt regular geometricshapes. As shown below, the latter are jointly determined by the deterministic dynamics and intrinsic stochastic effects.In the following, we elucidate this subtle interplay between noise and space by mapping – in the continuum limit – the

stochastic spatial dynamics onto a set of stochastic partial differential equations (SPDE) and, using tools of dynamical systems(such as normal forms and invariant manifolds), by recasting the underlying deterministic kinetics in the form of a complexGinzburg–Landau equation (CGLE).We have detailed these tools in the previous section. A remarkable findingwill be that theCGLE enables one to make analytical predictions for the spreading velocity and wavelength of the emerging spirals waveswhich quantitatively agree with the numerical findings.

4.2. Simulation results for the lattice gas model

Westartwith a description of the results obtained from stochastic simulations of the lattice gasmodel [73]. For simplicity,we consider equal reaction rates for selection and reproduction, and, without loss of generality, set the time-unit by fixingσ = µ ≡ 1. From the phase portrait of the May–Leonard model it is to be expected that an asymmetry in the parametersyields only qualitative but not quantitative changes in the system’s dynamics. The length scale is chosen such that the lineardimension of the lattice is the basic length unit, L ≡ 1. With this choice of units the diffusion constant measures the fractionof the entire lattice area explored by an individual in one unit of time.Typical snapshots of the steady states are shown in Fig. 14. 13When the mobility of the individuals is low, one finds that

all species coexist and self-arrange by forming patterns of moving spirals. Increasing the mobility D, these structures growin size, and disappear for large enoughD. In the absence of spirals, the system adopts a uniform state where only one speciesis present, while the others have died out. Which species remains is subject to a random process, all species having equalchances to survive in the symmetric model defined above.The transition from the reactive state containing spirals to the absorbing state with only one subpopulation left is a non-

equilibrium phase transition [113]. One way to characterize the transition is to ask how the extinction time T , i.e. the timefor the system to reach one of its absorbing states, scales with system size N . In our analysis of the role of stochasticity inSection 2 we have found the following classification scheme. If T ∼ N , the stability of coexistence is marginal. Conversely,longer (shorter) waiting times scaling with higher (lower) powers of N indicate a stable (unstable) coexistence. These threescenarios can be distinguished by computing the probability Pext that two species have gone extinct after a waiting timet ∼ N:

Pext = Prob [only one species left after time T ∼ N] . (65)

13 You may also want to have a look at the movies posted on http://www.theorie.physik.uni-muenchen.de/lsfrey/research/fields/biological_physics/2007_004. There is also a Wolfram demonstration project which can be downloaded from the web: http://demonstrations.wolfram.com/BiodiversityInSpatialRockPaperScissorsGames/.

4286 E. Frey / Physica A 389 (2010) 4265–4298

Fig. 14. Snapshots obtained from lattice simulations are shown of typical states of the system after long temporal development (i.e. at time t ∼ N) and fordifferent values of D (each color, blue, yellow and red, represents one of the species and black dots indicate empty spots). Increasing D (from left to right),the spiral structures grow, and outgrow the system size at the critical mobility Dc : then, coexistence of all three species is lost and uniform populationsremain (right). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)Source: Figure adapted from Ref. [73].

Fig. 15. The extinction probability Pext that, starting with randomly distributed individuals on a square lattice, the system has reached an absorbing stateafter a waiting time T ∼ N . Pext is shown as a function of the mobility D (and σ = µ = 1) for different system sizes: N = 20× 20 (green), N = 30× 30(red), N = 40 × 40 (purple), N = 100 × 100 (blue), and N = 200 × 200 (black). As the system size increases, the transition from stable coexistence(Pext = 0) to extinction (Pext = 1) sharpens at a critical mobility Dc ≈ (4.5 ± 0.5) × 10−4 . (For interpretation of the references to colour in this figurelegend, the reader is referred to the web version of this article.)Source: Figure adapted from Ref. [73].

In Fig. 15, the dependence of Pext on the mobility D is shown for a range of different system sizes, N . Increasing the systemsize, a sharpened transition emerges at a critical value Dc = (4.5 ± 0.5) × 10−4. Below Dc , the extinction probability Pexttends to zero as the system size increases, and coexistence is stable in the sense defined in Section 2. In contrast, above thecritical mobility, the extinction probability approaches one for large system size, and coexistence is unstable. As a centralresult, the agent-based simulations show that there is a critical threshold value for the individuals’ diffusion constant, Dc , suchthat a low mobility, D < Dc , guarantees the coexistence of all three species, while a high mobility, D > Dc , induces the extinctionof two of them, leaving a uniform state with only one species [73].

4.3. Reaction-diffusion equations

Before embarking into the endeavor of fully analyzing the non-equilibrium dynamics let us disregard for themoment theeffect of noise and consider the deterministic spatial dynamics. We consider a continuum limit where the linear dimensionof the lattice is chosen as the basic length unit, L ≡ 1, and hence the lattice constant becomes a = 1/L. Then the ensuingdiffusion-reaction equations for the density vector Ea(Er, t) = (a(Er, t), b(Er, t), c(Er, t)) are given by a set of partial differentialequations (PDE)

∂tEa(Er, t) = D∇2Ea+ EF(Ea) (66)with the macroscopic diffusion constant

D =ε

2N. (67)

We are interested in the limit N →∞ and want the macroscopic diffusion constant D to be finite in that limit. This impliesthat the rate ε becomes large compared to the selection and reproduction rates, µ and σ , and thus – in the lattice model –a large number of hopping and exchange events occurs between two reactions.These (deterministic) reaction-diffusion equations can be solved numerically. Fig. 16(d) shows the outcome of such a

simulation starting from a inhomogeneous initial condition (and using periodic boundary conditions) [73]. One realizes that

E. Frey / Physica A 389 (2010) 4265–4298 4287

(a) Typical spiral. (b) Agent based. (c) SPDE. (d) PDE.

Fig. 16. Spiral patterns. (a) Schematic drawing of a spiral with wavelength λ. It rotates around the origin at a frequency ω. (b) Agent-based simulationsfor D < Dc , when all three species coexist, show entangled, rotating spirals. (c) Stochastic partial differential equations, discussed in Section 4.4, showsimilar patterns as agent-based simulations. (d) Spiral pattern emerging from the dynamics of the deterministic diffusion-reaction equation starting froma spatially inhomogeneous initial state. Parameters are σ = µ = 1 and D = 1× 10−5 .Source: Figure adapted from Ref. [73].

ignoring noise in the equation of motion also gives rise to spiral patterns. They share the same wavelength and frequencywith those of the agent-based simulations, but, their overall size andnumber dependon the initial conditions and candeviatesignificantly from their stochastic counterparts.Significant theoretical progress can now be made upon employing the results of the nonlinear dynamics in Section 3.

Projecting the diffusion-reaction equation, Eq. (66), onto the reactive manifold M one obtains [73,113,77]:

∂tz = D∇2z + (c1 − iω)z − c2(1+ ic3)|z|2z. (68)

Here, we recognize the celebrated complex Ginzburg–Landau equation (CGLE), whose properties have been extensivelystudied in the past [114,115]. In particular, it is known that in two dimensions the latter gives rise to a broad range ofcoherent structures, including spiral waves whose velocity, wavelength and frequency can be computed analytically.

The linear spreading velocityIn the stochastic simulations we have seen that in the long-time regime the system exhibits traveling waves. In the

steady state, regions with nearly only A individuals are invaded by a front of C individuals, which is taken over by B inturn, and so on. This can be understood as a front propagation phenomenon into an unstable state as follows. The complexGinzburg–Landau equation has an unstable fixed point at the reactive center, z = 0. Hence any small perturbation willgrow and lead to a spreading pulse whose form is determined by the nonlinearities in the equation. Its velocity, however,can already be calculated by analyzing the linearized equations; for a recent review on the theory of front propagation intounstable states see Ref. [116].The CGLE (68) linearized around the coexistence state z = 0 reads

∂tz(Er, t) = D∆z(Er, t)+ (c1 − iω)z(Er, t). (69)

We perform a Fourier transformation

z(Ek, t) =∫∞

−∞

dEr z(Er, t)e−iEk·Er , (70)

and make the ansatz z(Ek, t) = z(Ek) e−iΩ(Ek)t for each Fourier mode. The linearized CGLE then gives the following dispersionrelation

Ω(k) = ω + i(c1 − Dk2), (71)

where k = |Ek|. As ImΩ(k) > 0 for k2 < c1/D, the state z = 0 is linearly unstable in this range ofwavevectors k. This confirmsthe analysis of the spatially homogeneous nonlinear dynamics, Eq. (38), where we already found that the coexistence fixedpoint is unstable. As for other systems characterized by fronts propagating into unstable states [116], from Eq. (69) one cannow compute the linear spreading velocity, i.e. the speed v∗ at which fronts propagate. For completeness, we repeat theclassical treatment which can, e.g., be found in Ref. [116]. A Fourier back-transform of the above results gives

z(Er, t) =∫

d2k(2π)2

z(Ek) eiEk·Er−iΩ(Ek)t . (72)

Say the front is propagating with a speed Ev∗ = v∗ex in the x-direction. Then in the co-moving frame, Eξ = Er − Ev∗t , thisvelocity is determined (self-consistently) such that the ensuing expression of the pulse form neither grows nor decays

z(Eξ, t) =∫

d2k(2π)2

z(Ek) eiEk·Eξ−i[Ω(Ek)−Ev∗·Ek]t . (73)

4288 E. Frey / Physica A 389 (2010) 4265–4298

For large times, t →∞, the integral may be performed by a saddle-point expansion with the saddle k∗ determined by

d[Ω(k)− v∗k]dk

∣∣∣∣k∗= 0⇒ v∗ =

dΩ(k)dk

∣∣∣∣k∗

(74)

and the integral given to leading order by z(Eξ, t) ∝ eiEk·Eξ−i[Ω(k∗)−Ev∗·Ek]t . In order for this to neither grow nor decay we must

have

ImΩ(k∗)− v∗Im k∗ = 0⇒ v∗ =ImΩ(k∗)Im k∗

. (75)

Hence the linear spreading velocity is obtained by determining a wave vector k∗ according to

dΩ(k)dk

∣∣∣∣k∗=ImΩ(k∗)Im k∗

≡ v∗. (76)

The first equality singles out k∗ and the second defines the linear spreading velocity v∗. Here, one finds Re k∗ = 0,Im k∗ =

√c1/D, and hence

v∗ = 2√c1D = 2

√D

√12

µσ

3µ+ σ. (77)

Wavelength and frequencyTo determine analytically the wavelength λ and the frequencyΩ of the spiral waves, the (cubic) nonlinear terms of the

CGLE, Eq. (68), have to be taken into account. From the understanding gained in the previous sections, we make a traveling-wave ansatz z(Er, t) = Ze−iΩt−iEk·Er leading to the following dispersion relation (with k = |Ek|)

Ω(k) = ω + i(c1 − Dk2)− c2(i+ c3)Z2. (78)Separating real and imaginary parts, we can solve for Z , resulting in Z2 = (c1 − Dk2)/c2. As already found above, the rangeof wave vectors that yields traveling wave solutions is therefore given by k <

√c1/D. The dispersion relation can, upon

eliminating Z , be rewritten as

Ω(k) = ω + c3(Dk2 − c1), (79)comprising of two contributions. First, there is ω, acting as a ‘‘background frequency’’, which stems from the nonlinearnature of the dynamics and is already accounted by (38) when the system is spatially homogeneous. The second term inEq. (79) is due to the spatially-extended character of the model and to the fact that traveling fronts propagate with velocityv∗, therefore generating oscillations with a frequency of v∗k. Both contributions superpose and, to sustain a velocity v∗, thedynamics selects a wavenumber ksel according to the relationΩ(ksel) = ω+v∗ksel [116]. Solving this equation for ksel underthe restriction ksel <

√c1/D yields

ksel =√c1

c3√D

(1−

√1+ c23

). (80)

Analytical expressions of the frequency Ω(ksel) and of the wavelength of the spirals, λ = 2π/ksel, can be obtainedimmediately from Eqs. (79) and (80). In fact, the frequency reads

Ω = Ω(qsel) = ω +2c1c3

(1−

√1+ c23

), (81)

and the wavelength is given by

λ =2πc3√D

√c1

(1−

√1+ c23

) . (82)

The expressions Eqs. (80)–(82) have been derived by considering a traveling wave Ansatz as described above. The latterhold in arbitrary dimensions. However, while traveling waves appear in one dimension, in higher dimensions, the genericemerging structures are somewhat different. For instance, rotating spirals arise in two dimensions, while scroll waves arerobust solutions of the CGLE (68) in three spatial dimensions [115]. However, the characteristic properties of these patterns,such as wavelength and frequency, still agree with those of traveling waves. As will be shown in the next section, the resultsfor the spreading velocity and the wavelength actually remain valid even in the presence of noise!

4.4. The role of noise

There is noise due to the stochastic nature of all processes and the discrete character of the individuals. It plays animportant role for the system’s dynamics. In particular it is responsible for the non-equilibrium phase transition from thereactive state into one of the absorbing states. How one has to deal with noise depends on the expected stationary state. If

E. Frey / Physica A 389 (2010) 4265–4298 4289

the stationary state contains many individuals of each species (a macroscopic number) one may perform a Kramers–Moyalexpansion (low noise limit) [58,59]. If, however, the stationary state is some kind of absorbing state where all fluctuationshave died out, a different approach is called for. Then one has to employ a Fock space formulation to map the dynamicsof the master equation to a path integral measure with an appropriate action [117,118]. From this action one may thenderive a Langevin equation for a set of complex density fields whose averages and correlation functions characterize thedynamics towards the absorbing state (strong noise limit). Though both of these approaches look superficially the same, theyare fundamentally different. In a low-noise approximation the noise always turns out to be real but it becomes a complex(imaginary) quantity in the strong noise limit!

Stochastic partial differential equationsAs we have learned from the agent-based simulation the abundances of each species are finite, i.e. there are high copy

numbers for each species, unless one is in the immediate vicinity of the threshold value for the diffusion constant. Hence, toanalyze the dynamics in the parameter regime where one finds biodiversity, we can restrict ourselves to the behavior of thesystem close to the reactive state and use a low noise approximation. The Kramers–Moyal expansion of the underlyingmasterequation leading to a Fokker–Planck equation is detailed in Appendix. Here, we give the equivalent set of Ito stochastic(partial) differential equations (SPDE) (often referred to as Langevin equations) [113]:

∂ta(Er, t) = D∆a(Er, t)+AA[Ea] + CA[Ea]ξA,∂tb(Er, t) = D∆b(Er, t)+AB[Ea] + CB[Ea]ξB,

∂tc(Er, t) = D∆c(Er, t)+AC [Ea] + CC [Ea]ξC , (83)

or in short

∂tEa(Er, t) = D∆Ea(Er, t)+A[Ea] + C[Ea] · Eξ (84)

where ∆ denotes the Laplacian operator. The reaction term A[Ea] derived in the Kramers–Moyal expansion is identical –as it must be – to the corresponding nonlinear drift term in the diffusion-reaction equation, EF [Ea] = A[Ea]. Noise arisesbecause processes are stochastic and population size N is finite. As it turns out (see Appendix), while noise resulting fromthe competition processes (reactions) scales as 1/

√N , noise originating from hopping (diffusion) only scales as 1/N . In

summary, this gives (multiplicative) Gaussian white noise ξi(Er, t) characterized by the correlation matrix

〈ξi(Er, t)ξj(Er ′, t ′)〉 = δijδ(Er − Er ′)δ(t − t ′) (85)

and amplitudes14 depending on the system’s configuration:

CA =1√N

√a(Er, t)

[µ(1− ρ(Er, t))+ σ c(Er, t)

],

CB =1√N

√b(Er, t)

[µ(1− ρ(Er, t))+ σa(Er, t)

],

CC =1√N

√c(Er, t)

[µ(1− ρ(Er, t))+ σb(Er, t)

]. (86)

The comparison of snapshots obtained from lattice simulations with the numerical solutions of the SPDE reveals aremarkable coincidence of both approaches (see Fig. 17). Of course, due to the inherent stochastic nature of the interactingparticle system, the snapshots do not match exactly for each realization. To reach a quantitative assessment on the validityof the SPDE (83) to describe the spatio-temporal properties of the system in the continuum limit, a closer look at correlationfunctions in the steady state is necessary [113,77]. Equal-time correlation functions yield information about the size of theemerging spirals. As an example, consider the spatial correlation of individuals A,

gAA(| Er − Er′|) = 〈a(Er, t)a(Er ′, t)〉 − 〈a(Er, t)〉〈a(Er ′, t)〉 (87)

where the brackets 〈. . .〉 stand for average over all histories. In the steady state, the time dependence drops out and,because of translational and rotational invariance, the correlation function depends only on the separating distance |Er −Er ′|.As can be inferred from Fig. 18 there is excellent agreement between results obtained from lattice simulations and thestochastic partial differential equations. The damped oscillations reflect the underlying spiraling spatial structures, wherethe subpopulations alternate. The autocorrelation function

gAA(|t − t ′|) = 〈a(Er, t)a(Er, t ′)〉 − 〈a(Er, t)〉〈a(Er, t ′)〉 (88)

for a fixed spatial position shows pronounced oscillations with a frequency numerically found to be Ωnum ≈ 0.103 (forσ = µ = 1). These oscillations are simply due to the rotation of the spiral waves, and the frequency is independent of thediffusion constantD, as is to be expected fromEq. (81). Again, there is excellent agreement between agent-based simulationsand a continuum description in terms of stochastic partial differential equations.

14 These amplitudes are related to the noise matrixB in the corresponding Fokker–Planck equations by CCT = B.

4290 E. Frey / Physica A 389 (2010) 4265–4298

Fig. 17. The reactive steady states. Snapshots emerging from simulations of the interacting particle system (37) (top row) and obtained by solving theSPDE (83) (bottom row). Each color (level of gray) represents a different species (black dots denote empty spots). From left to right, the diffusion constantis increased from D = 1 × 10−6 to D = 3 × 10−4 . The latter value is slightly below the critical threshold Dc . The system sizes used in the stochasticsimulations are L = 1000 in the upper two panels, L = 300 for that at bottom, and L = 500 for the other two (middle). The rates are chosen as σ = µ = 1.(For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)Source: Figure adapted from Ref. [77].

0

-0.1

0

0 100 200

0.1

0.2

0 0.1 0.2 0.3

0.05

0.1

(Cor

rela

tion)

gA

A (r

)

(aut

ocor

rela

tion)

gA

A (t

)

(distance) r (time) t

Fig. 18. Correlation functions. The spatial, gAA(r), and temporal, gAA(t), correlation functions are shown as functions of r and t , respectively. Results obtainedfrom stochastic simulations (red circles), numerical solutions of the Langevin equations (SPDE), Eq. (83), (blue squares), and deterministic reaction-diffusionequations (PDE), Eq. (66), (green triangles) are compared for µ = σ = 1. Spatial correlations (left): The spatial correlations decay on a length proportionalto√D; in the graph D = 3× 10−6 was chosen. The results for the deterministic diffusion reaction equations are markedly less damped than those arising

in the stochastic descriptions of the system. Temporal correlations (right): Temporal correlations show oscillations at frequencyΩnum ≈ 0.103; the latteris independent of the value of the diffusion D. These oscillations reflect the rotation of the spiral waves. The results from the SPDE and deterministic PDEhave been obtained using D = 10−5 , while stochastic simulations have been performed on a lattice of length L = 300 with D = 10−4 . (For interpretationof the references to colour in this figure legend, the reader is referred to the web version of this article.)

The spirals’ velocities, wavelengths, and frequenciesIn Section 4.3 we have employed the CGLE (68) to derive analytical results for the emergence of spiral waves, their

stability and their spreading velocity, as well as their wavelength and frequency. Now, we assess the accuracy and validityof these analytical predictions by comparing them with values obtained from the numerical solutions of the SPDE (83).In particular, we will ask whether the results derived for the deterministic diffusion reaction equations still hold in thepresence of noise. Let us first consider the spreading velocity v∗ of the emerging wave fronts. The analytical value, inferredfrom the CGLE, Eq. (77), reads v∗ = 2

√c1D, where c1 = µσ/[2(3µ+σ)] is a coefficient appearing in the CGLE, Eq. (68). The

comparison between this analytical prediction and results obtained from the numerical solution of the SPDE (83) shows niceagreement; see Fig. 19. The functional formof the spreading velocity as a function of the reproduction rate can be understoodas follows. For small values of µ, much lower than the selection rate (µ 1), reproduction is the dominant limiter of thespatio-temporal evolution. In the limit µ→ 0, the front velocity therefore only depends on µ. From dimensional analysis,it follows v∗ ∼

õ, as also confirmed by the analytical solution, Eq. (77). In contrast, if reproduction is much faster than

selection,µ 1, the latter limits the dynamics, and we recover v∗ ∼√σ . In Fig. 19, as σ = 1, this behavior translates into

v∗ being independent ofµ in this limit.While the numerical and analytical results coincide remarkably for low reproductionrates (i.e. µ ≤ 0.3), systematic deviations (≈10%) appear at higher values.In Fig. 20, the analytical estimates for the wavelength λ, Eq. (82), are compared with those obtained from the numerical

solution of the SPDE, Eq. (83). We notice that there is an excellent agreement between analytical and numerical results forthe functional dependence of λ on µ. For low reproduction rates we have λ ∼ 1/

õ, while when reproduction occurs

much faster than selection, the dynamics is independent of µ and λ ∼ 1/√σ . While the functional form of the analytical

prediction is valid, the analytical amplitude estimate forλ exceeds that obtained fromnumerical computations by a constantfactor ≈1.6, taken into account in Fig. 20. This deviation can be attributed to two factors. First, the normal form of thenonlinear dynamics which gives a limit cycle instead of heteroclinic orbits is only approximate. Second, the noise-induced

E. Frey / Physica A 389 (2010) 4265–4298 4291

1

0.5

0.25

0.125

(vel

ocity

) υ*

/ D

0.008 0.04 0.2 1 5 25

(reproduction rate) μ

Fig. 19. Spreading velocity. The velocity v∗ of the spreading front (rescaled by a factor√D) is displayed as a function of the reproduction rate µ. The time

scale is set by setting σ = 1. The analytical predictions (full red line), Eq. (77), obtained from the CGLE, are compared with numerical results (black circles),obtained from the numerical solutions of the SPDE, Eq. (83).Source: Figure adapted from Ref. [73].

D(w

avel

engt

h) λ

/

32

64

128

0.008 0.04 0.2 1 5 25

(reproduction rate) μ

Fig. 20. The spirals’ wavelength. The functional dependence of the wavelength λ on the rate µ (with σ = 1) is shown for the numerical solutions ofthe SPDE, Eq. (83), and compared to the analytical predictions (red line). The latter is obtained from the CGLE and is given by Eq. (82). It differs from thenumerics by a factor of 1.6; see text. Adjusting this factor, c.f. the blue line, the functional dependence is seen to agree very well with numerical results.(For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)Source: Figure adapted from Ref. [73].

spatial coupling between different regions yields a kind of ‘‘stochastic limit cycle’’ [72], whose features seem to be capturedremarkably well by the renormalized approximate limit cycle obtained from the normal form analysis.

Scaling relation and critical mobilityFinally, we want to discuss the mechanism driving the transition from a stable coexistence to extinction at the critical

mobility Dc . To address this issue, first note that varying the mobility induces a scaling effect, as illustrated in Fig. 14. Infact, increasing the diffusion constant D is equivalent to zooming into the system. The diffusion constant enters the noisyreaction-diffusion equations through a diffusive term D∆, where ∆ is the Laplace operator involving second-order spatialderivatives. Such a term is left invariant when D is multiplied by a factor α while the spatial coordinates are rescaled by

√α.

Hence, a rescaling D→ αD translates in a magnification of the system’s characteristic size by a factor√α. This implies that

the spirals’ wavelength λ is proportional to√D up to the critical Dc .

When the spirals have a critical wavelength λc , associated with the mobility Dc , these rotating patterns outgrow thesystem size which results in the loss of biodiversity. Measuring length in lattice size units and time in units of σ = 1, onenumerically finds a universal value λc = 0.8 ± 0.05, independent of the reproduction strength µ [73]. This allows us todetermine the dependence of the critical mobility Dc on the parameters of the system, simply by employing the scalingrelation, λ(µ,D) ∼

√D:

Dc(µ) =(

λc

λ(µ,D)

)2D. (89)

Upon using the value for the wavelength λ(µ,D) obtained from numerical simulations this yields to the phase diagramshown in Fig. 21. The solid red line shows the analytical prediction for λ(µ,D) derived from the complex Ginzburg–Landauequation and renormalized as described above. This figure corroborates the validity of the various approaches (SPDE, latticesimulations and CGLE),which all lead to the samephase diagramwhere the biodiverse and the uniformphases are identified.It suggests that the approach outlined in this sectionmay be employed quite generally to investigate the stochastic nonlineardynamics of spatial games.

4292 E. Frey / Physica A 389 (2010) 4265–4298

Fig. 21. Phase diagram. The critical diffusion constant Dc as a function of the reproduction rate µ yields a phase diagram with a phase where biodiversityis maintained as well as a uniform one where two species go extinct and only one survives. The time unit is set by σ = 1. Results from lattice simulations(blue circles with error bars), and the stochastic PDE (black dots, black lines are a guide to the eye) are compared with the (renormalized) analytical resultobtained from the complex Ginzburg–Landau equation (red line). Varying the reproduction rate, two different regimes emerge. If µ is much smaller thanthe selection rate, i.e. µ σ , reproduction is the dominant limiter of the temporal development. In this case, there is a linear relation with the criticalmobility, i.e.Dc ∼ µ, as follows from dimensional analysis. In the opposite case, if reproduction occursmuch faster than selection (µ σ ), the latter limitsthe dynamics and Dc depends linearly on σ , i.e. Dc ∼ σ . Here, as σ = 1 is kept fixed (time-scale unit), this behavior reflects in the fact that Dc approachesa constant value for µ σ . (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)Source: Figure adapted from Ref. [73].

4.5. Summary and discussion

The main point from a phenomenological point of view is that individuals’ mobility as well as intrinsic noise have acrucial influence on the self-formation of spatial patterns. A low exchange rate between neighboring individuals leads tothe formation of small and irregular patterns. In this case the coexistence of all subpopulations is preserved and the ensuingpatterns are mainly determined by stochastic effects. Larger exchange rates (yet of same order as the reaction rates) yieldthe formation of relatively regular spiral waves whose rotational nature is reminiscent of the cyclic and out-of-equilibriumensuing kinetics. In fact, the three subpopulations endlessly, and in turn, hunt each other. The location and density of thespirals’ vortices is either determined by initial spatial inhomogeneities, if these take a pronounced shape, or by stochasticity.In the latter case, internal noise leads to an entanglement of many small spirals and a universal vortex density of about0.5 per square wavelength. Increasing the diffusion rate (i.e. individuals’ mobility), the typical size of the spiral waves risesup to a critical value. When that threshold is reached, the spiral patterns outgrow the two-dimensional system and there isonly one surviving subpopulation covering the system uniformly [73].The main point from a theoretical point of view is that the language of interacting particles enabled us to devise a proper

treatment of the stochastic spatially-extended system and to reach a comprehensive understanding of the resulting out-of-equilibrium and nonlinear phenomena. In particular, we have illustrated how spatio-temporal properties of the system,formulated as a stochastic agent-based model, can be aptly described in terms of stochastic partial differential equations(SPDE), i.e. diffusion-reaction equations with noise emerging from the stochasticity of the interaction between individuals.Numerical solutions of the SPDE give the same statistics for the non-equilibrium steady states as the lattice simulations, withthe emerging spiral waves characterized in both cases by the same wavelength, overall sizes and frequency. We have alsostudied the influence of stochasticity on the properties of the coexistence state and its spatio-temporal structure, and foundthat the deterministic diffusion reaction equation (PDE) still yields spiral patterns. In addition,we found thatwavelength andfrequency of the spirals are not affected by internal noise. However, there are major differences between the stochastic anddeterministic descriptions of the system. One of the most important is the influence of the initial conditions. If initial spatialinhomogeneities are larger than the noise level, or if noise is absent as in the deterministic descriptions, these initial spatialstructures determine the position of the spirals’ vortices. In this situation, the system ‘‘memorizes’’ its initial state, and thelatter crucially influences the overall size of the emerging spiral waves. In contrast, for rather homogeneous initial densities(at values of the unstable reactive fixed point), the patterns emerging from the stochastic descriptions (lattice simulationsand SPDE) are caused by noise and characterized by a universal density of 0.5 spiral vortices per square wavelength.Employing a mapping of the diffusion-reaction equation onto the reactive manifold of the nonlinear dynamics it turned

out that the dynamics of the coexistence regime is in the same ‘‘universality class’’ as the complexGinzburg–Landau equation(CGLE). This fact reveals the generality of the phenomena discussed in this chapter. In particular, the emergence of anentanglement of spiral waves in the coexistence state, the dependence of spirals’ size on the diffusion rate, and the existenceof a critical value of the diffusion abovewhich coexistence is lost are robust phenomena. Thismeans that they do not dependon the details of the underlying spatial structure: While, for specificity, we have (mostly) considered square lattices, othertwo-dimensional topologies (e.g. hexagonal or other lattices) will lead to the same phenomena, too. Also the details of thecyclic competition have no qualitative influence, as long as the underlying rate equations exhibit an unstable coexistence

E. Frey / Physica A 389 (2010) 4265–4298 4293

Fig. 22. Snapshots of the biodiverse state for D = 1 × 10−5 . (a), For large rates σ , entangled and stable spiral waves form. (b), A convective (Eckhaus)instability occurs at σE ≈ 2; below this value, the spiral patterns blur. (c), At the bifurcation point σ = 0, only very weak spatial modulations emerge; wehave amplified them by a factor two for better visibility. The snapshots stem from numerical solution of an appropriate SPDE with initially homogeneousdensities a = b = c = 1/4.

fixed point and can be recast in the universality class of the Hopf bifurcations. It remains to be explored what kind ofmathematical structure corresponds to a broader range of game-theoretical problems.In this chapter, we have mainly focused on the situation where the exchange rate between individuals is sufficiently

high, which leads to the emergence of regular spirals in two dimensions. However, when the exchange rate is low (orvanishes), we have seen that stochasticity strongly affects the structure of the ensuing spatial patterns. In this case, the(continuum) description in terms of SPDE breaks down. In this situation, the quantitative analysis of the spatio-temporalproperties of interacting particle systems requires the development of other analytical methods, e.g. relying on fieldtheoretic techniques [108]. Fruitful insights into this regime have already been gained by pair approximations or larger-cluster approximations [119–121,91]. The authors of these studies investigated a set of coupled nonlinear differentialequations for the time evolution of the probability to find a cluster of a certain size in a particular state. While such anapproximation improves when large clusters are considered, unfortunately the effort for solving their coupled equations ofmotion also drastically increases with the size of the clusters. In addition, the use of these cluster mean-field approachesbecomes problematic in the proximity of phase transitions (near an extinction threshold) where the correlation lengthdiverges. Investigations along these lines represent a major future challenge in the multidisciplinary field of complexityscience.The cyclic rock–paper-scissor model as discussed in this section can be generalized in manifold ways. As already noted,

the model with asymmetric rates turns out to be in the same universality class as the one with symmetric rates [78].Qualitative changes in the dynamics, however, emerge when the interaction network between the species is changed. Forexample, consider a system where each agent can interact with its neighbors according to the following scheme:

AB1−→ AA

BC1−→ BB

CA1−→ CC

(90)

ABσ−→ A

BCσ−→ B

CAσ−→ C

(91)

Aµ−→ AA

Bµ−→ BB

Cµ−→ CC .

(92)

Reactions (90) describe direct dominance in aMoran-likemanner, where an individual of one species is consumed by anotherfrom a more predominant species, and the latter immediate reproduces. Cyclic dominance appears as A consumes B andreproduces, while B preys on C and C feeds on A in turn. Reactions (91) encode some kind of toxicity, where one species killsanother, leaving an empty site . These reactions occur at a rate σ , and are decoupled from reproduction, Eq. (92), whichhappens at a rateµ. Note that reactions (90) and (92) describe two different mechanisms of reproduction, both of which areimportant for ecological systems: In (90), an individual reproduces when having consumed a prey, due to thereby increasedfitness. In contrast, in reactions (92) reproduction depends solely on the availability of empty space. As can be inferred fromFig. 22 the spatio-temporal patterns sensitively depend on the strength σ of the toxicity effect. Actually, as can be shownanalytically [122], there is an Eckhaus instability, i.e., a convective instability: a localized perturbation grows but travelsaway. The instabilities result in the blurring seen in Fig. 22.

4294 E. Frey / Physica A 389 (2010) 4265–4298

It remains to be explored how more complex interaction networks with an increasing number of species andwith different types of competition affect the spatio-temporal pattern formation process. Research along these lines issummarized in a recent review [91].

5. Conclusions and outlook

In these lecture notes we have given an introduction into evolutionary game theory. The perspective we have taken wasthat starting from agent-based models, the dynamics may be formulated in terms of a hierarchy of theoretical models. First,if the population size is large and the population is well-mixed, a set of ordinary differential equations can be employedto study the system’s dynamics and ensuing stationary states. Game theoretical concepts of ‘‘equilibria’’ then map to‘‘attractors’’ of the nonlinear dynamics. Setting up the appropriate dynamic equations is a non-trivial matter if one is aimingat a realistic description of a biological system. For instance, as nicely illustrated by a recent study on yeast [22], a linearreplicator equation might not be sufficient to describe the frequency-dependence of the fitness landscape. We supposethat this is rather the rule than the exception for biological systems such as microbial populations. Second, for well-mixedbut finite populations, one has to account for stochastic fluctuations. Then there are two central questions: (i) What is theprobability of a certain species to go extinct or to become fixated in a population? (ii) How long does this process take? Thesequestions have to be answered by employing concepts from the theory of stochastic processes. Since most systems haveabsorbing states, we have found it useful to classify the stability of a given dynamic system according to the scaling of theexpected extinction time with population size. Third and finally, taking into account the finite mobility of individuals in anexplicit spatial model, a description in terms of stochastic partial differential equations becomes necessary. These Langevinequations describe the interplay between reactions, diffusion and noise, which give rise to a plethora of new phenomena. Inparticular, spatio-temporal patterns or, more generally, spatio-temporal correlations, may emerge which can dramaticallychange the ecological and evolutionary stability of a population. For non-transitive dynamics, like the rock–scissors–papergame played by some microbes [1], there is a mobility threshold which demarcates regimes of maintenance and loss ofbiodiversity [73]. Since, for the rock–scissors–paper game, the nature of the patterns and the transition was encoded in theflow of the nonlinear dynamics on the reactive manifold, one might hope that a generalization of the outlined approachmight be helpful in classifying a broader range of game-theoretical problems and identify some ‘‘universality classes’’.What are the ideal experimental model systems for future studies?We think thatmicrobial populationswill play amajor

role since interactions between different strains can be manipulated in a multitude of ways. In addition, experimental toolslike microfluidics and various optical methods allow for easy manipulation and observation of these systems, from the levelof an individual up to the level of a whole population. Bacterial communities represent complex and dynamic ecologicalsystems. They appear in the form of free-floating bacteria as well as biofilms in nearly all parts of our environment, and arehighly relevant for humanhealth anddisease [123]. Spatial patterns arise fromheterogeneities of the underlying ‘‘landscape’’or self-organized by the bacterial interactions, and play an important role in maintaining species diversity [27]. Interactionscomprise, amongst others, competition for resources and cooperation by sharing of extracellular polymeric substances.Another aspect of interactions is chemical warfare. As we have discussed, some bacterial strains produce toxins such ascolicin, which acts as a poison to sensitive strains, while other strains are resistant [1]. Stable coexistence of these differentstrains arises when they can spatially segregate, resulting in self-organizing patterns. There is a virtually inexhaustiblecomplexity in the structure and dynamics of microbial populations. The recently proposed term ‘‘socio-microbiology’’ [124]expresses this notion in a most vivid form. Investigating the dynamics of those complex microbial populations in achallenging interdisciplinary endeavor, which requires the combination of approaches from molecular microbiology,experimental biophysical methods and theoretical modeling. The overall goal would be to explore how collective behavioremerges and is maintained or destroyed in finite populations under the action of various kinds of molecular interactionsbetween individual cells. Both communities, biology as well as physics, will benefit from this line of research.Stochastic interacting particle systems are a fruitful testing ground for understanding generic principles in non-

equilibrium physics. Here biological systems have been a wonderful source of inspiration for the formulation of newmodels. For example, MacDonald [125] looking for a mathematical description for mRNA translation into proteins managedby ribosomes, which bind to the mRNA strand and step forward codon by codon, formulated a non-equilibrium one-dimensional transport model, nowadays known as the totally asymmetric simple exclusion process. This model has ledto significant advances in our understanding of phase transitions and the nature of stationary states in non-equilibriumsystems [126,127]. Searching for simplified models of epidemic spreading without immunization Harris [128] introducedthe contact process. In this model infectious individuals can either heal themselves or infect their neighbors. As a functionof the infection and recovery rate it displays a phase transition from an active to an absorbing state, i.e. the epidemic diseasemay either spread over the whole population or vanish after some time. The broader class of absorbing-state transitionshas recently been reviewed [129]. Another well studied model is the voter model where each individual has one of twoopinions and may change it by imitation of a randomly chosen neighbor. This process mimics in a naive way opinionmaking [130]. Actually, it was first considered by Clifford and Sudbury [131] as a model for the competition of speciesand only later named the voter model by Holley and Liggett [132]. It has been shown rigorously that on a regular latticethere is a stationary state where two ‘‘opinions’’ coexist in systems with spatial dimensions where the random walk isnot recurrent [133,130]. A question of particular interest is how opinions or strategies may spread in a population. Inthis context it is important to understand the coarsening dynamics of interacting agents. For a one-dimensional version

E. Frey / Physica A 389 (2010) 4265–4298 4295

of the the rock–paper–scissors game, Frachebourg and collaborators [134,135] have found that starting from some randomdistribution, the species organize into domains that undergo (power law) coarsening until finally one species takes overthe whole lattice. Including mutation, the coarsening process is counteracted and by an interesting interplay betweenequilibrium and non-equilibrium processes a reactive stationary state emerges [136]. The list of interesting examples, ofcourse, continues and one may hope that in the future there will be an even more fruitful interaction between biologicallyrelevant processes and basic research in non-equilibrium dynamics.

Acknowledgements

I am indebted to my students, Jonas Cremer, Alexander Dobrinevsky, Anna Melbinger, Tobias Reichenbach, SteffenRulands, Anton Winkler, and postdoctoral fellow, Mauro Mobilia, with whom I had the pleasure to work on game theory.They have contributed with a multitude of creative ideas and through many insightful discussions have shaped myunderstanding of the topic. Financial support of the German Excellence Initiative via the program ‘‘Nanosystems InitiativeMunich’’ and the German Research Foundation via the SFB TR12 ‘‘Symmetries and Universalities in Mesoscopic Systems’’ isgratefully acknowledged.

Appendix. Kramers–Moyal expansion

Reaction terms

Since noise terms stemming from the reactions (37) are local, theymay be derived considering the stochastic non-spatialsystem, i.e. thewell-mixed system. Denoting Ea = (a, b, c) the frequencies of the three subpopulations A, B, and C , theMasterequation for the time-evolution of the probability P(Ea, t) of finding the system in state Ea at time t reads

∂tP(Ea, t) =∑δEa

[P(Ea+ δEa, t)WEa+δEa→Ea − P(Ea, t)WEa→Ea+δEa

]. (A.1)

Hereby,WEa→Ea+δEa denotes the transition probability from state Ea to the state Ea+δEawithin one time step; summation extendsover all possible changes δEa. The relevant changes δEa in the densities result from the basic reactions (37); as an example,concerning the change in the density of the subpopulation A, it reads δa = 1/N in the reaction A

µ−→ AA, δa = −1/N

in the reaction CAσ−→ C, and zero in the remaining ones. Concerning the rates for these reactions, we choose the unit of

time such that, on average, every individual reacts once per time step. The transition rates resulting from the reactions (37)then readW = Nσac for the reaction CA

σ−→ C andW = Nµa(1 − a − b − c) for A

µ−→ AA. Transition probabilities

associated with all other reactions (37) follow analogously.The Kramers–Moyal expansion [137] of the Master equation is an expansion in the increment δEa, which is proportional

to N−1. Therefore, it may be understood as an expansion in the inverse system size N−1. To second order in δEa, it yields the(generic) Fokker–Planck equation [137]:

∂tP(Ea, t) = −∂i[Ai(Ea)P(Ea, t)] +12∂i∂j[Bij(Ea)P(Ea, t)]. (A.2)

Hereby, the summation convention implies sums carried over the indices i, j ∈ A, B, C. According to the Kramers–Moyalexpansion, the quantitiesAi andBij read [137]

Ai(Ea) =∑δEa

δaiW(Ea→ Ea+ δEa),

Bij(Ea) =∑δEa

δaiδajW(Ea→ Ea+ δEa). (A.3)

Note thatB is symmetric.As an example, we now present the calculation ofAA(Ea). The relevant changes δa result from the reactions A

µ−→ AA

and CAσ−→ C. The corresponding rates as well as the changes in the density of subpopulation A have been given above;

together, we obtainAA(Ea) = µa(1−a−b−c)−σac. The other quantities are computed analogously; eventually, one finds

AA(Ea) = µa(1− a− b− c)− σac,AB(Ea) = µb(1− a− b− c)− σab,

AC (Ea) = µc(1− a− b− c)− σbc, (A.4)

and

BAA(Ea) = N−1 [µa(1− a− b− c)+ σac] ,BBB(Ea) = N−1 [µb(1− a− b− c)+ σab] ,

BCC (Ea) = N−1 [µc(1− a− b− c)+ σbc] . (A.5)

4296 E. Frey / Physica A 389 (2010) 4265–4298

The well-known correspondence between Fokker–Planck equations and Itô calculus [59] implies that (A.2) is equivalent tothe following set of Itô stochastic differential equations:

∂ta = AA + CAAξA,

∂tb = AB + CBBξB,

∂tc = AC + CCCξC . (A.6)

Hereby, the ξi denotes (uncorrelated) Gaussian white noise terms. The matrix C is defined from B via the relation CCT =B [59]. AsB is diagonal, we may choose C diagonal as well, with the square roots of the corresponding diagonal entries ofB on the diagonal.

Diffusion term

Diffusion couples two nearest neighboring lattice sites Er and Er ′. The rate for an individual A to hop from Er to Er ′ is givenby εz−1a(Er)[1− a(Er ′)] (for simplicity, we drop the time-dependence). Together with the reverse process, i.e. hopping fromsite Er ′ to Er , this yields the non-diagonal part ofB(Er, Er ′) (see e.g. Ref. [137]):

B(Er, Er ′ 6= Er) = −ε

Nz

a(Er)[1− a(Er ′)] + a(Er ′)[1− a(Er)]

. (A.7)

Similarly, the diagonal entries ofB read

B(Er, Er) =ε

Nz

∑n.n.Er ′′

a(Er)[1− a(Er ′′)] + a(Er ′′)[1− a(Er)]

, (A.8)

where the sum runs over all nearest neighbors (n.n.) Er ′′ of the site Er . It follows from these expressions that

B(Er, Er ′) =ε

Nz

∑n.n.Er ′′

(δEr,Er ′ − δEr ′,Er ′′)×a(Er)[1− a(Er ′′)] + a(Er ′′)[1− a(Er)]

.

In the continuum limit, with δr → 0, we use the fact that δEr,Er ′ → δrdδ(Er − Er ′) and obtain

B(Er, Er ′) =ε

Nzδrd

d∑±,i=1

[δ(Er − Er ′)− δ(Er ± δrEei − Er ′)

] a(Er)[1− a(Er ± δrEei)] + a(Er ± δrEei)[1− a(Er)]

.

As above, we expand δ(Er ± δrEei − Er ′) and a(Er ± δrEei) to second order and observe that only quadratic terms in δr do notcancel. With ε = DdN2/d and δr = N−1/d, we thus find:

B(Er, Er ′) =DN2∂Er∂Er ′

[δ(Er − Er ′)a(Er)(1− a(Er))

]. (A.9)

The noise matrix B of the Fokker–Planck equation associated with the exchange processes therefore scales as N−2. In thecorresponding SPDE, the contribution to noise of the exchange processes scales as N−1.

References

[1] B. Kerr, M.A. Riley, M.W. Feldman, B.J.M. Bohannan, Local dispersal promotes biodiversity in a real-life game of rock–paper–scissors, Nature 418(2002) 171–174.

[2] M.J. Osborne, An Introduction to Game Theory, first edition, Oxford University Press, 2004.[3] J.F. Nash, Equilibrium points in n-person games, Proc. Natl. Acad. Sci. USA 36 (1950) 48–49.[4] R.M. Dawes, Social dilemmas, Ann. Rev. Psychol. 31 (1980) 169–193.[5] R. Axelrod, W.D. Hamilton, The evolution of cooperation, Science 211 (1981) 1390–1396.[6] J. Maynard Smith, G.R. Price, The logic of animal conflict, Nature 246 (1973) 15–18.[7] J. Maynard Smith, Evolution and the Theory of Games, Cambridge University Press, 1982.[8] A.J. Lotka, Undamped oscillations derived from the law of mass action, J. Amer. Chem. Soc. 42 (1920) 1595–1599.[9] V. Volterra, Variazioni e fluttuazioni del numero d’individui in specie animali conviventi, Mem. Accad. Lincei 2 (1926) 31.[10] J. Hofbauer, K. Sigmund, Evolutionary Games and Population Dynamics, Cambridge University Press, 1998.[11] J. Maynard Smith, Evolution and the Theory of Games, Cambridge University Press, Cambridge, 1982.[12] S.H. Strogatz, Nonlinear Dynamics and Chaos, Westview, 1994.[13] L. Hall-Stoodley, J.W. Costerton, P. Stoodley, Bacterial biofilms: from the natural environment to infectious diseases, Nat. Rev. Microbiol. 2 (2004) 95.[14] T.J. Battin, W.T. Sloan, S. Kjelleberg, H. Daims, I.M. Head, T.P. Curtis, L. Eberl, Microbial landscapes: new paths to biofilm research, Nat. Rev. Microbiol.

5 (2007) 76–81.[15] G.J. Velicer, Social strife in the microbial world, Trends Microbiol. 11 (2003) 330–337.[16] J.B. Xavier, K.R. Foster, Cooperation and conflict in microbial biofilms, Proc. Natl. Acad. Sci. USA 104 (2007) 876–881.[17] C.D. Nadell, J.B. Xavier, S.A. Levin, K.R. Foster, The evolution of quorum sensing in bacterial biofilms, PLoS Biol. 6 (2008) e14.[18] D. Greig, M. Travisano, The prisoner’s dilemma and polymorphism in yeast suc genes, Proc. Roy. Soc. Lond. B 271 (2004) S25–26.[19] A. Buckling, F. Harrison, M. Vos, M.A. Brockhurst, A. Gardner, S.A. West, A. Griffin, Siderophore-mediated cooperation and virulence in pseudomonas

aeruginosa, FEMS Microbiol. Ecol. 62 (2) (2007) 135–141.[20] R.L. Trivers, The evolution of reciprocal altruism, Quart. Rev. Biol. 46 (1971) 35.[21] C.M. Waters, B.L. Bassle, Quorum sensing: cell-to-cell communication in bacteria, Ann. Rev. Cell Dev. Biol. 21 (2005) 319–346.[22] J. Gore, H. Youk, A. van Oudenaarden, Snowdrift game dynamics and facultative cheating in yeast, Nature 459 (2009) 253–256.

E. Frey / Physica A 389 (2010) 4265–4298 4297

[23] J.S. Chuang, O. Rivoire, S. Leibler, Simpson’s paradox in a synthetic microbial system, Science 323 (2009) 272–275.[24] R. Durrett, S. Levin, Allelopathy in spatially distributed populations, J. Theoret. Biol. 185 (1997) 165–171.[25] R. Durrett, S. Levin, Spatial aspects of interspecific competition, Theor. Pop. Biol. 53 (1998) 30–43.[26] T.L. Czárán, R.F. Hoekstra, L. Pagie, Chemical warfare between microbes promotes biodiversity, Proc. Natl. Acad. Sci. USA 99 (2002) 786–790.[27] J.B. Xavier, E. Martinez-Gracia, K.R. Foster, Social evolution of spatial patterns in bacterial biofilms:when conflict drives disorder, Am. Nat. 174 (2009)

1–12.[28] R. Axelrod, The Evolution of Cooperation, Basic Books, New York, 1984.[29] M.A. Nowak, Five rules for the evolution of cooperation, Science 314 (2006) 1560.[30] T. Yamagishi, The provision of a sanctioning system as a public good, J. Pers. Soc. Psychol. 51 (1986) 110–116.[31] W.D. Hamilton, Narrow Roads of Gene Land: Evolution of Social Behaviour, Oxford University Press, Oxford, 1996.[32] S.A. West, A.S. Griffin, A. Gardner, S.P. Diggle, Social evolution theory for microorganisms, Nat. Rev. Microbiol. 4 (8) (2006) 597–607.[33] J.F. Crow, M. Kimura, An Introduction to Population Genetics, Blackburn Press, 2009.[34] J. Cremer, T. Reichenbach, E. Frey, The edge of neutral evolution in social dilemmas, New J. Phys. 11 (2009) 093029.[35] E. Ben-Jacob, I. Cohen, H. Levine, Cooperative self-organization of microorganisms, Adv. Phys. 49 (2000) 395–554.[36] O. Hallatschek, P. Hersen, S. Ramanathan, D.R. Nelson, Genetic drift at expanding frontiers promotes gene segregation, Proc. Natl. Acad. Sci. USA 104

(2007) 19926–19930.[37] C.J. Ingham, E.B. Jacob, Swarming and complex pattern formation in paenibacillus vortex studied by imaging and tracking cells, BMC Microbiol. 8

(2008) 36.[38] T.H. Henrici, The Biolology of Bacteria: The Bacillaceae, third edition, Heath and Company, 1948.[39] E. Ben-Jacob, I. Cohen, I. Golding, D.L. Gutnick, M. Tcherpakov, D. Helbing, I.G. Ron Open, Bacterial cooperative organization under antibiotic stress,

Phys. A 282 (2000) 247–282.[40] M. Matsushita, H. Fujikawa, Diffusion-limited growth in bacterial colony formation, Phys. A 168 (1990) 498–506.[41] R. Rudner, O. Martsinkevich, W. Leung, E.D. Jarvis, Classification and genetic characterization of pattern-forming bacilli, Mol. Microbiol. 27 (1998)

687–703.[42] M. Eisenbach, Functions of the flagellar modes of rotation in bacterial motility and chemotaxis, Mol. Microbiol. 4 (1990) 161–167.[43] J. Henrichsen, Gliding and twitching motility of bacteria unaffected by Cytochalasin B, A. Pathol. Microbiol. Scand. B B 80 (1972) 623.[44] J. Henrichsen, L.O. Froholm, K. Bovre, Studies on bacterial surface translocation. 2. Correlation of twitchingmotility and fimbriation in bolony variants

of Moraxella-Nonliquefaciens, M-Bovis, and M-Kingii, A. Pathol. Microbiol. Scand. B B 80 (1972) 445.[45] S. Park, P.M. Wolanin, E.A. Yuzbashyan, H. Lin, N.C. Darnton, J.B. Stock, P. Silberzan, R. Austin, Influence of topology on bacterial social interaction.

Proc. Natl. Acad. Sci. USA 100 13910–13915.[46] O. Hallatschek, D.R. Nelson, Gene surfing, Theor. Pop. Biol. 73 (2008) 158.[47] H.H. McAdams, A. Arkin, Trends Genet. 15 (1999) 65.[48] M. Kaern, T.C. Elston, W.J. Blake, J.J. Collins, Stochasticity in gene expression: from theories to phenotypes, Nat. Rev. Microbiol. 6 (2005) 451–462.[49] J.W. Veening, W.K. Smits, O.P. Kuipers, Bistability, epigenetics, and bet-hedging in bacteria, Annu. Rev. Microbiol. 62 (2008) 193–210.[50] W.K. Smits, O.P. Kuipers, J.W. Veening, Phenotypic variation in bacteria: the role of feedback regulation, Nat. Rev. Microbiol. 4 (2006) 259–271.[51] D. Dubnau, R. Losick, Bistability in bacteria, Mol. Microbiol. 61 (2006) 564–572.[52] M. Leisner, K. Stingl, E. Frey, B. Maier, Stochastic switching to competence, Curr. Op. Microbiol. 11 (2008) 553–559.[53] M. Delbrück, Statistical fluctuations in autocatalytic reactions, J. Chem. Phys. 8 (1940) 120.[54] A. Traulsen, J.C. Claussen, C. Hauert, Coevolutionary dynamics: from finite to infinite populations, Phys. Rev. Lett. 95 (2005) 238701.[55] T. Reichenbach, M. Mobilia, E. Frey, Coexistence versus extinction in the stochastic cyclic Lotka–Volterra model, Phys. Rev. E 74 (2006) 051907.[56] A. Traulsen, J.C. Claussen, C. Hauert, Coevolutionary dynamics in large, but finite populations, Phys. Rev. E 74 (2006) 011901.[57] J. Cremer, T. Reichenbach, E. Frey, Anomalous finite-size effects in the battle of the sexes, Eur. Phys. J. B 63 (2008) 373–380.[58] N.G. van Kampen, Stochastic Processes in Physics and Chemistry, 1st edition, North Holland Publishing Company, 1981.[59] C.W. Gardiner, Handbook of Stochastic Methods, Springer, Berlin, 2007.[60] P.A. Moran, The Statistical Processes of Evolutionary Theory, Clarendon Press Oxford, Oxford, 1964.[61] S. Wright, The differential equation of the distribution of gene frequencies, Proc. Natl. Acad. Sci. USA 31 (1945) 382–389.[62] S. Wright, Evolution and the Genetics of Populations, Chicago University Press, 1969.[63] W.J. Ewens, Mathematical Population Genetics, 2nd edition, Springer, New York, 2004.[64] M. Kimura, Genetic variability maintained in a finite population due to mutational production of neutral and nearly neutral isoalleles, Genet. Res.,

Camb. 11 (1969) 247–269.[65] M. Kimura, The Neutral Theory of Molecular Evolution, Cambridge University Press, 1983.[66] P.W. Jones, P. Smith, Stochastic processes, 1st edition, Arnold, 2001.[67] T. Antal, I. Scheuring, Fixation of strategies for an evolutionary game in finite populations, Bull. Math. Biol. 68 (2006) 1923.[68] C. Taylor, Y. Iwasa, M.A. Nowak, A symmetry of fixation times in evolutionary dynamics, J. Theoret. Biol. 243 (2006) 245–251.[69] M. Ifti, B. Bergersen, Survival and extinction in cyclic and neutral three-species systems, Eur. Phys. J. E 10 (2003) 241–248.[70] M. Ifti, B. Bergersen, Crossover behaviour of 3-species systems with mutations or migrations, Eur. Phys. J. B 37 (2004) 101–107.[71] M. Berr, T. Reichenbach, M. Schottenloher, E. Frey, Zero-one survival behavior of cyclically competing species, Phys. Rev. Lett. 102 (2009) 048102.[72] A. Dobrinevski, E. Frey, Extinction in neutrally stable stochastic Lotka–Volterra models (Manuscript in preparation).[73] T. Reichenbach, M. Mobilia, E. Frey, Mobility promotes and jeopardizes biodiversity in rock–paper–scissors games, Nature 448 (2007) 1046–1049.[74] A.J. Nicholson, An outline of the dynamics of animal populations, Aust. J. Zool. 2 (1954) 9–65.[75] R.M. May, W.J. Leonard, Nonlinear aspects of competition between species, SIAM J. Appl. Math 29 (1975) 243–253.[76] S. Wiggins, Introduction to Applied Nonlinear Dynamical Systems and Chaos, first edition, Springer, 1990.[77] T. Reichenbach, M. Mobilia, E. Frey, Self-organization of mobile populations in cyclic competition, J. Theoret. Biol. 254 (2008) 368–383.[78] M. Peltomaki, M. Alava, Three- and four-state rock–paper–scissors games with diffusion, Phys. Rev. E 78 (2008) 031906.[79] R.M. May, Stability and Complexity in Model Ecosystems, 2nd edition, Princeton Univ. Press, Princeton, 1974.[80] J.D. Murray, Mathematical Biology, 3rd edition, Springer Verlag, 2002.[81] A.M. Turing, The chemical basis of morphogenesis, Philos. Trans. R. Soc. Lond. B 237 (1952) 37–72.[82] M.A. Nowak, R.M. May, Evolutionary games and spatial chaos, Nature 359 (1992) 826–829.[83] M.P. Hassell, H.N. Comins, R.M. May, Spatial structure and chaos in insect population dynamics, Nature 353 (1991) 255–258.[84] M.P. Hassell, H.N. Comins, R.M. May, Species coexistence and self-organizing spatial dynamics, Nature 370 (1994) 290–292.[85] B. Blasius, A. Huppert, L. Stone, Complex dynamics and phase synchronization in spatially extended ecological systems, Nature 399 (1999) 354–359.[86] A.A. King, A. Hastings, Spatial mechanism for coexistence of species sharing a common natural enemy, Theor. Pop. Biol. 64 (2003) 431–438.[87] G. Szabo, C. Hauert, Phase transitions and volunteering in spatial public goods games, Phys. Rev. Lett. 89 (2002) 118101.[88] C. Hauert, M. Doebeli, Spatial structure often inhibits the evolution of cooperation in the snowdrift game, Nature 428 (2004) 643–646.[89] T.M. Scanlon, K.K. Caylor, I. Rodriguez-Iturbe, Positive feedbacks promote power-law clustering of Kalahari vegetation, Nature 449 (2007) 209–212.[90] S. Kefi, M. Rietkerk, C.L. Alados, Y. Pueyo, V.P. Papanastasis, A. ElAich, P.C. de Ruiter, Spatial vegetation patterns and imminent desertification in

mediterranean arid ecosystems, Nature 449 (2007) 213–217.[91] G. Szabó, G. Fath, Evolutionary games on graphs, Phys. Rep. 446 (2007) 97–216.[92] M. Perc, A. Szolnoki, G. Szabó, Cyclical interactions with alliance-specific heterogeneous invasion rates, Phys. Rev. E 75 (2007) 052102.[93] M.A. Nowak, Evolutionary Dynamics, 1st edition, Belknap Press, 2006.[94] A.N. Zaikin, A.M. Zhabotinsky, Concentration wave propagation in two-dimensional liquid-phase self-oscillating system, Nature 225 (1970) 535.

4298 E. Frey / Physica A 389 (2010) 4265–4298

[95] R. Kapral, K. Showalter (Eds.), Chemical Waves and Patterns, Part One, Kluwer Academic Publishers, Dordrecht, 1995.[96] B.T. Grenfell, O.N. Bjornstad, J. Kappey, Travelling waves and spatial hierarchies in measles epidemics, Nature 414 (2001) 716–723.[97] D.A.T. Cummings, R.A. Irizarry, N.E. Huang, T.P. Endy, A. Nisalak, K. Ungchusak, D.S. Burke, Travellingwaves in the occurrence of dengue haemorrhagic

fever in thailand, Nature 427 (2004) 344–347.[98] C.B. Muratov, E. Vanden-Eijnden, E. Weinan, Noise can play an organizing role for the recurrent dynamics in excitable media, Proc. Natl. Acad. Sci.

USA 104 (2007) 702–707.[99] J. Lechleiter, S. Girard, E. Peralta, D. Clapham, Spiral calcium wave propagation and annihilation in Xenopus laevis oocytes, Science 252 (1991)

123–126.[100] M. Falcke, Reading the patterns in living cells — the physics of Ca2+ signaling, Adv. Phys. 53 (2004) 255–440.[101] M.D. Bootmann, D.R. Higazi, S. Coombes, H. Llewelyn Roderick, Calcium signalling during excitation–contraction coupling in mammalian atrial

myocytes, J. Cell Sci. 119 (2006) 3915–3925.[102] O.A. Igoshin, R. Welch, D. Kaiser, G. Oster, Waves and aggregation patterns in myxobacteria, Proc. Natl. Acad. Sci. USA 101 (2004) 4256–4261.[103] S.A. Levin, Dispersion and population interactions, Am. Nat. 108 (1974) 207–228.[104] A. McKane, D. Alonso, Extinction dynamics in mainland-island metapopulations: an n-patch stochastic model, Bull. Math. Biol. 64 (2002) 913–958.[105] R. Durrett, S. Levin, The importance of being discrete (and spatial), Theor. Pop. Biol. 46 (1994) 363–394.[106] E. Liebermann, C. Hauert, M.A. Nowak, Evolutionary dynamics on graphs, Nature 433 (2005) 312.[107] M.Mobilia, I.T. Georgiev, U.C. Täuber, Fluctuations and correlations in latticemodels for predator–prey interaction, Phys. Rev. E 73 (2006) 040903(R).[108] M. Mobilia, I.T. Georgiev, U.C. Täuber, Phase transitions and spatio-temporal fluctuations in stochastic lattice Lotka–Volterra models, J. Stat. Phys.

128 (2007) 447.[109] J.B.C. Jackson, L. Buss, Allelopathy and spatial competition among coral reef invertebrates, Proc. Natl. Acad. Sci. USA 72 (1975) 5160–5163.[110] O. Gilg, I. Hanski, B. Sittler, Cyclic dynamics in a simple vertebrate predator–prey-community, Science 302 (2001) 866–868.[111] B. Sinervo, C.M. Lively, The rock–scissors–paper game and the evolution of alternative male strategies, Nature 380 (1996) 240–243.[112] B.C. Kirkup, M.A. Riley, Antibiotic-mediated antagonism leads to a bacterial game of rock–paper–scissors in vivo, Nature 428 (2004) 412–414.[113] T. Reichenbach, M. Mobilia, E. Frey, Noise and correlations in a spatial population model with cyclic competition, Phys. Rev. Lett. 99 (2007) 238105.[114] M.C. Cross, P.C. Hohenberg, Pattern formation outside of equilibrium, Rev. Modern Phys. 65 (1993) 851–1112.[115] I.S. Aranson, L. Kramer, The world of the complex Ginzburg-Landau equation, Rev. Modern Phys. 74 (2002) 99.[116] W. van Saarloos, Front propagation into unstable states, Phys. Rep. 386 (2003) 29.[117] U.C. Täuber, M. Howard, B.P. Vollmayr-Lee, Applications of field-theoretic renormalization groupmethods to reaction-diffusion problems, J. Phys. A:

Math. Gen. 38 (2005) R79–R131.[118] D.C. Mattis, M.L. Glasser, The uses of quantum field theory in diffusion-limited reactions, Rev. Modern Phys. 70 (1998) 979.[119] K. Tainaka, Vortices and strings in a model ecosystem, Phys. Rev. E 50 (1994) 3401–3409.[120] K. Sato, N. Konno, T. Yamaguchi, Paper–scissors–stone game on trees, Mem. Muroran Inst. Tech. 47 (1997) 109–114.[121] G. Szabó, A. Szolnoki, R. Izsak, Rock–scissors–paper game on regular small-world networks, J. Phys. A: Math. Gen. 37 (2004) 2599.[122] T. Reichenbach, E. Frey, Instability of spatial patterns and its ambiguous impact on species diversity, Phys. Rev. Lett. 101 (2008) 058102.[123] M.E. Hibbing, C. Fuqua, M.R. Parsek, S. Brook Peterson, Bacterial competition: surviving and thriving in themicrobial jungle, Nat. Rev. Micro. 8 (2010)

15–25.[124] M.R. Parsek, E.P. Greenberg, Sociomicrobiology: the connections between quorum sensing and biofilms, Trends Microbiol. 13 (2005) 27–33.[125] C.T. MacDonald, J.H. Gibbs, A.C. Pipkin, Kinetics of biopolymerization on nucleic acid templates, Biopolymers 6 (1968) 1.[126] G.M. Schütz, Exactly solvable models for many-body systems far from equilibrium, in: Phase Transitions and Critical Phenomena, vol. 19, Academic

Press, 2001, pp. 1–251.[127] M.Mobilia, T. Reichenbach, H. Hinsch, T. Franosch, E. Frey, Generic principles of active transport, Banach Center Publ. 80 (2008) 101–120. arXiv:cond-

mat/0612516.[128] T.E. Harris, Contact interactions on a lattice, Ann. Probab. 2 (1974) 969–988.[129] H. Hinrichsen, Non-equilibrium critical phenomena and phase transitions into absorbing states, Adv. in Phys. 49 (2000) 815–958.[130] C. Castellano, S. Fortunato, V. Loreto, Statistical physics of social dynamics, Rev. Modern Phys. 81 (2009) 591–646.[131] P. Clifford, A. Sudbury, Model for spatial conflict, Biometrika 60 (1973) 581–588.[132] R. Holley, T.M. Liggett, Survival of contact processes, Ann. Probab. 6 (1978) 198–206.[133] T.M. Liggett, Stochastic Interacting Systems: Contact, Voter and Exclusion Processes, Springer Verlag, 1999.[134] L. Frachebourg, P.L. Krapivsky, E. Ben-Naim, Spatial organization in cyclic Lotka–Volterra systems, Phys. Rev. E 54 (1996) 6186–6200.[135] L. Frachebourg, P.L. Krapivsky, E. Ben-Naim, Segregation in a one-dimensional model of interacting species, Phys. Rev. Lett. 77 (1996) 2125.[136] A. Winkler, T. Reichenbach, E. Frey, Coexistence in a one-diemnsional cyclic dominance process (2010) (Manuscript in preparation).[137] Uwe C. Täuber, Graduate Textbook (Work in Progress): Critical Dynamics, 2008, http://www.phys.vt.edu/~tauber.