patch dynamics based on prisoner's dilemma game: superiority of golden rule

Ecological Modelling 150 (2002) 295–307

Patch dynamics based on Prisoner’s Dilemma game:superiority of golden rule

Kei-ichi Tainaka *, Yu ItohDepartment of Systems Engineering, Shizuoka Uni�ersity, Hamamatsu 432-8561, Japan

Received 13 February 2001; received in revised form 22 June 2001; accepted 27 August 2001

Abstract

There has been much literature on ecological model of Prisoner’s Dilemma (PD) game. This game illustrates thatcooperation can evolve in situations where individuals tend to look after themselves. In order to explain somebehaviors of altruism in animal societies, the strategy All Cooperate (AC), often called the Golden Rule, is moreappropriate than other strategies. However, very little is known about the superiority of AC. In the present article,we study patch dynamics based on non-iterated PD game, applying two different methods: island and lattice models.Each patch is assumed to be either vacant or composed of a population of AC or All Defect (AD), where AD meansa selfish strategy. Both models exhibit a phase transition between a phase where both AC and AD survive, and aphase where AD is extinct. The latter phase means that AC beats AD completely. In the case of lattice model, theextinction of AD easily occurs and the abundance of AC takes a larger value, compared with the island model. Ourmodels can be also extended to general iterated PD game; we describe the reason why AC can outperform any otherstrategy. © 2002 Elsevier Science B.V. All rights reserved.

Keywords: Prisoner’s Dilemma game; Lattice Lotka–Volterra model; Patch dynamics; Altruism; Golden Rule

www.elsevier.com/locate/ecolmodel

1. Introduction

Various forms of cooperation emerges in hu-man and animal societies without central author-ity. A Prisoner’s Dilemma (PD) game (Axelrod,1984, 1997; Nowak and May, 1992; Nakamaru, etal., 1997) clearly illustrates that cooperation canevolve in situations where individuals (players)tend to look after themselves and their own first.

So far, many authors have reported effectivestrategies against an egoist, such as Tit-For-Tat(abbreviated as TFT) (Axelrod, 1984; Axelrodand Hamilton, 1981) and Pavlov (Kraines andKraines, 1993; Nowak and Sigmund, 1993; Posch,1999). These strategies well explain the emergenceof cooperation, while they never sufficiently ac-count for altruistic behaviors of animals, espe-cially for human beings. In fact, TFT and Pavlovare based on revanchism: a player of TFT orPavlov always defects, if the opponent playerdefected at the previous move. Perhaps, the mostwidely accepted moral standard is the ‘Golden

* Corresponding author. Tel./fax: +81-5347-81228.E-mail address: [email protected] (K.-i.

Tainaka).

0304-3800/02/$ - see front matter © 2002 Elsevier Science B.V. All rights reserved.

PII: S0304 -3800 (01 )00481 -1

mailto:[email protected]

K.-i. Tainaka, Y. Itoh / Ecological Modelling 150 (2002) 295–307296

Rule’ (altruism), do what you want from theother. In the context of the PD game, the GoldenRule would seem to imply that you should alwayscooperate. This interpretation suggests that thebest strategy from the point of morality is thestrategy All Cooperate (AC) (Axelrod, 1984).Nowak et al. (Nowak and May, 1992; Nowak, etal., 1994) first found the superiority of AC fornon-iterated PD game. In the present paper, wefind another, possibly more general, case thatincludes migration (Tainaka, 2000). This can bealso extended to general iterated PD game.

A PD game is played by a pair of individuals.In one move, each individual represents one oftwo options: either to cooperate or to defect. Ifboth players cooperate, both get the pay-off R,standing for reward. If one cooperates and theother defects, then the former (latter) gets pay-offS (T) which means sucker (temptation). If bothdefect, both get pay-off P, standing for punish-ment. The PD game is usually defined by thefollowing relations:

S�P�R�T, (1a)

T+S�2R. (1b)

Whenever you want to get the highest score T,you have a risk to have a low pay-off P(dilemma). In most case, pay-offs are set as S=0,P=1, R=3, and T=5, we call them standard�alues.

It is known that TFT and Pavlov differ fromevolutionarily stable strategy(ESS; MaynardSmith and Price, 1973; Maynard Smith, 1989;Dawkins, 1976). No pure strategy is evolutionar-ily stable in the repeated PD game (Boyd andLorberbaum, 1987). Actually, TFT is beaten byPavlov in a noise containing case, where the noisemeans an accidental error (Lindgren, 1991;Nowak and Sigmund, 1993; Ikegami, 1994). Onthe other hand, Pavlov is easily invaded by AD.In the present paper, we describe the superiorityof evolutionarily maintainable strategy (EMS)(Tainaka and Araki, 1999) which is defined by themost sustainable strategy in a single patch. In thecontext of the PD game, EMS gains the highestpay-off (fitness) in the game between players ofidentical strategy. From (Eqs. (1a) and (1b)), we

can easily prove that EMS corresponds to thestrategy AC (Molander, 1985); a pair of AC al-ways get the maximum value 2R for each move.Hence, AC can be the best strategy in situationswhere AC forms a contagious distribution (highlocal density).

We study patch dynamics of PD game underthe assumptions that a biospecies lives in somepatches. It is not a special condition for a biospe-cies to live in a patchy environment (Hanski andGilpin, 1997; Matter, 1999; Hargrove, et al., 2000;Li, 2000; Kindvall and Petersson, 2000). We mod-ify the lattice system introduced by Nowak et al.(1994). They assumed that a single patch wouldbe dominated by a certain strategy in a relativelyshort period, and that each lattice site representeda patch which contained several players of a singlestrategy. Main modification in the present paperis to apply an idea of group selection (Eshel, 1972;Levin and Kilmer, 1974; Levin and Pimentel,1981; Aoki, 1982; Wilson, 1983), we introducesome ecological properties, such as migration,patch extinction and invasion.

The patch occupied by AC will be invaded bythe migration of another strategy, say AD. How-ever, the total fitness gained in the population ofAD is very poor, so that the AD patch may goextinct during a long period. In order for a popu-lation to survive, it is necessary that its populationsize is sufficiently larger than the so-called mini-mum viable population (MVP) (Soule, 1987;Tainaka and Itoh, 1996). The MVP size surelydiffers for different species, whereas empiricalworks for MVP suggest that the MVP size may bevery large (Thomas, 1990; Wilcove et al., 1993).Moreover, many authors have pointed out that bythe time a species is listed as endangered, itsnumbers have fallen well below a sustainable pop-ulation size (Noss and Murphy, 1995). Hence, it isnot so easy for a small population to survive fora long time. In contrast, AC gets the highestfitness in its community; AC can survive for along time, and it may build new patches(colonies).

Heretofore, several patch models (Fig. 1) havebeen presented by many authors (Hanski andGilpin, 1997; Levin, 1974; Maynard Smith, 1982).We use the island (Levins, 1969) and lattice (Has-

K.-i. Tainaka, Y. Itoh / Ecological Modelling 150 (2002) 295–307 297

sel, et al., 1991; Caswell and Cohen, 1995) modelsas illustrated in Fig. 1(b and d), respectively. TheLotka–Volterra equation (LVE; Hofbauer andSigmund, 1988; Takeuchi, 1996) is applied to theformer, and lattice Lotka–Volterra model(LLVM; Tainaka, 1988, 1989; Matsuda, et al.,1992; Itoh and Tainaka, 1994; Sato, et al., 1994,2001; Kobayashi and Tainaka, 1997) is applied tothe latter. Note that our patch systems are entirelydifferent from usual patch models (Hanski andGilpin, 1997; Durrett and Levin, 1994), we useLVE and LLVM not for the population dynamicsinside each patch but for interdemic dynamics.

In the next section, we describe our patchmodel which is further divided into island andlattice models. In Section 3, theoretical results for

the island model (LVE) are reported. This theorypredicts a phase transition which means that ACcompletely beats AD. Similarly, the lattice model(LLVM) reveals the phase transition (Section 4).In the case of LLVM, the extinction of AD easilyoccurs compared with LVE. In the final section,some conclusions for our patch dynamics arereported; especially, we describe the reason whyAC can be superior to other strategies. Psycholog-ical and ecological meanings of the result arediscussed.

2. Model and method

2.1. Island and lattice models

We assume that a target biospecies lives in apatchy environment (Li, 2000; Kindvall and Pe-tersson, 2000), and that interaction between dif-ferent patches rarely occurs (Levin and Pain,1974). A single patch would be dominated by acertain strategy in a relatively short period.Hence, each lattice site represented a patch whichcontained some players of a single strategy. Ourattention is mainly paid not to such local andshort-term scales but to regional (metapopulation)and long-term scales. We modify the lattice sys-tem introduced by Nowak et al. (1994), in partic-ular, we take into account the effect of migration.Fig. 1 illustrates typical patch models, where thearrows denote the direction that individuals canmigrate. (a) Stepping stones, each patch interactswith several neighboring patches; (b) islands, in-teraction is allowed between any pair of patches;(c) continent and islands, migration occurs from alarge continent to small islands; (d) lattice, eachpatch can interact with adjacent patches. In thispaper, we apply the island and lattice models asillustrated in Fig. 1(b and d), respectively.

Each patch is assumed to be vacant or com-posed of a population of either AC or All Defect(AD). Note that there are only two strategies ACand AD for non-iterated PD game. We study thefollowing patch reactions:

AD+AC�2AD, (2a)

AC+O�r

2AC, (2b)

Fig. 1. Several patch models. The arrows denote the directionthat individuals can migrate (interact). (a) Stepping stones,each patch interacts with several patches; (b) islands, interac-tion is allowed between any pair of patches (Levins, 1969); (c)continent and islands, migration occurs from a large continentto small islands (Boorman and Levitt, 1980); (d) lattice, eachpatch can interact with the adjacent patches (Hassel, et al.,1991; Caswell and Cohen, 1995). In the present paper, weapply the island (b) and lattice (d) models. In the case oflattice model, each patch is allowed to interact with z patches(z=4 for square lattice).


AD�d

O. (2c)

where O represents a vacant patch. The abovesystem is based on the actual moves of PD game.The first reaction (2a) denotes the in�asion, if fewindividuals of AD migrate into a AC patch, thenthe population size of AD (AC) immediately in-creases (decreases) in this patch. The AC patch iseasily invaded by AD. The second reaction (2b)means the colonization of AC, if some individualsof AC migrate into a vacant site, its populationsize may grow up. Namely, the process (Eq. (2b))denotes the reproduction of AC patches. The lastreaction (2c) represents the extinction of a habitatof AD. Since the AD population gains the leastvalue of fitness, it is very hard for this populationto survive for a long time. The parameter r de-notes the colonization (reproduction) rate of AC,while d means the extinction (death) rate of theAD patches. When the size of MVP of the targetspecies is large, then d takes a large value.

A patchy environment is critical for our system(Eqs. (2a), (2b) and (2c)). If all individuals live ina single habitat, then the reaction (2a) determinesthe winner, the strategy AD usually beats AC.Moreover, the vacant site (O) is essential to takeinto account a long-term stability. Note that inthe system (Eqs. (2a), (2b) and (2c)), a coloniza-tion process of AD (namely, AD+O�2AD) isignored. This is considered to be reasonable, be-cause it is very hard for AD to survive. On thecontrary, the population of AC gains the highestscore among all strategies, so that we assume notonly that the colonization (Eq. (2b)) occurs butalso that the extinction process AC�O is ne-glected. The system (Eqs. (2a), (2b) and (2c)) canbe extended to iterated PD game as discussed inthe last section.

2.2. Simulation methods

We apply the LVE to island model, and theLLVM to lattice model. First, we explain thesimulation method of the LLVM which is a sim-ple extension to the contact process (Liggett,1985; Durrett, 1988; Konno, 1994; Marro andDickman, 1989; Tainaka et al., 2000). Simulationis carried out as follows:

1. initially, we distribute two kinds of strategies,AC and AD, over some square-lattice sites;each lattice site is, therefore, labeled by one ofthree states (AC, AD, or O).

2. Reactions (2) are performed in the followingtwo steps.

(i) We perform two-body reactions, namely,reactions (2a) and (2b). Choose one lat-tice site randomly, and then specify oneof four nearest-neighbor sites. Let themreact according to (Eqs. (2a) and (2b)).For example, if you pick up a pair of siteslabeled by AC and O, the site O willbecome AC by a probability (rate) r.Here we employ periodic boundary condi-tions.

(ii) A single particle reaction, the process (Eq.(2c)), is performed. Choose one lattice siterandomly, and let it react according to(Eq. (2c)), if the AD site is picked up, itwill become O by the rate d.

3. Repeat step (2) by L×L times, where L×L isthe total number of the square-lattice sites.This step is called Monte Carlo step (Tainaka,1988, 1989). In this paper, we set L=100 andL=160.

4. Repeat step (3) for 1000–2000 Monte Carlosteps.

Next, the simulation method for the islandmodel is described. The above steps (1)– (4) arethe same, however, in the case of island model,the process (i) at step (2) should be replaced asfollows, choose two lattice sites randomly andindependently, and react them according to (Eqs.(2a) and (2b)). Hence, for the island model, spa-tial dimension becomes meaningless.

3. Results of island model

When the number of total patches (L2) is suffi-ciently large, the population dynamics of islandmodel is expressed by:

P� AC=2(−PACPAD+rPACPO), (3a)

P� AD=2PACPAD−dPAD, (3b)


Fig. 2. For the island model, the steady-state densities (Eq. (5)) are shown against the extinction rate (d) of AD patch (r=1). Inthe case d�2, AC beats AD completely.

where the dots denote the derivative with respectto the time t which is measured by the MonteCarlo step, and Pi is the density of patches ofstrategy i (i=AC, AD, O). Note that the totaldensity is unity:

�i Pi=1. (4)

Each term in (Eqs. (3a) and (3b)) comes fromrespective reaction in (Eqs. (2a), (2b) and (2c)).For example, the first term in the right-hand sideof (Eq. (3a)) is originated in the reaction (2a),where the factor 2 denotes that there are two waysfor the left-hand side of (Eq. (2a)), that is, AC+AD and AD+AC. Similarly, the second term of(Eq. (3a)) comes from reaction (2b), and so on.Note that the time dependence of PO is deter-mined by Eqs. (3a), (3b) and (4)). Basic (Eqs. (3a)and (3b)) represent a standard Lotka–Volterramodel with logistic prey growth (Hofbauer andSigmund, 1988; Murray, 1989; Takeuchi, 1996).

The steady-state solution can be obtained bysetting all the time derivatives in (Eqs. (3a) and(3b)) to be zero, it follows that:

PAC=d2

, PAD=r(1−PAC)

1+r, PO=

1−PAC

1+r.

(5)

According to the linear stability analysis (Hof-bauer and Sigmund, 1988), the densities of bothstrategies reach the stationary values (Eq. (5)) inthe case d�2. Irrespective of initial conditions,the system evolves into the stationary state (stablefocus). In Fig. 2, the steady-state densities of PAC

and PAD is depicted against the parameter d.When d�2, we have PAC=1 and PAD=0.Hence, LVE exhibits a phase transition between aphase where both AC and AD survive (d�2),and a phase where AD is extinct (d�2). Thephase boundary is represented by d0=2. Namely,AC beats AD completely, when d takes a largevalue (d�d0). Recall that d denotes the extinction(death) rate of AD patch. The cause of extinctionof AD is thus reasonable.

So far, the total number of patches (L2) wasassumed to be sufficiently large. In this case, theaverage densities (Eq. (5)) in final stationary statenever depend on both densities PAC and PAD at


t=0 (initial condition). Now, we consider thecase that L2 takes a small value. Then, the dy-namics becomes a stochastic process, and it de-pends on an initial condition. It is obvious thateither strategy AC or AD which has a lowerdensity in stationary state tends to go extinct.(Eq. (5)) reveals that PAC�PAD for d�2r/(1+2r), when d takes a small value, AC often goesextinct. It is, however, emphasized that AD can-not survive without AC; in other words, ADimmediately goes extinct after the extinction ofAC. On the contrary, in the case of a large valueof d, the strategy AD often goes extinct. Afterthis extinction, the system (Eqs. (2a), (2b) and(2c)) becomes (Eq. (2a)), so that AC occupies thewhole patches. It is, therefore, concluded that fora small value of L2, the strategy AD easily goesextinct.

4. Results of lattice model

4.1. Dependence on d

Simulations for lattice model (LLVM) are car-ried out for various values of the parameters dand r. First, we describe the dependence onparameter d. It is found from the simulation thatthe population dynamics exhibits a stable focusas predicted by the island model (LVE). Thelattice system evolves into a stationary state,where both densities PAC and PAD become con-stant in time, but always experience fluctuations.Nevertheless, the configuration of patch distribu-tion dynamically varies, AD runs after AC. InFig. 3, typical stationary patterns are illustratedfor large and small values of d. From this figure,we notice that both strategies are more or less

Fig. 3. Snapshot of a typical stationary pattern of patches for lattice model (LLVM). The colors of dark blue, red, and whiterepresent the patch of AC, AD, and O, respectively. The extinction rate (d) of AD patch takes a small (d=0.04) or large (d=0.88)value. The other parameter is set as r=1. When the value of d exceeds a critical value d0 (d0�0.9), the whole patches are occupiedby AC.


Fig. 4. For the lattice model, the steady-state density of ADpatch is plotted against the extinction rate (d) of AD patch(r=1). Each plot is obtained by the long-time average in theperiod 200� t�1000 with the square lattice (100×100),where the time t is measured by the Monte Carlo step. Thesymbols ISLAND and PA by the solid curves represent theisland model and pair approximation (see Appendix A), re-spectively.

extinction rate (d) of AD patch. The symbolsISLAND and PA in these figures denote thepredictions of island model and pair approxima-tion, respectively. This approximation (PA) some-times gives a good prediction (Katori and Konno,1991; Matsuda, et al., 1992; Tainaka, 1993,1994a), but it sometimes fails (Tainaka, 1994b,1995; Sato, et al., 1994). In Appendix A, the resultof PA for our system (Eqs. (2a), (2b) and (2c)) isdescribed. It is found from Figs. 4 and 5 that thephase transition occurs as predicted by the islandmodel (LVE), the strategy AC occupies the wholepatches, when d exceeds a critical value d0. Thedensity of the AC patch (AD patch) for LLVM issignificantly higher (lower) than the prediction ofisland model; the critical value for LLVM (d0�0.9) is much smaller than the prediction of LVE(d0=2). The reason why d0 takes a smaller valuein the lattice model is originated in the clumpingbehavior of both AC and AD patches (see theright pattern in Fig. 3), AD cannot easily invadeAC. Moreover, in the case of lattice model, wenotice a counterintuiti�e response never seen inisland model (Fig. 4): even if the value of ddecreases and approaches zero, the density PAD

decreases. It is still unclear whether PAD ap-proaches zero or not in the limit d�0 (Satulovskyand Tome, 1994).

4.2. Dependence on r

It is obvious that AD goes extinct with increas-ing d. Next, we fix d, and change the parameter r.In Fig. 6, the steady-state densities of PAC andPAD are plotted against the colonization rate r ofAC, where we set d=0.6. This figure reveals thefollowing results.

(i) The density of AC (PAC) increases in spite ofthe decrease of r. In particular, when r ap-proaches r0 from above (r0�0.17), PAC

abruptly increases.(ii) The density PAD for the lattice model

(LLVM) is much lower than the predictionof LVE or PA. Especially, when r�r0, ADdisappears (extinct phase).

In the case of PA, the density PAC takes inter-mediate value between the results of LLVM andLVE, but divergent behavior of AC population

Fig. 5. Same as Fig. 4, but the longitudinal axis denotes thesteady-state density of AC patch.

clumped, AC patches are strongly clumped, whileAD patches relatively disperse in order to catchAC effectively. Such a pattern formation resem-bles the pattern reported by Nowak and May(1992). If the extinction rate (d) of AD patchtakes a small (d=0.04) value, AC patches formcompact clusters (Fig. 3). On the contrary, if dtakes a large value (d=0.88), not only AC butalso AD are strongly clumped.

In Figs. 4 and 5, the steady-state densities ofPAC and PAD are, respectively, plotted against the


and the extinction of AD are not explained. Wenotice that the phase transition is caused by acounterintuitive situation: AD completely disap-pears, even though the number of AC patchesabruptly increases. Hence, the global pattern for-mation strongly influences on the winner in PDgame.

To know the spatial correlation, we obtain theclumping degree of strategy AD defined by:

RAD,AD=PAD,AD

PAD2 , (6)

where PAD,AD denotes the probability finding thestrategy AD at neighboring two lattice sites(Tainaka and Fukazawa, 1992; Tainaka, 1994a).When the distribution of this strategy is random,we get RAD,AD=1. When RAD,AD�1 (RAD,AD�1), the strategy AD forms a clumped (uniform)pattern. The simulation result of RAD,AD is de-picted against r−r0 (Fig. 7), where we use r0=0for PA and r0=0.17 for LLVM. Fig. 7 clearlyreveals that the AD patch always forms a clumpedpattern (RAD,AD�1). Especially, when r ap-proaches r0 from above, the degree of clumpingbecomes rapidly high; we find from Fig. 7 that:

RAD,AD� (r−r0)−�, ��1. (7)

This result is theoretically derived by the pairapproximation (PA: Appendix A). The distribu-tion of AD near r0 resembles the right pattern(d=0.88) in Fig. 3. Near the extinction point, ADpatches are forced to be strongly clumped. Clump-ing behavior Eq. (7) explains that the AC popula-tion increases and AD goes extinct with decreasingr, if r approaches r0 from above, AD patches arestrongly clumped, so that they cannot easily catchAC. The pair approximation explains the clusterformation (Eq. (7)) but not the non-zero valueof r0.

5. Concluding remarks

There has been much literature on ecologicalmodel of PD game. In most cases, AC (indicatingunconditional cooperation) has been inferior toother strategies, especially to AD and Pavlov. Inthe present paper, we illustrate that AC beats AD.Patch dynamics based on PD game is studied bytwo different methods: island and lattice models.Both models exhibit the phase transition betweena phase where AC and AD survive, and a phasewhere AD is extinct. The latter phase means thatAC beats AD completely. Our system (Eqs. (2a),(2b) and (2c)) is essentially equivalent to the prey–predator (host–parasite) model in an ecosystem(Hofbauer and Sigmund, 1988; Tainaka andFukazawa, 1992; Tainaka, 1994a; Satulovsky andTome, 1994; Sutherland and Jacobs, 1994;Pacheco et al., 1997; Hance and Van Impe, 1999),the strategies AC and AD, respectively, corre-spond to prey and predator. Although the prey isbeaten by predator, the abundance of prey be-comes larger than that of predator in most cases.

So far, we considered the non-iterated PD game,where there are two strategies AC (cooperator)and AD (defector). However, our system (Eqs.(2a), (2b) and (2c)) can be extended to a generaliterated PD game which contains the noise. SinceAC gains the highest pay-off in the game betweenplayers of identical strategy, the colonization(extinction) rate of AC takes the highest (lowest)value among all strategies. If AD in thesystem (Eqs. (2a), (2b) and (2c)) is replaced

Fig. 6. The steady-state densities are plotted against the repro-duction (colonization) rate of AC (r), where we put d=0.6.The symbols ISLAND and PA represent the same meanings asin Figs. 4 and 5.


Fig. 7. For the lattice model (LLVM), the relation between RAD,AD and r−r0 is displayed. The ratio RAD,AD represents the degreeof clumping of AD. The window illustrates the log– log plot of the same data, where the solid curve is the theoretical prediction ofPA. We use r0=0 for PA, and r0=0.17 for LLVM.

into another strategy S, then we should add thecolonization of S to (Eqs. (2a), (2b) and (2c)) asfollows: S+O�2S. In this case, the system (Eqs.(2a), (2b) and (2c)) becomes equivalent to themodel of intraguild predation (Polis et al., 1989;Holt and Polis 1997); both prey (AC) and predator(S) consume the same resource (O). By the compe-tition between AC and S, AC may go extincteasily. Nevertheless, in certain range of parameterspace, AC can survive or AC may beat S.

The island model predicts that the phase transi-tion (extinction of AD) occurs, only when theextinction rate (d) of the AD habitat exceeds acritical value (d�d0). Such a cause is natural,

since the AD population gains the least fitness.For the lattice model, we observe another cause ofphase transition: When the colonization rate (r) ofAC is decreased, then the AC density rapidlyincreases, and the extinction of AD occurs (Fig.6). In the latter extinction, the global patternformation plays an important role, if r approachesr0 from above, AD patches are forced to bestrongly clumped (Fig. 7), so that they go extinctat a finite value of r. In general, the clumpingnature gives a large effect on stationary state(Tainaka and Itoh, 1991; Tainaka, 1993, 1994a;Harada and Iwasa, 1994), and the long-term re-sponse is very difficult to predict by both simula-


tion and theories (Yodzis, 1988; Pimm, 1993;Kobayashi and Tainaka, 1997; Schmitz, 1997;Nakagiri, et al., 2001).

We discuss an ecological meaning of our results.For animals, altruism widely evolves (Wynne-Ed-wards, 1962; Dawkins, 1976; Boorman and Levitt,1980; Matsuda, 1987; Taylor, 1992; Van Baalenand Rand, 1998). The evolution of altruism amongrelated animals is well explained by the theory ofkinship (Hamilton, 1963, 1964) or selfish genes(Dawkins, 1976). However, this theory fails for thealtruism among unrelated animals. In the lattercase, altruism is often called reciprocal altruism(Trivers, 1971; Wilkinson, 1984; Nakamaru, et al.,1997, 1998; Boyd and Richerson, 1988) and ex-plained by nice strategies, such as TFT and Pavlov,where the property of being nice is to say neverbeing the first to defect. However, we consider thatvarious types of altruism have been developed inanimal societies. The strategy AC or Golden Ruleis more appropriate than TFT or Pavlov to explainsome behaviors of altruism (Axelrod, 1984).

The psychological meaning of our results isdiscussed. So far, TFT and Pavlov are consideredto be effective strategies to use against an egoist. Itis, however, fair to say that TFT and Pavlov arefar from a moral standard in human society, sincethey are based on the revanchism or the principleof eye for eye. In noise-containing case, averagepayoffs in their patches become less than that inAC patch (Molander, 1985). We consider thatPavlov and TFT are selfish strategies to someextent. In the present paper, we describe the supe-riority of the strategy AC which explains the originof Golden Rule.

We modify the lattice system introduced byNowak, et al. (1994). Our modifications are asfollows.1. We take into account the invasion of AD into

AC patches; if few individuals of AD migrateinto a AC patch, then the population size ofAD (AC) immediately increases (decreases) inthis patch. The AC patch should be easilyinvaded by AD.

2. We emphasize the effect of MVP for the ex-tinction process of patches. The AD patchshould easily go extinct compared with ACpatch.

3. Our model is applicable to iterated PD gamewith noise; AC can beat any other strategy.

Moreover, we discuss several assumptions con-tained in this work:

(i) the migration between different patches isassumed to occur rarely. If migration fre-quently occurs, all patches are effectivelyconnected. In this case, AD beats AC.

(ii) In the system (Eqs. (2a), (2b) and (2c)), weneglect the reaction AC�O. If this process iscontained, our system becomes equivalent tothe host–parasite model (Sato, et al., 1994;Tainaka and Araki, 1999) which also exhibitsthe phase transition; that is, AC can beat ADcompletely. While the parasite (AD) cannotsurvive without host (AC), the latter cansurvive without the former.

Acknowledgements

The authors thank to J. Yashimura and M.Nakamaru for their helpful comments. This studywas carried out under the ISM Cooperative Re-search Program.

Appendix A. Pair approximation

The lattice model (LLVM) has basic equations:

P� AC= −2PAC,AD+2rPAC,O, (A.1a)

P� AD=2PAC,AD−dPAD, (A.1b)

P� O= −2rPAC,O+dPAD, (A.1c)

where Pi,j denotes the two-body density finding astrategy i at a lattice site (patch) and a strategy jat its adjacent site (i, j=AC, AD, O). The firstapproximation to solve (Eqs. (A.1a), (A.1b) and(A.1c)) is the mean-field theory:

Pi, j=Pi Pj.

Inserting above equation into (Eqs. (A.1a),(A.1b) and (A.1c)), we have the rate equations(Eqs. (3a) and (3b)) for the island model.Similarly to (Eqs. (A.1a), (A.1b) and (A.1c)), the


evolution equations for the two-body densities areexpressed by:�z

4�

P� AC,AC=rPAC,O+ (z−1)[rPAC,ACO −PAC,AD

AC ],

(A.2a)�z4�

P� AD,AD

=PAC,AD−2dPAD,AD+ (z−1)PAD,AD,AC (A.2b)�z

4�

P� O,O=2dPAD,O−r(z−1)PAC,OO , (A.2c)

�z2�

P� AC,AD

= −2dPAC,AD−PAC,AD

+ (z−1)[PAC,ADAC +rPAC,AD

O −PAD,ADAC ],

(A.2d)�z2�

P� AD,O

=2d(PAD,AD−PAD,O)

+ (z−1)[PAD,OAC −rPAC,AD

O ], (A.2e)�z2�

P� AC,O

=2dPAC,AD−rPAC,O

+ (z−1)[rPAC,OO −rPAC,AC

O −PAD,OAC ], (A.2f )

where Pj,ki denotes three-body probability finding

a strategy i at a site and strategy j and k at nearestneighbors of that site, and z=4 for square lattice.From definitions of probabilities, the followingrelations hold:

Pi, j=Pj,i, �j Pi, j=Pi,

Pj,ki =Pk, j,

i � kPj,ki =Pij.

The second approximation to solve (Eqs.(A.2a), (A.2b), (A.2c), (A.2d), (A.2e) and (A.2f))is the pair approximation (PA) defined by:

Pj,ki =

Pi, j Pi,k

Pi

. (A.3)

Setting all the time derivatives in (Eqs. (A.1a),(A.1b) and (A.1c)) and (Eqs. (A.2a), (A.2b),(A.2c), (A.2d), (A.2e) and (A.2f)) to be zero, and

using (Eq. (A.3)), we obtain the steady-state solu-tion in the following explicit forms (Tainaka,1994b):

PAC=QAC

1+QAC+QO

, PAD=1

1+QAC+QO

,

(A.4)

where

QO=b+ [b2+4a(z−1)dY ]1/2

2a, (A.5a)

QAC=XQO+Y, (A.5b)

and

a=2(z−2)rX,

b= (z−1)d(X+1+r)−2(z−2)rY,

X=2(z−2)rd+4d2(1+R)

(z−2)(3r+2d−2rd)−4d2,(A.6)

Y=(z−1)(z−2)rd

2(z−2)(3r+2d−2rd)−8d2. (A.7)

The probabilities Pi,j (i� j ) in stationary stateare expressed as:

PAC,AD=dPAD

2, (A.8)

PAC,O=PAC,AD

r, (A.9)

PAD,O=POPAD

2PAD

+2dPO

z−1−PAC,O. (A.10)

From above relations, we can get the otherjoint probabilities Pi,i :

PAC,AC=PAC−PAC,AD−PAC,O, (A.11)

PAD,AD=PAD−PAC,AD−PAD,O, (A.12)

PO,O=PO−PAC,O−PAC,O. (A.13)

The probability PAD,AD can be represented inthe following simple form:

PAD,AD

PAD

=14+

(z−1)dPAD

8PAC

. (A.14)

Self-organized clumping patterns Eq. (7) can beproved by the PA theory. For PA model, thecritical point (r0) of phase transition is given byr0=0. In the limit r�r0, we get from (Eqs. (A.4),(A.5a), (A.5b) and (A.14)) that:


PAD�r,PAD,AD

PAD

�14, (A.15)

where we put z=4. This relation and r0=0 leadto:

RAD,AD� (r−r0)−�, �=1. (A.16)

This is Eq. (7) in the text.

References

Aoki, K., 1982. A condition for group selection to prevail overcounteracting individual selection. Evolution 36, 832–842.

Axelrod, R., 1984. The Evolution of Cooperation. PrincetonUniversity Press, Princeton.

Axelrod, R., 1997. The Complexity of Cooperation. BasicBooks, New York.

Axelrod, R., Hamilton, W.D., 1981. The evolution of coopera-tion. Science 211, 1390–1396.

Boyd, R., Lorberbaum, J.P., 1987. No pure strategy is evolu-tionarily stable in the repeated Prisoner’s Dilemma game.Nature 327, 58–59.

Boyd, R., Richerson, J.P., 1988. The evolution of reciprocityin sizable groups. J. Theor. Biol. 132, 337–356.

Boorman, S.A., Levitt, P.R., 1980. The Genetics of Altruism.Academic Press, New York.

Caswell, H., Cohen, J.E., 1995. Red, white and blue: environ-mental variance spectra and coexistence in metapopula-tions. J. Theor. Biol. 176, 301–316.

Dawkins, R., 1976. The Selfish Genes. Oxford UniversityPress, Oxford.

Durrett, R., 1988. Lecture Notes on Particle Systems andPercolation. Wadsworth and Brooka/Cole AdvancedBooks & Software, California.

Durrett, R., Levin, S.A., 1994. The importance of beingdiscrete (and spatial). Theor. Popul. Biol. 46, 363–394.

Eshel, I., 1972. On the neighbor effect and the evolution ofaltruistic traits. Theor. Popul. Biol. 3, 258–277.

Hamilton, W.D., 1963. The evolution of altruistic behaviour.Am. Nat. 97, 354–356.

Hamilton, W.D., 1964. The genetical evolution of social be-haviour. J. Theor. Biol. 7, 1–52.

Hance, T.H., Van Impe, G., 1999. The influence of initial agestructure on predator–prey interaction. Ecol. Mod. 114,195–211.

Hanski, I.A., Gilpin, M.E., 1997. Metapopulation Biology:Ecology, Genetics, and Evolution. Academic Press, SanDiego.

Hargrove, W.W., Gardner, R.H., Turner, M.G., Romme,W.H., Despain, D.G., 2000. Simulating fire patterns inheterogeneous landscapes. Ecol. Mod. 135, 243–263.

Hassel, M.P., Comins, H.N., May, R.M., 1991. Spatial struc-ture and chaos in metapopulation dynamics. Nature 353,255–258.

Harada, Y., Iwasa, Y., 1994. Lattice population dynamics forplant with dispersing seeds and vegetative propagation.Res. Popul. Ecol. 36, 237–249.

Hofbauer, J., Sigmund, K., 1988. The Theory of Evolutionand Dynamical Systems. Cambridge University Press,Cambridge.

Holt, R.D., Polis, G.A., 1997. A theoretical framework forintraguild predation. Am. Naturalist 149, 745–764.

Ikegami, T., 1994. From genetic evolution to emergence ofgame strategies. Phys. D 75, 310–327.

Itoh, Y., Tainaka, K., 1994. Stochastic limit cycle with power-Law spectrum. Phys. Lett. A 189, 37–42.

Katori, M., Konno, N., 1991. Upper bounds for survivalprobability of the contact process. J. Stat. Phys. 63, 115–130.

Kindvall, O., Petersson, A., 2000. Consequences of modellinginterpatch migration as a function of patch geometry whenpredicting metapopulation ex tinction risk. Ecol. Mod.129, 101–109.

Kobayashi, K., Tainaka, K., 1997. Critical phenomena incyclic ecosystems: parity law and selfstructuring extinctionpattern. J. Phys. Soc. Jpn. 66, 38–41.

Konno, N., 1994. Phase Transition of Interacting ParticleSystems. World Scientific, Singapore.

Kraines, D., Kraines, V., 1993. Learning to cooperate withPavlov. An adaptive strategy for the iterated Prisoner’sDilemma game. Theory Decision 35, 107–150.

Levin, S.A., 1974. Dispersion and population interactions.Am. Nat. 108, 202–228.

Levin, B.R., Kilmer, W.L., 1974. Interdemic selection and theevolution of altruism: a computer simulation study. Evolu-tion 28, 527–545.

Levin, S.A., Pain, R.T., 1974. Disturbance, patch formation,and community structure. Am. Nat. 108, 308–315.

Levin, S.A., Pimentel, D., 1981. Selection of intermediate ratesof increase in parasite–host system. Proc. Nat. Acad. Sci.71, 2744–2744.

Levins, R., 1969. Some demographic consequences of environ-mental heterogeneity for biological control. Bull. Entomol.Soc. Am. 15, 237–240.

Li, B.L., 2000. Fractal geometry applications in descriptionand analysis of patch patterns and patch dynamics. Ecol.Mod. 132, 33–50.

Liggett, T.M., 1985. Interacting Particle Systems. Springer,New York.

Lindgren, K., 1991. Evolutionary phenomena in simple dy-namics. In: Langton, C. (Ed.), Artificial Life II. Addison-Wisley Publishing Company, pp. 295–312.

Marro, J., Dickman, R., 1989. Nonequilibrium Phase Transi-tion in Lattice Models. Cambridge University Press,Cambridge.

Matsuda, H., 1987. Condition for the evolution of altruism.In: Ito, Y., Brown, J.L., Kikkawa, J. (Eds.), Animal Soci-eties: Theory and Facts. Japan Sci. Soc. Press, Tokyo, pp.67–80.

Matsuda, H., Ogita, N., Sasaki, A., Sato, K., 1992. Statisticalmechanics of population: the lattice Lotka–Volterramodel. Prog. Theor. Phys. 88, 1035–1049.


Matter, S.F., 1999. Population density and area: the role ofwithin- and between-generation processes over time. Ecol.Mod. 118, 261–275.

Maynard Smith, J., 1982. Evolutionary Genetics. CambridgeUniversity Press, Cambridge.

Maynard Smith, J., 1989. Evolution and the Theory of Games.Oxford University Press, Oxford.

Maynard Smith, J., Price, G.E., 1973. The logic of animalconflict. Nature 246, 15–16.

Molander, P., 1985. The optimal level of generosity in aselfish, uncertain environment. J. Conflict Resol. 29, 611–618.

Murray, J.D., 1989. Mathematical Biology. Springer, Berlin.Nakamaru, M., Matsuda, H., Iwasa, Y., 1997. The evolution

of cooperation in a lattice-structured population. J. Theor.Biol. 184, 65–81.

Nakagiri, N., Tainaka, K., Tao, T., 2001. Indirect relationbetween species extinction and habitat destruction. Ecol.Mod. 137, 109–118.

Nakamaru, M., Nogami, H., Iwasa, Y., 1998. Score-dependentfertility model for the evolution of cooperation in a lattice.J. Theor. Biol. 194, 101–124.

Noss, R.F., Murphy, D.D., 1995. Endangered species lefthomeless in Sweet Home. Conserv. Biol. 9, 229–231.

Nowak, M.A., Bonhoeffer, S., May, R.M., 1994. More spatialgames. Int. J. Bifurcation Chaos 4, 33–56.

Nowak, M.A., May, R.M., 1992. Evolutionary games andspatial chaos. Nature 359, 826–829.

Nowak, M.A., Sigmund, K., 1993. A strategy of win-stay,lose-shift that outperforms tit-for tat in the Prisoner’sDilemma game. Nature 364, 56–58.

Pacheco, J.M., Rodriguez, C., Fernandez, I., 1997. Hopf bifur-cations in predator–prey systems with social predator be-havior. Ecol. Mod. 105, 83–87.

Pimm, S.L., 1993. Discussion: understanding indirect effects: isit possible. In: Kawanabe, H., Cohen, J.E., Iwasaki, K.(Eds.), Mutualism and Community Organization. OxfordUniversity Press, Oxford, pp. 199–209.

Polis, G.A., Myers, C.A., Holt, R.D., 1989. The ecology andevolution of intraguild predation: potential competitorsthat eat each other. Ann. Rev. Ecol. Syst. 20, 297–330.

Posch, M., 1999. Win-stay, lose-shift strategies for repeatedgames-memory length, aspiration levels and noise. J.Theor. Biol. 198, 183–195.

Sato, K., Matsuda, H., Sasaki, A., 1994. Pathogen invasionand host extinction in lattice structured populations. J.Math. Biol. 32, 251–268.

Sato, K., Yoshida, N. Konno, N., 2001. Parity law for popula-tion dynamics of N-species with cyclic advantage competi-tions, Appl. Math. Comput., in press.

Satulovsky, J.E., Tome, T., 1994. Stochastic lattice gas modelfor a predator–prey system. Phys. Rev. E 49, 5073–5079.

Schmitz, O.J., 1997. Press perturbations and the predictabilityof ecological interactions in a food web. Ecology 78,55–69.

Soule, M.E., 1987. Viable Populations for Conservation. Cam-bridge University Press, Cambridge.

Sutherland, B.R., Jacobs, A.E., 1994. Self-organization andscaling in a lattice prey–predator model. Complex Syst. 8,385–405.

Tainaka, K., 1988. Lattice model for the Lotka–Volterrasystem. J. Phys. Soc. Jpn. 57, 2588–2590.

Tainaka, K., 1989. Stationary pattern of vortices or strings inbiological systems: lattice version of the Lotka–Volterramodel. Phys. Rev. Lett. 63, 2688–2691.

Tainaka, K., 1993. Paradoxical effect in a 3-candidates votermodel. Phys. Lett. A 176, 303–306.

Tainaka, K., 1994a. Intrinsic uncertainty in ecologicalcatastrophe. J. Theor. Biol. 166, 91–99.

Tainaka, K., 1994b. Vortices and strings in a model ecosystem.Phys. Rev. E 50, 3401–3409.

Tainaka, K., 1995. Indirect effect in cyclic voter models. Phys.Lett. A 207, 53–57.

Tainaka, K., 2000. Patch dynamics for Prisoner’s Dilemmagame. Sem. Notes Math. Sci. 3 (2000), 1–18.

Tainaka, K., Itoh, Y., 1991. Topological phase transition inbiological ecosystems. Europhys. Lett. 15, 399–404.

Tainaka, K., Fukazawa, S., 1992. Spatial pattern in a chemicalreaction system: prey and predator in the position-fixedlimit. J. Phys. Soc. Jpn. 61, 1891–1894.

Tainaka, K., Itoh, Y., 1996. Glass effect in inbreeding-avoid-ance systems: minimum viable population for outbreeders.J. Phys. Soc. Jpn. 65, 3379–3385.

Tainaka, K., Araki, N., 1999. Press perturbation in latticeecosystems: parity law and optimum strategy. J. Theor.Biol. 197, 1–13.

Tainaka, K., Hoshiyama, M., Takeuchi, Y., 2000. Dynamicprocess and variation in the contact process. Phys. Lett. A272, 416–420.

Takeuchi, Y., 1996. Global Dynamical Properties of Lotka–Volterra System. World Scientific, Singapore.

Taylor, P.D., 1992. Altruism in viscous populations—an in-clusive fitness model. Evol. Ecol. 6, 352–356.

Thomas, C.D., 1990. What do real population dynamics tell usabout minimum viable population sizes. Conserv. Biol. 4,324–327.

Trivers, R., 1971. The evolution of reciprocal altruism. Q.Rev. Biol. 46, 35–57.

Van Baalen, M., Rand, D.A., 1998. The unit of selection inviscous populations and the evolution of altruism. J.Theor. Biol. 193, 631–648.

Wilcove, D.S., Mcmillan, M., Winston, K.C., 1993. Whatexactly is an endangered species? An analysis of the USendangered species list: 1985–1991. Conserv. Biol. 7, 87–93.

Wilkinson, G.S., 1984. Reciprocal food sharing in the vampirebat. Nature 308, 181–184.

Wilson, D.S., 1983. The group selection controversy: historyand current status. Ann. Rev. Ecol. Syst. 14, 159–187.

Wynne-Edwards, V.C., 1962. Animal Dispersion in Relationto Social Behavior. Oliver & Boyd, Edinburgh.

Yodzis, P., 1988. The indeterminacy of ecological interactionsas perceived through perturbation experiments. Ecology69, 508–515.

patch dynamics based on prisoner's dilemma game: superiority of golden rule

Documents