mark broom - game theory and dynamic games€¦ · for an evolutionary game we also need a...

Evolutionary games

Mark Broom

City University London

Game theory summer schoolCampione d’Italia

September 4-9 2016

Mark Broom (City University London) Campione d’Italia, Sep 4-9 1 / 111

Credits

Credits

This talk is largely based upon the book:Broom, M. and Rychtar, J. (2013) Game-Theoretical Models in Biology,Chapman and Hall/ CRC Press.The numerous references to other works that are in the book above areomitted from the talk for the sake of brevity.


Introduction to evolutionary games

Outline

1 Introduction to evolutionary gamesWhat is a game?Two approaches to game analysisSome classic games

2 Nonlinear and multi-player gamesPlaying the fieldMulti-player games

3 Games in finite and structured populationsFinite populations and the Moran processGames in finite populationsEvolution and evolutionary games on graphs


Introduction to evolutionary games What is a game?

The key game components

In game theory, a game is comprised of three key elements.1. The players: often there are two players, who sometimes can be viewed asdistinct, so it matters which is “player 1” and which “player 2”. Often inevolutionary games this distinction is not important.2. The strategies: these are the choices of the players. Strategies can be pureor mixed, as described below.3. The payoffs: these are the rewards to the players, and are a function of thestrategies chosen.For an evolutionary game we also need a population, and a means for apopulation to evolve, an evolutionary dynamics.Games can be represented in either normal form or in some cases extensiveform. All of the games we consider here are of the former type.



Pure strategies

A pure strategy is a single choice of what strategy to play.There can be a finite or infinite number of pure strategies. In the Prisoner’sdilemma game (see later) the pure strategies are “play Cooperate” and “playDefect.”If the game is modified so that the players play the Prisoner’s dilemma gameover a number of rounds, the number pure strategies can get very large.A pure strategy in such a case specifies what to play in every round,conditional on every possible sequence played previously.Biology plays an important role in trimming the set of pure strategies.For example, if players have a short term memory, a strategy can be a rule like“start with Cooperate and then play whatever the opponent played in theprevious round”.



Mixed strategies I

If the pure strategies comprise a finite set {S1, S2, . . . , Sn}, then a mixedstrategy is defined as a probability vector p = (p1, p2, . . . , pn) where pi is theprobability that the player will choose pure strategy Si.For example, in the Prisoner’s dilemma, a player may choose to play each ofCooperate and Defect half of the time, which would be represented by thevector (1/2, 1/2).The Support of p, S(p), is defined by S(p) = {i : pi > 0}, so that it is the setof pure strategies which have non-zero chance of being played by a p-player.For example, the support of the above strategy (1/2, 1/2) is {1, 2}.



Mixed strategies II

A pure strategy can be seen as a special case of a mixed strategy; Si can beidentified with a “mixed strategy” (0, . . . , 0, 1, 0, . . . , 0) with 1 at the ith place.The set of all mixed strategies can be represented as a simplex in Rn withvertices at {S1, S2, . . . , Sn}.We can see a mixed strategy as a convex combination of pure strategies,

p = (p1, p2, . . . , pn) =

n∑i=1

piSi. (1)

The notion of a mixed strategy is naturally extended even to cases where theset of pure strategies is infinite, as we see with the “war of attrition” game.



Simplex representation of mixed strategies

1

1

pure strategyS1 = (1, 0)


mixed strategyp = (p1, p2) = p1S1 + p2S2

p1

p2

pure strategyS1 = (1, 0, 0)


mixed strategyp = (p1, p2, p3)= p1S1 + p2S2 + p3S3

p1

p2

p3p′3

p′1

mixed strategyp′ = (p′1, 0, p

′3)

= p′1S1 + p′3S3


[]pure strategyS1 = (1, 0)


distance 1

mixed strategyp = (p2, p1)

p1 p2

[]


p = (p1, p2, p3)

p3

p2p1



Figure: Two ways to visualize pure and mixed strategies



Payoff matrices I

In general, payoffs for a game played by two players with each having onlyfinitely many pure strategies can be represented by two matrices.For example, if player 1 has available the strategy set S = {S1, . . . , Sn} andplayer 2 has the strategy set T = {T1, . . . ,Tm}, then the payoffs in this gameare completely determined by the pair of matrices

A = (aij)i=1,...,n;j=1,...,m,B = (bij)i=1,...,m;j=1,...,n, (2)

where aij and bji represent rewards to players 1 and 2 respectively after player1 chooses pure strategy Si and player 2 chooses pure strategy Tj.We thus have all of the possible rewards in a game given by a pair of n× mmatrices A and BT , which is known as a bimatrix representation.Entries are often given as a pair of values in a single matrix.



Payoff matrices II

Sometimes, as in the Prisoner’s Dilemma, the choice of which player is player1 and which player 2 is arbitrary, and thus the strategies that they haveavailable to them are identical, i.e. n = m and (after possible renumbering)Si = Ti for all i.Further, since the ordering of players is arbitrary, we can switch the twowithout changing their rewards, so that their payoff matrices satisfy bij = aij,i.e. A = B.We thus now have all of the possible rewards in a game given by a singlen× n matrix

A = (aij)i,j=1,...,n, (3)

where in this case, aij is a reward to the player that played strategy Si while itsopponent played strategy Sj.



Payoffs as a matrix product

Consider a game whose payoffs are given by a matrix A.If player 1 plays p and player 2 plays q, then the proportion of games thatinvolve player 1 playing Si and player 2 playing Sj is simply piqj.The reward to player 1 in this case is aij. The expected reward to player 1,which we shall write as E[p,q], is thus obtained by averaging, i.e.

E[p,q] =∑

i,j

aijpiqj = pAqT. (4)

When comparing payoffs, we can ignore difficult cases involving equalities byassuming our games are generic. In most of the following we will make thisassumption.



Games in biological settings

So far we have considered a single game contest between individuals.However, individuals are not usually involved in a single contest only.More often, they are repeatedly involved in the same game (either with thesame or different opponents).Each round of the game contributes a relatively small portion of the fitness tothe total reward.Of ultimate interest is the function E [σ; Π] describing the fitness of a givenindividual using a strategy σ in a given population represented by Π.We shall represent by δp a population where the probability of a randomlyselected player being a p-player is 1.



Fitness functions

We note that E [σ; Π] can be observed by biologists only if there are playersusing σ in the population.However, from the mathematical point of view, even expressions like E [q; δp]for q 6= p are considered.In an infinite population there may be a sub-population (e.g. a singleindividual) which makes up a proportion 0 of the population, and so suchpayoffs are logical.For any strategy p, as described above, we let δp denote the population wherea randomly selected player plays strategy p with probability 1.Let δi denote the population consisting of individuals playing strategy Si (withprobability 1). Similarly,

∑i piδi means the population where the proportion

of individuals playing strategy Si is pi.


Introduction to evolutionary games Two approaches to game analysis

Evolutionary dynamics

The evolution of a population can be modelled using evolutionary dynamics,where the number/ proportion of individuals playing a given strategy changesaccording to their fitness. We shall assume a population consisting only ofpure strategists.Consider a population described by pT =

∑i piδi, i.e. the frequency of

individuals playing strategy Si is pi.To simplify notation, let fi(p) denote the fitness of individuals playing Si inthe population in this section.Further, for the purpose of deriving the equation of the dynamics, assume thatthe population has N individuals and Ni = piN of those are using strategy Si

(this is convenient for the immediate derivations below, but often we shallassume infinite populations and only the frequencies matter).



The discrete replicator dynamics I

Suppose that the population has discrete generations that are non-overlappingand asexual reproduction. We assume that each individual playing strategy Si

generates fi(p) copies of itself in the next generation so thatNi(t + 1) = Ni(t)fi

(p(t)

)and thus get the discrete replicator dynamics

pi(t + 1) =Ni(t + 1)

N(t + 1)=

Ni(t)fi(p(t)

)∑j

Nj(t)fj(p(t)

)=pi(t)

fi(p(t)

)f̄(p(t)

) ,(5)

wheref̄ (p) =

∑i

pifi(p) (6)

is the average fitness in the population.



The discrete replicator dynamics II

In the case of matrix games where payoffs are given by matrix A, we getfi(p) = (ApT)i + β and f̄ (p) = pApT + β where β is a background fitness.Thus the dynamics (5) becomes

pi(t + 1) = pi(t)

(A(p(t)

)T)

i+ β

p(t)A(p(t)

)T+ β

. (7)

Note that, in contrast to the continuous dynamics below, where β has noeffect, the background fitness it can have a significant effect on the dynamicshere.In general, for small β evolution occurs faster, but the process is less stablethan for larger β. When β →∞ and the generation times tend to zero, thecontinuous dynamics discussed below is a limiting case of the discretedynamics.



The continuous replicator dynamics I

If the population is very large, has overlapping generations and asexualreproduction, we may consider Ni and pi = Ni/N to be continuous variables.Population growth is given by the differential equation

ddt

Ni = Nifi(p(t)

), (8)

and we get the continuous replicator dynamics,

ddt

pi = pi

(fi(p(t)

)− f̄ (p(t))

). (9)

As before, for matrix games, the dynamics (9) becomes

ddt

pi = pi

((A(p(t)

)T)

i− p(t)A

(p(t)

)T). (10)



The continuous replicator dynamics II

As an example, consider the dynamics (10) for the Rock-Scissors-Papergame, defined by the following payoff matrix. 0 a3 −b2

−b3 0 a1a2 −b1 0

. (11)

In this case, there is a unique internal equilibrium given by

p =1K

(a1a3 + b1b2 + a1b1, a1a2 + b2b3 + a2b2, a2a3 + b1b3 + a3b3), (12)

where K is chosen so that the three terms add to 1.There are three qualitatively different outcomes of the dynamics, dependingupon the sign of a1a2a3 − b1b2b3 as shown below in Figure 2.



The continuous replicator dynamics III

[] (1,0,0) (0,1,0)

(0,0,1)

(1/3,1/3,1/3) (5/19, 8/19, 6/19)Attracting rest point

[] (1,0,0) (0,1,0)

(0,0,1)

(1/3,1/3,1/3)

[] (1,0,0) (0,1,0)

(0,0,1)

(1/3,1/3,1/3) (4/16, 7/16, 5/16)Unstable rest point

Figure: Continuous replicator dynamics for the RSP game given by matrix (11); (a)a1 = a2 = b3 = 2, a3 = b1 = b2 = 1, an asymptotically stable and globally attractingequilibrium, (b) a1 = a2 = a3 = b1 = b2 = b3 = 1, a stable (but not asymptoticallystable) equilibrium with closed orbits, (c) a1 = a2 = a3 = b1 = 1, b2 = b3 = 2, anunstable and globally repelling equilibrium.

Note that there are a range of other dynamics used, for example the imitationdynamics. Of particular importance is Adaptive Dynamics.



The static approach

Static analysis does not consider how the population reached a particularpoint in the strategy space.Instead, we assume that the population is at that point, and ask, is there anyincentive for any members of the population to change their strategies?A strategy S is a best reply (alternatively a best response) to strategy T if

f (S′,T) ≤ f (S,T); for all strategies S′, (13)

where f (S,T) denotes the payoff to a player using S against a player using T .A strategy p* is a Nash equilibrium if

f (S′, S) ≤ f (S, S); for all strategies S′. (14)



Evolutionarily Stable Strategies

Any “good” strategy must be a best reply to itself. However, a strategy beinga best reply to itself does not prevent invasion.Consider a population consisting of individuals, the vast majority of whomadopt a strategy S, while a very small number of “mutants” adopt a strategy M.The strategies S and M thus compete in the population (1− ε)δS + εδM forsome small ε > 0 (rather than in the population δS), and it is against such apopulation that S must outcompete M.We say that a strategy S is evolutionarily stable against strategy M if there isεM > 0 so that for all ε < εM we have

E [S; (1− ε)δS + εδM] > E [M; (1− ε)δS + εδM]. (15)

S is an evolutionarily stable strategy (ESS) if it is evolutionarily stable againstM for every other strategy M 6= S.



ESSs for matrix games I

In the case of matrix games, the linearity of payoffs gives

E [p; (1− ε)δp + εδq] = E[p, (1− ε)p + εq] = (16)

pA((1− ε)p + εq)T = (1− ε)pApT + εpAqT. (17)

A strategy p is an ESS if for every other strategy q 6= p there is εp > 0 suchthat for all ε < εp we have

E[p, (1− ε)p + εq] > E[q, (1− ε)p + εq]. (18)



ESSs for matrix games II

From payoff linearity, we have the equivalent definition for matrix games.

Theorem 1A (pure or mixed) strategy p is an Evolutionarily Stable Strategy (ESS) for amatrix game, if and only if for any mixed strategy q 6= p

E[p,p] ≥ E[q,p], (19)

If E[p,p] = E[q,p], then E[p,q] > E[q,q]. (20)

The proof is straightforward, and we will not show it here.If (20) does not hold, then p may be invaded by a mutant that does equallywell against the majority of individuals in the population (that adopts p) but isgetting a (tiny) advantage against them by doing better in the (rare) contestswith like mutants.



ESSs for matrix games III

Alternatively there is the possibility that the mutant and the residents doequally well against the mutants too.In this latter case invasion can occur by “drift”; both types do equally well, soin the absence of selective advantage random chance decides whether thefrequency of mutants increases or decreases.Thus, both conditions (19) and (20) are needed for p to resist invasion by amutant q.If the conditions hold for any q 6= p, then p can resist invasion by any raremutant and so p is an evolutionarily stable strategy.



Dynamics versus statics

Since it may be argued (with some justification) that the dynamics approachmay model the biology in a more realistic way, we point out the similaritiesand difference between the two approaches in this section.Dynamic and static analyses are mainly complementary, however therelationship between the two is not straightforward, and there is someapparent inconsistency between the theories.As the concept of an ESS as an uninvadable strategy is partially based on thesame idea as that of replicator dynamics, we look at replicator dynamics.Comparing ESS analysis and replicator dynamics, we see that the informationrequired for each type of analysis is different.



Contrasting populations

To determine whether p is an ESS, we need the minimum of a function

q→ E [p; (1− ε)δp + εδq]− E [q; (1− ε)δp + εδq] (21)

to be attained for q = p for all sufficiently small ε > 0.To understand the replicator dynamics, we require E [Si; pT] for all i and all p.Thus a major difference between static analysis and replicator dynamics isthat the static analysis is concerned with monomorphic populations δp whilethe replicator dynamics studies mixed populations pT =

∑i piδi.

Both analyses can produce the same (or at least similar) results only if there isan identification between δp and pT such as in the case of matrix games.Note that most of the comparative analysis between ESSs and replicatordynamics assumes matrix games.



ESSs and replicator dynamics in matrix games

Theorem 2 (Folk theorem of evolutionary game theory) In the following, weconsider a matrix game with payoffs given by matrix A. (10).

1) If p is a Nash equilibrium of a game, in particular if p is an ESS of amatrix game, then pT is a rest point of the dynamics , i.e. the populationdoes not evolve further from the state pT =

∑i piδi.

2) If p is a strict Nash equilibrium, then p is locally asymptotically stable,i.e. the population converges to the state pT =

∑i piδi if it starts

sufficiently close to it.

3) If the rest point p∗ of the dynamics is also the limit of an interior orbit(the limit of p(t) as t→∞ with p(0) ∈ int(S)), then it is a Nashequilibrium.

4) If the rest point p is Lyapunov stable (i.e. if all solutions that start outnear p stay near p forever), then p is a Nash equilibrium.



Long-term dynamic behaviour

Theorem 3Any ESS is an attractor of the replicator dynamics (i.e. has some set of initialpoints in the space that lead to the population reaching that ESS). Moreover,the population converges to the ESS for every strategy sufficiently close to it;and if p is an internal ESS, then global convergence to p is assured.Also, if the replicator dynamics has a unique internal rest point p*, undercertain conditions we have

limt→∞

1T

∫ T

0pi(t)dt = p∗i . (22)

Thus the long-term average strategy is given by this rest point, even if at anytime there is considerable variation.



Different invaders I

The results above are very helpful because it means that, for matrix games,identifying ESSs and Nash equilibria of a game gives a lot (sometimespractically all) of the important information about the dynamics.For example, if p is an internal ESS, and there is no other Nash equilibrium,then global convergence to p is assured.Yet, there are cases when an ESS analysis does not provide a completepicture. In particular, there are attractors that are not ESSs.To see this, consider the matrix 0 1 −1

−2 0 22 −1 0

. (23)



Different invaders II

We note that the above matrix is a variant of the Rock-Scissors-Paper game.We can show that the replicator dynamics for this game has a unique internalattractor. However, this internal attractor is not an ESS.This occurs because we can find an invading strategy for p where the payoffsto the different components change in such a way under the dynamics that it isinevitably forced into a combination that no longer invades.Thus if the invader is comprised of a combination of pure strategists it isbeaten, but if it is comprised of mixed strategists it is not.


Introduction to evolutionary games Some classic games

Hawk-Dove game I

Each individual plays against other randomly chosen individuals.A contest contains two equally able players competing for a reward (e.g. aterritory) of value V > 0. Each of the contestants may play one of two purestrategies, Hawk (H) and Dove (D).If two Doves meet, they each display, and each will gain the reward withprobability 1/2, giving an expected reward of E[D,D] = V/2.If a Hawk meets a Dove, the Hawk will escalate, the Dove will retreat(without injury) and so the Hawk will gain the reward V , and the Dove willreceive 0. Hence, E[H,D] = V and E[D,H] = 0.If two Hawks meet, they will escalate until one receives an injury, which is acost C > 0 (the equivalent of a reward of −C). The injured animal retreats,leaving the other with the reward. The expected reward for each individual isthus E[H,H] = (V − C)/2.



Hawk-Dove game II

We have a matrix game with two pure strategies, where the payoff matrix is

Hawk Dove

HawkV − C

2V

Dove 0V2

. (24)

We denote a mixed strategy p = (p, 1− p) to mean to play Hawk withprobability p and to play Dove otherwise.Since E[H,D] > E[D,D], Dove is never an ESS.Hawk (or the strategy (1, 0)) is a pure ESS if E[H,H] > E[D,H] i.e. ifV > C. In fact Hawk is a pure ESS if V ≥ C.



Hawk-Dove game III

For a mixed strategy p = (p, 1− p) to be an ESS we require that

E[H,p] = E[D,p]⇒ pV − C

2+ (1− p)V = (1− p)

V2

(25)

which occurs if p = V/C.It is easy to show that the required stability condition E[p,p] > E[q,p] holdsfor all q 6= p.Thus we get that p = (V/C, 1− V/C) is the unique ESS when V < C.This means that there is a unique ESS in the Hawk-Dove game, irrespective ofthe values of V and C.



Hawk-Dove game IV

Letting p = (p, 1− p) describe the mixed population of pure strategists withproportion of Hawks p, the replicator dynamics gives

dpdt

= p((Ap)1 − pApT) = p(1− p)

V − pC2

. (26)

For V < C, p is driven towards the ESS value p = V/C in all cases.

0 VC 1 p 0 1 p

dpdt

a) b)

Dove

Hawk Hawk

Dove

dpdt

Figure: Hawk-Dove game replicator dynamics; (a) V < C, b) V > C.



Prisoner’s dilemma I

In the Prisoner’s dilemma, a pair of individuals can either cooperate (play C)or try to obtain an advantage by defecting and exploiting the other (play D).We obtain a two player game where the payoffs are given by the payoff matrix

(Cooperate DefectCooperate R SDefect T P

), (27)

where R, S,T and P are termed the Reward, Sucker’s payoff, Temptation andPunishment respectively.Whilst the individual numbers are not important, for the classical dilemma weneed T > R > P > S.It turns out that the additional condition 2R > S + T is also necessary for theevolution of cooperation.



Prisoner’s dilemma II

One prisoner sits in their cell and thinks about what to do. If the othercooperates then the reward for the first to play Defect is bigger than that if heplays Cooperate, so they should play Defect.But if the other plays Defect it is also the case that Defect is best. Thusirrespective of what the other will do, Defect is best.The other prisoner comes to the same conclusion and so both players defect,and receive a payoff of P.The outcome seems paradoxical because R > P, i.e. they would both be betteroff if they cooperated.Each player following their own self interest in a rational manner has led tothe worst possible collective result, with a total payoff of 2P.



The war of attrition I

The two games above are classic examples of a matrix game, where there area finite number of pure strategies. We now introduce a game with an infinite(and uncountable) number of pure strategies.Consider a situation that arises in the Hawk-Dove game type conflict, namelytwo individuals compete for a reward of value V and both choose to display.Both individuals keep displaying for some time, and the first to leave does notreceive anything, the other gaining the whole reward.For simplicity, we assume that individuals have no active strategy, i.e. thetime they are prepared to wait is determined before the conflict begins.Although there is no real harm done to the individuals during their displays,each individual pays a cost related to the length of the contest.



The war of attrition II

A pure strategy of an individual is to be prepared to wait for a predeterminedtime t ≥ 0.We will denote such a strategy by St. It is clear that there are uncountablymany pure strategies.A mixed strategy of an individual is a measure p on [0,∞).The measure p determines that an individual chooses a strategy from a set Awith probability p(A).One can see the mixed strategy p given by a density function p(x).



The war of attrition III

The reward for the winner of the contest is given by V . The one that isprepared to display the longer gets the reward, but the cost of the contest oflength x is C(x) = cx for some given constant c.For pure strategies Sx and Sy we get

E[Sx, Sy] =

V − cy x > y,V/2− cx x = y,−cx x < y.

(28)

For mixed strategies p,q we get

E[p,q] =

∫∫(x,y)∈[0,∞)2

E[Sx, Sy]dp(x)dq(y), (29)



The war of attrition IV

It is immediately clear that there is no pure ESS.If p with a density function p is an ESS, then for almost all t ≥ 0 for whichp(t) > 0 we need

E[St,p] = E[p,p], (30)

which, by (29) yields∫ t

0(V − cx)p(x)dx +

∫ ∞t

(−ct)p(x)dx = E[p,p]. (31)



The war of attrition V

Differentiating (31) with respect to t (assuming that such a derivative exists)gives

(V − ct)p(t)− c∫ ∞

tp(x)dx + ctp(t) = 0. (32)

Solving this equation with appropriate boundary conditions yieldsp(t) = (c/V) exp(−ct/V).We can verify that this is indeed an ESS by proving stability, but we will notdo this here.



The sex ratio game I

Why is the sex ratio in most animals close to a half? A naive look at thisproblem would contend that for the good of the species, there should be farless males than females.Indeed it is often the case that most offspring are sired by a relatively smallnumber of males and most males make no contribution at all.To make the problem more specific, let us suppose that in a given population,any individual will have a fixed number of offspring, but that it can choose theproportion which are male and female.We also assume that all females (males) are equally likely to be the mother(father) of any particular child in the next generation.Our task is to provide a reasonable argument to answer the question of what isthe best choice of the sex ratio.



The sex ratio game II

A strategy of an individual will be a choice of the proportion of maleoffspring. We consider a small proportion of the population (fraction ε) ofindividuals playing a (potentially) different strategy to the rest.Let p denote the strategy of our invading group, and m denote the strategyplayed by the rest of the population.Since every individual has the same number of offspring, we cannot use thisnumber as the payoff, and we consider the number of grandchildren.Assume that the total number of individuals in the next generation is N1, andthe total number in the following generation is N2.In fact, as we shall see, these numbers are irrelevant, but it is convenient toconsider them at this stage.



The sex ratio game III

The proportion of males in the generation of offspring is m1 = (1− ε)m + εp.Given m, what is the best reply p to m? A strategy p then will be an ESS if p isthe best reply to itself and uninvadable by any other strategy.The expected number of offspring of one of our focal individual’s offspring is

E [p; δm] = p× 1m1N1

× N2 + (1− p)× 1(1− m1)N1

× N2

=N2

N1

(p

m1+

1− p1− m1

)≈ N2

N1

(pm

+1− p1− m

).

(33)

To find the best choice of p, we maximise this function E [p; δm]. This givesp = 1 if m < 1/2 and p = 0 if m > 1/2. For m = 1/2 we need ε > 0, andthen m = 1/2 performs strictly better than any mutant.


Nonlinear and multi-player games

Outline





Nonlinear and multi-player games Playing the field

Playing the field

In this section we consider payoff functions of the form

E [p; Π] =∑

pifi(Π) (34)

where the fi’s are in general nonlinear functions, and Π represents the strategyplayed by the population.Playing the field games are perhaps the most straightforward way ofincorporating nonlinearity into a game model, as the fitness function of theindividuals involved automatically includes the population frequencies of thedifferent strategies.



Example: the sex ratio game

An example is the sex ratio game that we considered earlier, where a strategywas effectively mixed, with two pure strategies “male” and “female”. Thefitness of an individual with strategy p was given by (33) as

E [p; δm] =pm

+1− p1− m

(35)

so that in the notation of equation (34) we have

f1(m) =1m, (36)

f2(m) =1

1− m. (37)



Optimal foraging

Consider a single species foraging on N patches, with resources ri > 0 fori = 1, . . . ,N which are shared equally by all who choose the patch.There are N pure strategies for this game, each corresponding to foraging onone patch only, a mixed strategy x = (xi) meaning to forage at patch i withprobability xi.The general payoff to an individual using strategy x = (xi) against apopulation playing y = (yi) is

E [x; δy] =

∞, if xi > 0 for some i such that yi = 0,

N∑i;xi>0

xiri

yiotherwise,

(38)

where ri is a constant corresponding to the quality of a patch i.



Otpmial foraging II

It is obvious from (38) that if there is an ESS p, it must be internal, i.e. wemust have pi > 0 for all i = 1, . . . ,N.Thus any problematic cases with potentially infinite payoffs do not arise inany analysis.In particular Theorem 6 still holds even though the fitness functions are notcontinuous everywhere, since they are continuous in the vicinity of the ESS.Note that the sex ratio game is a special case with N = 2 and r1 = r2.



Optimal foraging: the ESS I

Now, let us show that p = (pi) given by pi/∑N

i=1 ri is an ESS.Clearly, E [q; δp] = E [p; δp] for all q. Moreover (since this game satisfiespolymorphic monomorphic equivalence) we have

E [x; (1− u)δy + uδz] = E [x; δ(1−u)y+uz] (39)

and thus

hp,q,u = E [p; (1− u)δp + uδq]− E [q; (1− u)δp + uδq] (40)

=

N∑i=1

(pi − qi)ri

pi + u(qi − pi)(41)

=N∑

i=1

pi − qi

piri

(1− u

qi − pi

pi+ . . .

), (42)



Optimal foraging: the ESS II

This implies that

∂

∂u

∣∣∣u=0

hp,q,u =

N∑i=1

ri

(pi − qi

pi

)2

> 0. (43)

Thus p is an ESS, using Theorem 4.Also, note (and this is called Parker’s matching principle), that at the ESSstrategy p, we have

pi

pj=

ri

rj. (44)


Nonlinear and multi-player games Multi-player games

Introducing multi-player games

In the previous sections we have considered games with two individuals only,or games played against “the population”.We now consider situations where individual conflicts consist of a number ofindividuals greater than two.Such games have only rarely been considered with regard to biologicalpopulations, although multi-player games are common in economics.All the games that we consider are contests involving a randomly selectedgroup of (at least three) players from a large population.



Introducing multi-player matrix games

We shall consider an infinite population, from which groups of m playerswere selected at random to play a game.The expected payoff to an individual is obtained by averaging over therewards, weighted by their probabilities, as for matrix games.In its most general form where the ordering of individuals matter, extendingthe bimatrix game case to m players, the payoff to each individual in positionk is governed by an m-dimensional payoff matrix.It is assumed that there is no significance to the ordering of the players, as anatural extension of matrix games (in contrast to bimatrix games).Thus the payoff to an individual depends only upon its strategy and thecombination of the strategies of its opponents and only one suchm-dimensional matrix is needed.



Symmetric multi-player matrix games

We will call such games symmetric and write the payoffs for a 3-playern-strategy game as A = (A1,A2, . . . ,An) where Aj are the payoffs assumingthe focal player plays pure strategy Sj, and

Aj =

aj11 aj12 · · · aj1n

aj21 aj22 · · · aj2n...

.... . .

...ajn1 ajn2 · · · ajnn

. (45)

To add an extra player, the number of matrices required in this formulation ismultiplied by the number of strategies n. There are, in a full general case, nm

entries of the matrices.



Multi-player matrix payoffs I

However, we have some symmetry conditions.For the three player case, these are

apqr = aprq, for all p, q, r = 1, 2, . . . , n. (46)

In general these areai1...im = ai1σ(i2)...σ(im) (47)

for any permutation σ of the indices i2, . . . , im.The payoff to an individual playing p in a contest with individuals playingp1,p2, . . . ,pm−1 respectively is written as E[p; p1,p2, . . . ,pm−1].As the ordering is irrelevant, when some strategies are identical a powernotation is used, for example E[p; p1,p2,p3

m−3].



Multi-player matrix payoffs II

General payoffs are given as follows

E[p; p1,p2, . . . ,pm−1] =

n∑i=1

pi

n∑i1=1

· · ·n∑

im−1=1

aii1i2...i(m−1)

k−1∏j=1

pj,ij , (48)

where pj = (pj,1, pj,2, . . . , pj,n).As long as groups are selected from the population completely at random, asis usually assumed, then there is no real difference between symmetric andnon-symmetric games.For example in a population playing 3-player games every individual isequally likely to occupy any of the ordered positions, and in particular theterm aijk will have identical weighting to aikj in the payoff to an i-player. Thusthe sum of these two can be replaced by twice their average.



Multi-player matrix payoffs III

In the context of matrix games, a 2-player game is called (super-)symmetric ifaij = aji for all i, j and a multi-player matrix is super-symmetric if

ai1...im = aσ(i1)...σ(im) (49)

for any permutation σ of the indices i1, . . . , im.For example, for super-symmetric three-player three strategy games, there areten distinct payoffs.Without loss of generality we can define the three payoffsa111 = a222 = a333 = 0, and this leaves seven distinct payoffs to considera112, a113, a221, a223, a331, a332 and a123.Before we specify the payoff E [p; Π] to an individual playing p in thepopulation described by Π, let us consider the specific case of two strategygames.



Payoffs for two strategies, three and four players I

The complete payoffs to the three-player two strategy game can be written as(a111 a112a121 a122

)(a211 a212a221 a222

)(50)

where, as in (47) a112 = a121 and a212 = a221.Similarly for the four player two strategy game we have(

a1111 a1112a1121 a1122

) (a1211 a1212a1221 a1222

)(

a2111 a2112a2121 a2122

) (a2211 a2212a2221 a2222

) (51)

with symmetry conditions a1112 = a1121 = a1211 etc, see the following figure.



Payoffs for two strategies, three and four players II

a111 a121

a112 a122

a211 a121

a212 a222

a111 a122a112 = a121

a211 a222a212 = a221

a1111

a1112

a1121a1211

a1221

a1222

a1212

a1122

a2112

a2111a2121

a2212

a2122a2222

a2221

a2211

a1111

a2111

a1112a1121a1211

a1122a1212a1221 a1222

a2222a2112a2121a2211

a2122a2212a2221

Figure: Visualization of payoffs in 2-strategy m-player games.



Example payoffs

Thus, the payoffs of the game are completely determined by specifying thepayoffs to an individual playing pure strategy i = 1, 2 against m− 1 players, jof which play strategy S1 (and the other m− 1− j play strategy S2).Let us denote these payoffs by αij.Later we look at the game with payoffs(

−3/32 00 −13/96

) (0 −13/96

−13/96 0

)(

0 −13/96−13/96 0

) (−13/96 0

0 −3/32

).

(52)



Mixed strategy payoffs

Consider an individual playing strategy x in a population playing y.A group of m− 1 opponents is chosen and each one of them chooses to playstrategy S1 with probability y1 and strategy S2 with probability y2. Thus

E [x; δy] =

m−1∑l=0

(m− 1

l

)yl

1ym−1−l2 E[x; Sl

1Sm−1−l2 ], (53)

where

E[x; Sl1, S

m−1−l2 ] =

2∑i=1

xiαil. (54)

Iit does not matter whether the population is polymorphic or monomorphicand playing the mean strategy, and so multi-player matrix games have thepolymorphic-monomorphic equivalence property.



ESSs for multi-player games I

The following definition of an ESS for an m-player game is a naturalextension of the definition for a two player game.A strategy p in an m-player game is called evolutionarily stable against astrategy q if there is an εq ∈ (0, 1] such that for all ε ∈ (0, εq]

E [p; (1− ε)δp + εδq] > E [q; (1− ε)δp + εδq], (55)

where

E [x; (1− ε)δy + εδz] =

m−1∑l=0

(m− 1

l

)(1− ε)lεm−1−lE[x; yl, zm−1−l]. (56)

We say that p is an ESS for the game if for every q 6= p, there is εq > 0 suchthat (55) is satisfied for all ε ∈ (0, εq].



ESSs for multi-player games II

Similarly as in Theorem 1, we get the following Theorem whose proof wewill not show.

Theorem 4For an m-player matrix game, the mixed strategy p is evolutionarily stableagainst q if and only if there is a j ∈ {0, 1, . . . ,m− 1} such that

E[p; pm−1−j,qj] > E[q; pm−1−j,qj], (57)

E[p; pm−1−i,qi] = E[q; pm−1−j,qi] for all i < j. (58)

A strategy p is called an ESS at level J if, for every q 6= p, the conditions(57)-(58) of Theorem 4 are satisfied for some j ≤ J and there is at least oneq 6= p for which the conditions are met for j = J precisely.



ESSs for multi-player games III

If p is an ESS, then by Theorem 4, for all q,

E[p; pm−1] ≥ E[q; pm−1]. (59)

Since the payoffs are linear in the strategy of the focal player it follows that

E[p; pm−1] = E[q; pm−1], for all q with S(q) ⊆ S(p). (60)

In the generic case, any pure ESS is of level 0. A mixed ESS cannot be oflevel 0 but in the generic case, any mixed ESS must be of level 1.



ESS supports for multi-player games I

Analogues of the strong restrictions on possible combinations of ESSs formatrix games do not hold for multi-player games.The Bishop-Cannings Theorem (which implies that the support of one ESScannot be a subset of the support of another) fails already for m = 3.For m > 3, there can be more than one ESS with the same support as we shallsee in a later example. On the other hand, we still have the following form = 3.A pattern of ESSs is a collection of ESS supports. A pattern is attainable if amatrix exists which has that pattern.

Theorem 5It is not possible to have two ESSs with the same support in a three playermatrix game.



ESS supports for multi-player games II

ProofSuppose that p is an ESS of a 3-player game. Then, by Theorem 4 exactly oneof the following three conditions holds for any q 6= p,

(i) E[p; p,p] > E[q; p,p],

(ii) E[p; p,p] = E[q; p,p] and E[p; q,p] > E[q; q,p],

(iii) E[p; p,p] = E[q; p,p],E[p; q,p] = E[q; q,p] andE[p; q,q] > E[q; q,q].

Moreover, since q is also an ESS with S(p) = S(q) we have,

E[p; p,p] = E[q; p,p], (61)

E[q; q,q] = E[p; q,q]. (62)



ESS supports for multi-player games III

Hence p must satisfy condition (ii) and thus

E[p; q,p] > E[q; q,p] = E[q; p,q]. (63)

However, by repeating the same process yet starting with q as an ESS, we getthe reverse inequality which is a contradiction.



The incentive function I

Recall that the payoffs of the m-player two strategy matrix game are given byαil for i = 1, 2 and l = 0, 1, . . . ,m− 1.Let us define βl = α1l − α2l and consider

h(p) = E [S1; δ(p,1−p)]− E [S2; δ(p,1−p)] (64)

=

m−1∑l=0

(m− 1

l

)βlpl(1− p)m−l−1. (65)

The incentive function h quantifies the benefits of using strategy S1 overstrategy S2 in a population where everybody else uses strategy p = (p, 1− p).Note that h is differentiable, and that the replicator dynamics now becomes

dqdt

= q(1− q)h(q). (66)



The incentive function II

Theorem 6In a generic two strategy m-player matrix game

1 pure strategy S1 is an ESS (level 0) if and only if βm−1 > 0,2 pure strategy S2 is an ESS (level 0) if and only if β0 < 0,3 an internal strategy p = (p, 1− p) is an ESS, if and only if

a) h(p) = 0, and

b) h′(p) < 0.

We will not prove this result here.



The incentive function III

0 1 0 1 0 1

Figure: The incentive function and ESSs in multi-player games. The solid dots showthe equilibrium points and the arrows show the direction of evolution under thereplicator dynamics.



The number of ESSs for the two strategy case

Thus, the possible sets of ESSs are one of the following:1 0 pure ESSs, and l internal ESSs with l ≤ bm

2 c;2 1 pure ESS, and l internal ESSs with l ≤ bm

2 − 1c;3 2 pure ESSs, and l internal ESSs with l ≤ bm

2 − 2c.There can be more than one ESS with the same support in a 4-player game asshown in the next Example.



An example

Now consider an example with the following payoffs:(α13 00 α22

) (0 α11α22 0

)(

0 α11α22 0

) (α11 00 α20

) (67)

with α11 = α22 = −1396 , α13 = α20 = − 3

32 . Thusβ0 = 3/32, β1 = −13/96, β2 = 13/96, β3 = −3/32 giving

h(p) = − 332

p3 +1332

p2(1− p)− 1332

p(1− p)2 +332

(1− p)3 (68)

= −(

p− 14

)(p− 1

2

)(p− 3

4

), (69)

and thus the game has two internal ESSs at p = (1/4, 3/4) andp = (3/4, 1/4) (and no pure ESSs).Mark Broom (City University London) Campione d’Italia, Sep 4-9 72 / 111


Dynamics of multi-player matrix games I

Consider the replicator dynamics for super-symmetric 3-strategy, 3-playergames. The mean payoff over the population is given by

W =

3∑i=1

3∑j=1

3∑k=1

aijkpipjpk. (70)

The payoff to an individual playing pure strategy Si in such a population is

3∑j=1

3∑k=1

aijkpjpk =13∂W∂pi

, (71)

The continuous replicator equation is given by

dpi

dt= pi

(13∂W∂pi−W

)1 ≤ i ≤ 3. (72)



Dynamics of multi-player matrix games II

There are some results that occur for the dynamics on the three playersuper-symmetric games that cannot occur for two players.Two player super-symmetric games are important because they represent thegenetic case where strategies represent alleles and a game a mating, where itcan be reasonably assumed that the payoff to both players is the same.If new strategies are allowed to enter a population sequentially and thepopulation is allowed to converge to a new ESS, any new strategy that caninvade the current ESS must subsequently feature in the support of the newESS.This is not true in the multi-player case, e.g. with payoffsa111 = 0, a222 = 1.5, a333 = 0, a112 = 0.6, a113 = 0.05, a221 = 0,a223 = −1, a331 = 0, a332 = 1/2 and a123 = 0.6 which is shown in thefollowing figure.



Dynamics of multi-player matrix games III

Figure: This figure shows fitness contours. Even though a new strategy may be ableto invade what was an ESS, it need not be represented in the final ESS. Specifically,strategy 3 can invade (1, 2) but the outcome is (2).



Dynamics of multi-player matrix games IV

In the two player case any ESS can be reached by an appropriately orderedsequential introduction of strategies.This is again not true for the multi-player case, using the example game wherethe game with payoffsa112 = a113 = a221 = a223 = a331 = a332 = −1, a123 = 1yields an unreachable ESS. This concept is not often discussed in static gamesor in replicator dynamics, where no new strategies are introduced, but is ofparticular interest in adaptive dynamics.



Dynamics of multi-player matrix games V

In this example we have ESSs with supports (1), (2), (3) and (1, 2, 3). Withsequential introduction of strategies, the ESS with support (1, 2, 3) can never

be reached.


Games in finite and structured populations

Outline





Games in finite and structured populations Finite populations and the Moran process

The Moran process

Here we consider a population of finite size N.We shall start by assuming individuals have a fixed fitness ri for individual Si,depending upon type, but independent of interactions with others.The standard dynamics applied to this population is as follows.At each time step an individual is chosen for reproduction at random with aprobability proportional to its fitnessIts offspring replaces a randomly chosen individual (which could be itsparent).This is called the Moran process.



Neutral fitness

We shall start by considering the neutral fitness case, where ri = 1 for all Si,as in the original Moran process.Suppose we have N individuals, made up of mi individuals of type i,i = 1, . . . , n.The population is thus described by a (row) vector m = (mi) with

∑i mi = N.

At each time point a random individual is chosen to give birth, and another todie, selected independently of each other.Let ei be a (row) vector with 0’s everywhere except on the ith place wherethere is a 1.



A Markov process

The possible transitions in the population, together with the probabilities ofthose transitions, are as follows:

P(m→ m∗) =

mi

Nmj

Nm∗ = m + ei − ej, i 6= j,

n∑i=1

(mi

N

)2m∗ = m,

0 otherwise.

(73)

Thus at any time t, m(t) only depends upon m(t − 1) and no earlier timepoints are relevant; thus the Moran process is a Markov process.



Non-neutral evolution

Now suppose that not all of the values of ri are equal.We note that there are many ways we can incorporate this in the process.As described above, we shall make the birth rate depend upon fitness and thedeath rate not.One way of thinking of the fitness of an individual is as the number ofoffspring that it will have that will survive to adulthood.We can thus perhaps think that at any given time step, and for every type Si

there are miri offspring of that type that may be born.An offspring of type Si will thus be born with probability miri/ (

∑l mlrl).



The revised transition probabilities

The transition probabilities now become

P(m→ m∗) =

miri∑nl=1 mlrl

mj

Nm∗ = m + ei − ej, i 6= j.

n∑i=1

miri∑nl=1 mlrl

mi

Nm∗ = m.

0 otherwise.

(74)

Multiplying all ri by a constant leaves the probabilities unchanged, so withoutloss of generality we set one of our fitnesses to be 1.There is a non-zero probability that any given type will reach fixation, sinceri > 0 for all i, and it is certain that one type will eventually do so.



The fixation probability

The long-term outcome of the process described above is a populationconsisting of just a single type.The important question is, which type is likely to dominate, i.e. how likely iseach such fixation to occur?Thus the probability of fixation, the fixation probability, is the single mostimportant property of a finite evolutionary system.This is usually considered as the probability of fixation of a single mutant in apopulation otherwise entirely made up of a resident type.It thus makes sense for us to consider the case where we have two types ofindividuals only, type A with fitness r, and type B with fitness 1.



The Chapman-Kolmogorov equations

The state of the population is described by a single number, NA, the number ofindividuals of type A.We can find an expression for the probability of the population containing imutants at time t + 1, πi(t + 1), in terms of the probabilities of occupying thedifferent population sizes at time t and these transition probabilities using theequation

πi(t + 1) =∑

j

pj,iπj(t), (75)

where pi,j = Prob((i,N − i)→ (j,N − j)

)is the probability that NA = j at

time point t + 1 given that NA = i at time point t.



Transition probabilities for two types of individual

By (74), we get

pi,i+1 =ir

ir + N − iN − i

N, (76)

pi,i−1 =N − i

ir + N − iiN, (77)

pi,i =ir

ir + N − iiN

+N − i

ir + N − iN − i

N(78)

= 1− irir + N − i

N − iN− N − i

ir + N − iiN. (79)

In the terminology of Markov processes, (75) are the Chapman-Kolmogorovforward equations.



The fixation probability equation

Denoting Pi as the fixation probability of A given NA = i, we obtain thefollowing difference equation

Pi = Pi−1pi,i−1 + Pipi,i + Pi+1pi,i+1 (80)

with the obvious boundary conditions, the fixation probabilities on theabsorbing states, P0 = 0,PN = 1.We can solve the above equations to obtain

Pi =

1− (1/r)i

1− (1/r)N r 6= 1,

i/N r = 1.(81)



The Moran probability

This in turn gives the fixation probability of a single mutant of type A in apopulation of type Bs as

PA = P1 =

1− (1/r)

1− (1/r)N r 6= 1,

1/N r = 1.(82)

This is the Moran probability, plotted on the next slide.This is the benchmark against which fixation probabilities in more complexstructured populations are compared.By symmetry, the fixation probability of a B mutant in an A population ifr 6= 1 is

PB =r − 1

rN − 1. (83)



A plot of the Moran probability

0.001 0.01 0.1 1 10 100 10000

0.25

0.5

0.75

1

r

Fixa

tion

prob

abili

ty

N=2N=4N=16

Figure: The Moran fixation probability for various N.


Games in finite and structured populations Games in finite populations

The payoff matrix

The Moran process approach above has been extended to playing games infinite populations.For two types of individuals, mutants M and residents R, we consider thestandard 2× 2 payoff matrix

(M RM a bR c d

). (84)



Mean payoffs

The average payoffs to a mutant individual in a population where there aremM = i mutants in total is thus

EM,i =a(i− 1) + b(N − i)

N − 1, (85)

Similarly the average payoff to a resident individual in such a population is

ER,i =ci + d(N − i− 1)

N − 1. (86)

Note that we assume above that an individual cannot play a game with itselfand hence we have the factors (i− 1) and (N − i− 1).



ESSs in finite populations I

If a resident strategy is an ESS in an infinite population, it is fitter than anymutant in a population comprising a mixture of a sufficiently small mutantgroup and the remainder playing the resident strategy.A natural extension to finite populations is that a mutant should be less fit thana resident in a population of one mutant and N − 1 residents (selectionopposes M invading R) i.e. EM,1 < ER,1 which is

b(N − 1) < c + d(N − 2). (87)

However, this is insufficient for stability, since a chance increase in the mutantpopulation can lead to greater mutant fitness. Thus we have a secondcondition, that selection opposes the replacement of R by M,

PM <1N. (88)



ESSs in finite populations II

We thus have the definition:For a finite population size N and a 2× 2 matrix game (84), a pure strategy Ris called an evolutionarily stable strategy, ESSN , if (87) and (88) hold.Here PM is given by

PM =1

1 +∑N−1

j=1∏j

k=1pk,k−1pk,k+1

, (89)

where pk,k+1 and pk,k−1 are the transition probabilities equivalent to (76) and(77).



The intensity of selection

Since the formulae (76) and (77) use fitness, we need to find a way to translatethe payoffs of the game EM,i and ER,i into the fitness of the respective typesrM,i and rR,i.If we assume that the fitness is equal to the payoff, i.e. rM,i = EM,i andrR,i = ER,i, the stability condition (88) is very complex.Thus the idea of intensity of selection is often used.Assume that the fitness is given by

rM,i = 1− w + wEM,i, (90)

rR,i = 1− w + wER,i, (91)

where 0 < w ≤ 1 is the intensity of selection.



Weak selection

A small w represents weak selection and means that the game has a smalleffect on the process of evolution. This gives the transition probabilities (76)and (77) as

pi,i+1 =i(1− w + wEM,i)

i(1− w + wEM,i) + (N − i)(1− w + wER,i)

N − iN

, (92)

pi,i−1 =(N − i)(1− w + wER,i)

i(1− w + wEM,i) + (N − i)(1− w + wER,i)

iN. (93)

For small w ≈ 0, pi,i−1/pi,i+1 ≈ 1 + w(ER,i − EM,i), and after substitutinginto (89) we get that the stability condition (88) is equivalent to

a(N − 2) + b(2N − 1) < c(N + 1) + d(2N − 4). (94)



The rule of 1/3

For a large population the conditions for ESSN (87) and (94) reduce to

b < d, and (95)

a + 2b < c + 2d. (96)

The first of these two conditions is the standard condition for an ESS in aninfinite population.The addition of the second condition leads to the rule of 1/3. The rule saysthat if a > c and b < d (so in an infinite population there are two pure ESSs)selection favours M replacing R, if the unstable internal equilibrium value isless than a third i.e.

d − ba− c + d − b

<13. (97)


Games in finite and structured populations Evolution and evolutionary games on graphs

Representing population structure by a graph

In this part of the talk, we assume that a population consists of N individualsand that each individual occupies a vertex in a given graph G = (V,E).G is thus a finite and undirected graph, which we assume is connected andsimple, i.e. no vertex is connected to itself and there are no parallel edges.We moreover assume that every vertex is occupied, and by one individualonly.Two individuals can interact only if they are connected by an edge of thegraph.



The graph weights

The graph structure is represented by a matrix W = (wij), wherewij is the probability of replacing a vertex j by a copy of a vertex i, given thatvertex i was selected for reproduction.wij = 0 if there is no edge between vertices i and j.For connected vertices, we often assume equal weightings, i.e. wij = 1/ei

where ei is the degree of vertex i, see the figure on the next slide.The well-mixed population that we have considered up until now is a specialcase of this. It is represented by the complete graph, the graph where everypair of vertices are connected, with all weights equal.We note that this treatment with weights wij is sufficiently general to allow usto consider directed graphs or graphs where edges carry a different weight.



Graphs with four vertices

(a) (b) (c) (d) (e) (f)

1 2

3

4

1 2 3 41 0 1

212 0

2 12 0 1

2 0

3 13

13 0 1

3

4 0 0 1 0

Figure: Connected undirected graphs with 4 vertices. For one of these, the graph andits corresponding weighting matrix W, in the case of equal weights.



Evolutionary dynamics on the graph

We suppose that the population evolves according to an evolutionarydynamics and the evolutionary process can be represented as a discrete timeMarkov chain.Supposing that C ⊆ V is the set of vertices occupied by mutants, then at thenext time step the set occupied by mutants will become either

[1)] C ∪ {j}, j 6∈ C, provided a) a vertex i ∈ C was chosen for reproductionand b) it placed its offspring into vertex j; or [2)] C \ {i}, i ∈ C, provideda) a vertex j 6∈ C was selected for reproduction and b) it placed itsoffspring into i; or [3)] C, provided an individual from C (V \ C) replacesanother individual from C (V \ C).

The states ∅ and V are the absorbing points of the dynamics.



The Invasion Process

It is generally assumed that at the beginning of the evolutionary process, allvertices are occupied by residents and then one vertex is chosen uniformly atrandom and replaced by a mutant.We have outlined the possible transitions in the Markov chain, but not theprobabilities.Whilst the possible transitions are generally the same for any of theevolutionary dynamics commonly used, the transition probabilities are notand depend upon a choice of evolutionary dynamics.We shall initially assume the Invasion Process (IP) where an individual isselected to give birth proportional to its fitness, and then copies itself into oneof its neighbours (usually at random, with equal probability).An example of one step of the IP dynamics is shown in the following figure.



An illustration of the Invasion Process

r

11

1

Initial population Selection for reproduction

11

1

11

1

11

1

r3+r

13+r

13+r

11

1

13+r

Neighbour replacement

13

13

13

12

12

12

12

11

r

r

r

r

Figure: One step of an Invasion Process.



The fixed fitness case

The individual at vertex i is selected for reproduction with probabilityproportional to the corresponding fitness 1 or r.The invasion process yields

PC =

∑i∈C

∑j 6∈C

(rwijPC∪{j} + wjiPC\{i}

)∑i∈C

∑j 6∈C

(rwij + wji

) (98)

with P∅ = 0 and PV = 1, where PC denotes the probability of mutant fixationgiven C is the set currently occupied by mutants.Note that the system (98) of linear equations is very large.



Games on graphs

When considering games on graphs the fitness of each individual dependsupon the types of all of its neighbours.Using the standard payoff matrix, the payoffs to an M individual at vertex iand an R individual at vertex j are given by

fi =aNM,i + bNR,i

NM,i + NR,i, (99)

fj =cNM,j + dNR,j

NM,j + NR,j, (100)

where NM,i (NR,i) is the number of neighbours of i of type M (R).Here we consider the example of a classical game, the Hawk-Dove game.



An example Hawk-Dove game I

Theoretical formulae for the exact solutions of fixation probabilities,absorption times and fixation times for the star and the circle have beengenerated.In the following example the payoffs matrix (84) of the Hawk-Dove gamebecomes ( Hawk Dove

Hawk a = (15− C)/2 b = 10Dove c = 5 d = 15/2

), (101)

which is equivalent to a reward V = 5, an arbitrary cost C, plus a“background fitness” of 5.



Fixation on three graphs

Figure: The Fixation probability (left figure) and the mean time to absorption (rightfigure) for a mutant Hawk in a population of Doves for a star (crosses), a circle(circles) and a complete graph (boxes) when N = 100 and C varies.



An example Hawk-Dove game II

Changing C has a gradual effect on the fixation probability on the circle, asudden and dramatic effect on the fixation probability on the complete graphand almost no effect on the fixation probability on the star.Comparing the complete graph and the circle, it is clear that the fixationprobability can be significantly different for different regular graphs (asopposed to the fixed fitness case, where this is the Moran probability).The time to absorption is hardly affected by the value of C on the circle or star(except as C approaches 15 and so the payoff of a mutant against anothermutant approaches 0).The larger times for intermediate C on the complete graph corresponds towhen the equivalent infinite population has an ESS corresponding of roughlyequal numbers of Hawks and Doves.



Dynamics and fitness I

Evolutionary game models consider a number of dynamics, not just theInvasion Process, the most common being the four related ones below.IP dynamics or BD-B - an individual is chosen for reproduction withprobability proportional to its fitness and its offspring replaces a randomlychosen neighbour. BD-D process - an individual is chosen for reproduction atrandom and its offspring replaces a neighbouring individual which is chosenwith probability inversely proportional to its fitness. Voter model or DB-D -an individual first dies with a probability inversely proportional to its fitnessand is then replaced by the offspring of a randomly chosen neighbour. DB-Bprocess - an individual first dies at random and is then replaced by anoffspring of a neighbour that is chosen with probability proportional to itsfitness.



Dynamics and fitness II

In general, the dynamics are similar for regular graphs. In particular thefixation probability for the IP and the VM are identical.The fixation probabilities for the BD-D and DB-B processes are different forsmall population sizes, these differences disappear for sufficiently largepopulations.However, the choice of a dynamics can be very important for irregular graphsas seen in the next slide, which considers evolution on the star for theHawk-Dove game.



Dynamics and fitness III

In all of the processes except the IP, Hawks have lower fixation probabilitieson the star than on the complete graph for low costs, whereas the reverse istrue for the IP.The differences between the fixation probabilities for the different processesare very large for medium to large values of the population size n + 1.Thus the dynamics used can have a profound effect on the fixation probability,and indeed it can have similar effects on other properties such as the fixationtime.It should be noted that a similar figure can be obtained for the fixed fitnesscase as well.



Fixtation probabilities for different dynamics

[a]

Dyn Games Appl (2011) 1:386–407 401

Fig. 5 The average fixation probability of a single mutant Hawk on a star graph under the IP (crosses), theBD-D process (diamonds), the VM (circles) and the DB-B process (boxes) in the Hawk–Dove game describedby the payoff matrix (4.15) in the case where (a) C/V = 1.5 and n varies, (b) n = 60 and C/V varies. fb = 2and w = 1. The thick lines represent the respective case on the complete graph and the dashed-dotted linerepresents the fixation probability of a single mutant in the case of neutral drift, 1/(n + 1)

Figure 5b suggests that when Hawks are favoured over Doves in the different updaterules, the complete graph promotes the fixation of Hawks compared to the star graph inthe BD-D, VM and DB-B process. Moreover, in the IP, favoured Hawks have much higherchance to fixate on a star graph.

Note that in the case of weak selection, in large stars and complete graphs, Hawks arefavoured over Doves if the simple condition C/V < 2 holds, in all update rules.

In the case where a mutant Dove invades into Hawks, all the above results can be easilyobtained by interchanging the two strategies, i.e. by exchanging α and δ, and β and γ .

Mean Time to Absorption and Fixation Starting from a Single Mutant Hawk A comparisonof the absorption times for varying population size and varying C/V for the game with pay-off matrix (4.15) is shown in Fig. 6. The absorption times on the complete graph as the ratioC/V varies is also shown in Fig. 6b for comparison. On the complete graph, the time neededfor mutants to either fixate or die out is almost unaffected by the update rule followed. Inlarge populations, values of the payoffs such that ρCG ≈ 1 lead to the highest times beforeabsorption and fixation occur, in all the update rules. However, on the star, as in the fixedfitness case, we observe that the speed to absorption and fixation might significantly varywhen following different update rules. There is again a quantitative and qualitative distinc-tion between birth–death and death–birth processes. In most of the cases the birth–deathprocesses yield much higher absorption and fixation times than the death–birth processes.In large populations, both the absorption and fixation times in the two birth–death processesachieve local maxima for parameter values such that ρIP ≈ 1 and ρBD-D ≈ 1 since thenthe two strategies coexist for a long time before absorption/fixation occurs. In the VM andDB-B process, although the absorption and fixation times increase as C/V increases, theyare affected less by the variation of C/V . In our example, we can see that for the VM asC/V → 5 (i.e. the fitness of a Hawk individual when playing with just another Hawk tendsto 0+), H T VM (and similarly H F VM) sharply increases. An initial Hawk on a leaf can beeliminated by chance, but if it is not, eventually it will occupy the center. At that moment,a Hawk on the leaves has a very very small fitness, so it will be eliminated and replaced by

[b]

Figure: Average fixation probability of a Hawk on a star graph; IP (crosses), BD-D(diamonds), VM (circles) and DB-B (boxes) with (a) V = C = 1 and n varies, (b)n = 100,V = 1 and C varies. Background fitness is 2. The thick lines represent thecomplete graph and the dashed-dotted line neutral drift.


mark broom - game theory and dynamic games€¦ · for an evolutionary game we also need a...

Documents