Evolution & Learning in GamesEcon 243B

Jean-Paul Carvalho

Lecture 7:Evolutionary Stability

1 / 21

Evolutionary Stable States (ESS)Maynard Smith and Price (1973) defined the notion of an evolutionarystable strategy as immune to invasion by mutants:

I Their focus was on monomorphic populations: every memberplays the same strategy, which can be a mixed strategy.

I We are concerned with a polymorphic population of agents eachprogrammed with a pure strategy.

I We have seen the equivalence of these two problems.

Hence we can adapt the concept of an evolutionary stable strategy toa population setting:

I The term we shall use is evolutionary stable state (ESS).

2 / 21

Invasion

Let the state be x =

x1x2. . .xn

.

Consider a game F, where F(x) =

F1(x)F2(x)

. . .Fn(x)

.

Consider an invasion of mutants who make up a fraction ε ofthe post-entry population.

The shares of each strategy in the mutant population are

represented by y =

y1y2. . .yn

.

3 / 21

Invasion

Therefore, the post-entry population state is:

xε = (1− ε)x + εy =

(1− ε)x1 + εy1(1− ε)x2 + εy2

. . .(1− ε)xn + εyn

.

The average payoff in the incumbent population in thepost-entry state is x′F

((1− ε)x + εy

).

The average payoff in the mutant population in the post-entrystate is y′F

((1− ε)x + εy

).

4 / 21

Uniform Invasion Barrier

The average payoff in the incumbent population is higher if:

(y− x)′F((1− ε)x + εy

)< 0. (1)

State x is said to admit a uniform invasion barrier if thereexists an ε > 0 such that (1) holds for all y ∈ X− {x} andε ∈ (0, ε).

That is, for all possible mutations y, as long as the mutantpopulation is less than fraction ε of the postentry population,the incumbent population receives a higher average payoff.

5 / 21

ESS

DEFINITION. State x ∈ X is an evolutionary stable state (ESS)of F if there exists a neighborhood O of x such that:

(y− x)′F(y) < 0 for all y ∈ O− {x}. (2)

In other words, if x is an ESS, then for any state y sufficientlyclose to x, a population playing x will receive a larger averagepayoff in state y than a population playing y (i.e. x is a betterreply to y than y is to itself).

Note that this considers invasions of other states y by x ratherthan invasions of x by other states. Hence it is not clear, atpresent, why this should be a stability condition.

6 / 21

ESS and Invasion Barriers

Theorem 7.1. State x ∈ X is an evolutionary stable state (ESS)if and only if it admits a uniform invasion barrier.

Thus if x is stable in the face of an arbitrarily large populationof entrants who mutate to a nearby state, then it is stable in theface of a sufficiently small population of entrants who mutateto an arbitrary state.

7 / 21

ESS and NE

What is the relationship between ESS and NE?

DEFINITION. Suppose that x ∈ X is a NE. Then (y− x)′F(x) ≤ 0for all y ∈ X.

In addition, suppose there exists a neighborhood of x that doesnot contain any other NE.

Then x is an isolated NE.

Proposition 7.2. Every ESS is an isolated NE.

8 / 21

Proof

Let x be an ESS of F, O be the nhd posited in (2) andy ∈ X− {x} (not necessarily in O).

Then for all ε > 0 sufficiently small, the postentry statexε = εy + (1− ε)x is in O.

Given x is an ESS, this implies that:

(xε − x)′F(xε) < 0(εy + (1− ε)x− x)′F(xε) < 0

ε(y− x)′F(xε) < 0(y− x)′F(xε) < 0.

(3)

9 / 21

Proof

Taking ε→ 0 yields:

(y− x)′F(x) ≤ 0,

by the continuity of F. That is, x is a NE.

To establish that x is isolated, note that if w ∈ O− {x} were aNE then (w− x)′F(w) ≥ 0, contradicting the supposition that xis an ESS [by (2)]. �

The converse of Proposition 7.2 is not true.I The mixed equilibrium of a two-strategy coordination

game is a counterexample.

10 / 21

More on ESS and Nash

Therefore, ESS is stronger than NE.

In particular, an ESS satisfies the additional property:

Suppose there exists a state y which is an alternative best replyto x, i.e. (y− x)′F(x) = 0.—Then (y− x)′F(y) < 0, i.e. x is a better reply to y than y is toitself.

Therefore:

I A strict NE is an ESS.

I A polymorphic population state (equivalent to a mixedNE) cannot be strict and hence must satisfy the additionalproperty.

11 / 21

More on ESS and Nash

In the case in which agents are matched uniformly at random toplay a normal form game (the case we have been focussing on),then it is easy to see why the additional property is required.

Suppose (y− x)′F(x) = 0, i.e. y is an alternative best reply to x.

Then:

(y− x)′F(εy + (1− ε)x) = ε(y− x)′F(y) + (1− ε) (y− x)′F(x)︸ ︷︷ ︸=0

= ε(y− x)′F(y).(4)

Therefore, (y− x)′F(y) must be negative for (1) to hold andhence, by Theorem 7.1, for x to be an ESS.

12 / 21

Example: Hawk Dove

Hawk DoveHawk −2

−20

4Dove 4

00

0

ESS: x = ( 23 ,

13 ). ESS payoff = 0.

13 / 21

Example: Hawk Dove

I Consider a mutation y such that y1 > x1 = 23 .

I Check that (y− x)′F((1− ε)x + εy

)< 0 for all such y:

(y1− x1)[−2((1− ε)x1 + εy1

)+ 4((1− ε)(1− x1)+ ε(1− y1)

)].

I This equals:

(y1 − x1)ε[−2y1 + 4(1− y1)]

because −2× 23 + 4× (1− 2

3 ) = 0. This in turn equals:

(y1 − x1)ε[4− 6y1]

which is negative because y1 > 23 by hypothesis.

I A similar argument can be applied to the case y1 < x1.Hence x is an ESS.

14 / 21

The Prisoners’ Dilemma

C DC 3

35

0D 0

51

1

NE/ESS: x = (0, 1).

Therefore, an ESS is not necessarily efficient.

15 / 21

Not Every Game has an ESS

A B CA 1

10

22

0B 2

01

10

2C 0

22

01

1

x = ( 13 ,

13 ,

13 ) is the unique NE and therefore the only possible

ESS.

16 / 21

Not Every Game has an ESS

Note that x is a polymorphic population state (equivalent to amixed strategy), so any basis vector (pure strategy) is analternative best reply to x.

Check that the additional property holds: (e1 − x)′F(e1) < 0,where e1 = (1, 0, 0), i.e. the pure-strategy A.

This is not the case: x′F(e1) = e′1F(e1) = 1.

17 / 21

The Iterated Prisoners’ Dilemma

I Two players engage in a series of PD games.

I The engagement ends after the current round withprobability δ < 1

2 . We call this the stopping probability.

I Consider a population in which three strategies arepresent:I C—always cooperate,I D—always defect,I T—tit-for-tat, i.e. start by cooperating, thenceforth

cooperate in period t if partner cooperated in t− 1.

18 / 21

Expected PayoffsWithin each pairing:

C D TC 3

δ 0 3δ

D 5δ

1δ 4 + 1

δ

T 3δ

1δ − 1 3

δ

Over all pairings:

FC(x) = (xC + xT)3δ

FD(x) = xC5δ + xD

1δ + xT

(4 + 1

δ

)FT(x) = (xC + xT)

3δ + xD(

1δ − 1)

19 / 21

All-T is not an ESS

Let x = (xD, xC, xT) = (0, 0, 1).Consider any alternative state y such that yD = 0.

(y− x)′F(y) = (0 yC yT − 1)

FD(y)FC(y)FT(y)

= yCFC(y) + (yT − 1)FT(y)= [yC + (yT − 1)] 3

δ (recall that yD = 0)

= [yC + (1− yC − 1)] 3δ (because yT = 1− yC)

= 0.

This violates (2). Hence all-T is not an ESS.This is a case of evolutionary drift.

20 / 21

Vector Field

C

D

T21 / 21