Evolving Cooperation in the N-player Prisoner's

Dilemma: A Social Network Model

Dept Computer Science and Software Engineering

Golriz Rezaei Michael Kirley

Jens Pfau

ACAL09 Conference – practice talk Oct 2009

Motivation

The Dilemma: - Contribution to the social community beneficial for everybody - Autonomous self-interested individuals rational, maximize their utility

“Tragedy of the Commons” [Hardin 1968 science] - Theoretical biology / Game theory they should “Defect” - Nature / Reality they “Cooperate”

Important Question in many areas :

How/Why does cooperation emerge? What about Artificial Multi-agent systems?

Frame work: N-player Dilemma games on social groups.

Distributed Artificial Intelligence (DAI) Physics (Statistical Physics) Biology (Theoretical biology, Nature) Evolutionary Computation (IEEE Trans, CEC) Multi agent systems (AAMAS)

Overview

What is a social network Brief overview of Prisoner’s Dilemma

(N-player PD game) PD on network Proposed model Evaluation by experiments Conclusion Questions

Complex Networks every where

Social Networks

Networks

Topology Function

Social ties Behaviour

Network Basics

Network graph, G(N, E), N finite set of nodes (vertices) E finite set of edges (links) G represented by N×N adjacency matrix

aij = 1 there is an edge between node i and j

aij = 0 otherwise

Examples of Social Net

Internet-Map

Red, blue, or green: departments Yellow: consultants

Grey: external expertswww.orgnet.com

Structure of an organization

Topological properties

Degree, ki , of a node

Path length, L average separation between any two nodes

Clustering coefficient, Ci , of a node

probability that two nearest neighbours of a node are also nearest neighbours of each other.

N

jjiE

0

Prisoner’s Dilemma (2 players)

(D,D) Nash Equilibrium

Ccooperate

DDefect

Ccooperate

b-c -c

DDefect

b 0

• 2 players / agents• 2 choices (C or D)• Payoff joint actions

N-Player Prisoner’s Dilemma

Natural extension

Utility [Boyd and Richerson 1988 J. Th. Biology]

Conditions defection is preferred for individuals

contribution to social welfare is beneficial for the group

Conventional EG (D,D, … all D)

0 cb

Nbc /

PD on Spatial structure

Local neighbourhood interaction

Clusters of cooperators

Enhance cooperation

Related work

• Santos Et. Al. [2009 Nature]Heterogeneous graphs (number and size of the game)

Promotes cooperation

• Ohtsuki Et. Al. [2006 Nature] Correlation cost and benefit & the underlying connectivity of agents

• Ellis & Yao [2007 IEEE CEC] Reputation mechanism reputation scores embedded in social

network

Contribution - Hypothesis

Introducing more cognitive agents (base their decision on some function of the opponents)

Incorporating “social network” into N-player PD (network evolves by cooperative behaviour)

Encourage high levels of cooperation Persist for longer Analyse the state of underlying network

Proposed model

Algorithm: Social network based N-PD modelRequire: Population of agents P, iteration = imax, players N 2

1: for i = 0 to imax do2: G = 0;3: while g = NextGame(P,G, N) do4: G = G {g}5: PlayGame(g)6: AdaptLinks(g)7: end while8: a,b = Random Sample(P)9: CompareUtilityAndSelect(a,b)10: end for

Decision

How

Game Execution

Two scenarios (cognitive abilities)Pure strategy (always cooperate/defect)

Mixed strategy (play probabilistically)

Based on a function of average links weight ( )(β generosity)(α gradient of the function)

– Agents receive corresponding payoff based on outcomes (Boyd and Richerson function)

Decision

)(gWi

Link adaptation

Agents play cooperatively form social links (reinforced)

One agent defects breaks his links with the opponents

How

slow positive / fast negative

Snapshots of the model

Self-organize social ties based on their self-interest

Strategy update cultural evolution

(a) Iteration 5 (b) Iteration 100 (c) Iteration 1000

Experimental Setup

population size = 1000 ε = 0.9 (game formation) b = 5 and c = 3 (payoff values benefit & cost) pure strategy scenario (50% pure C – 50% pure D) mixed strategy scenario (33.3% each) α = 1.5 and β = 0.1 (decision function) average20 independent trials up to 40000 iterations

What is the equilibrium state and network topology?

Experiment 1 Group size vs. Strategy

2

4

5

10 15 20

2

4

5

10

15

20

Pure strategy Mixed strategy

Rat

io o

f Coo

pera

tion

Rat

io o

f Coo

pera

tion

Time (iteration) Time (iteration)

Experiment 2 Emergent social networks

Pure strategy Mixed strategy

2

4

510 15 20

24

5

10

15

20

Clu

ste

r C

oe

ffici

en

t

Clu

ste

r C

oe

ffici

en

t

Time (iteration) Time (iteration)

Experiment 3 Final Degree Distribution

N = 2 N = 10

log(k) log(k)

log

(P(k

))

log

(P(k

))

Conclusion

– Results validate the hypothesis

Incorporating “social network” into N-player PD

encourage high levels of cooperation and persist for longer– Social nets important in promoting and sustaining

cooperation (specially with cognitive agent)– Endogenous network formation – Analysis of the emergent social networks

high average clustering broad-scale heterogeneity

– Local structure hierarchical organization of cooperation

Questions?

Thank you