effect of information on collusion strategies in single winner, multi-agent games

Effect of Information on Collusion Strategies in

Single winner, multi-agent games

December 2, 2010  

Nick GramskyKen Knudsen

Contents

1. Motivation

2. Identification of Collusion

3. Classification of Coalitions

4. Implementation

5. Results

6. Conclusions

Motivation

Explicit Collusions Alliances Survival Truces

Implicit Collusions Minimax against strongest

player Tit-for-tat

Reasons to Collude Improve position relative to other agent(s) Self-preservation / Survival

Contents

1. Motivation

2. Identification of Collusion

3. Classification of Coalitions

4. Implementation   5. Results

6. Conclusions

Identification

Find course grained collusive behavior

1. Offensive-based collusion Multiple agents attacking a single agent for a fixed

number of rounds In our examples, we limited this to 1 round.

2. Defensive-based collusion Multiple agents not attacking each other over a fixed

number of rounds. In our examples, we limited this to 2 rounds.

IdentificationOffensive based coalitions

IdentificationDefensive based coalitions

Contents

1. Motivation

2. Identification of Collusion

3. Classification of Coalitions

4. Implementation

5. Results

6. Conclusions

1. Socially inclined behavior For some predefined time, if target satisfies the

following, then we define the actions of the attacking players as being 'socially oriented‘

h(x) is a heuristic function for any adversary. vh(x) when dealing with different layers of fog

2. Else: Some other collusive behavior

Classification Offensive based behaviors

Classification Offensive based algorithm

Classification Defensive based algorithm

ClassificationMissed opportunities 

Classify a missed opportunity by finding players that: for a predefined period were not attacked

above a certain percentage and… satisfy either their power heuristic or visual

heuristic (below) threshold

Contents

1. Motivation

2. Identification of Collusion

3. Classification of Coalitions

4. Implementation

5. Results

6. Conclusions

Implementation

Used Warfish to play games of Risk. Free website warfish.net

  Risk is a zero-sum game where players seek (simulated) world

domination! 

Only one winner, the last remaining contestant.

Attacks are made via dice (random number generator)

Amass armies, grow in power, rule the world! Or at least the world represented on a board...

ImplementationEnvironment

Reduced resource strategies

Randomized players

Set card trade-in values to be constant (5)

Disabled card capture on elimination

Multiple map types Larger than original Risk board Reduces board specific strategies in analysis

ImplementationWorld Map

ImplementationEurope Map

ImplementationFog of War

Varied amount of information available to all agents via different levels of 'fog of war'.

6 different levels of fog available in game Level 0: No fog (perfect information) Level 1: See all occupations, neighboring units only Level 2: See all occupations (no units) Level 3: Only see neighboring occupations and units Level 4: See only neighboring occupations Level 5: Complete fog (only know about self)

Tested with 3 levels of fog {0,1,3}

ImplementationOracles

Participants who annotated their strategies and behaviors as games were played

Compared oracle annotations to game data Spot-check that analysis found collusion Though noisy, analysis and annotations were

inline with game history.

Contents

1. Motivation

2. Identification of Collusion

3. Classification of Coalitions

4. Implementation

5. Results

6. Conclusions

ResultsCollusion vs Game length

x-axis: Number of turnsy-axis: Number of "interesting" windowsθh = 1.3 per 1 turn window

ResultsOffensive

1. Players all gang up on Yellow.

2. Validated by Oracle annotations.

Game: 98478150 Map: World Fog Level: 1

ResultsOffensive

1. Minmax against Blue

2. Confirmed by reading through the transcript.1. Blue quickly gained

power

2. Challenged remaining players to team up against him Game: 97976903

Map: Europe Fog Level: 0

“Right now (Yellow) knows that if he does not get both you (Red) and (Green) on his side, this game will be won by me”

ResultsOffensive

x-axis: Number of turnsy-axis: Number of "interesting" windowsθh = 1.3 / 1 turn window

Games 98478150 (left) and 97976903 (right)

ResultsOffensive & Defensive

1. Minimax against strongest player

2. Towards the end of the game, explicit truce between top 2 players

Game: 12069561 Map: Europe Fog Level: 0

Scatter plot of number of windows classified as defensive-oriented for all games.x-axis: number of turns y-axis: number of interesting windowsθ = 0.05

*Game: 12069561

ResultsDefensive

ResultsOracle

1. Oracle self-interest annotations (Blue)

Game: 88318444 Map: World Fog Level: 1

x-axis: Number of turnsy-axis: Number of "interesting" windowsθh = 1.3 / 1 turn window

ResultsFog Level 3

1. Typical of the layer 3 games.

2. Everything breaks down. Players can’t figure out who is in the lead until it is too late.

Game: 67785982 Map: Europe Fog Level: 3

Results

Collusion % is percentage of available windows where remaining players direct more than 75% of attacks towards target.

Social % is percentage of available windows with same criteria as above BUT the target satisfies heuristic thresholds from earlier

θh = 1.3 / 1 turn window

Target’s residual power 43.3% (4-player) 65% (3 player)

θh = 1.6 / 1 turn window

Target’s residual power 53.3% (4-player) 80% (3 player)

ResultsEurope Map

θh = 1.3 θh = 1.6

ResultsWorld Map

θh = 1.3 θh = 1.6

Contents

1. Motivation

2. Identification of Collusion

3. Classification of Coalitions

4. Implementation

5. Results

6. Conclusions

Conclusions

Presented a basic algorithm to identify and classify collusion

Games with unusually large number of collusive behaviors tended to prolong games beyond the average.

As fog increased (information decreased), collusive behaviors diminished.

Results were consistent across maps.

Level 0 data was consistent between our volunteers and the public.

Analysis supported by Oracle annotations and in-game conversations.

Conclusions

Visual heuristic does not hold well for fog games Based on a knowledge of territories and bonuses

Limited data sets Time limitation

Short time-frame for project Games averaged 20 days to complete

Require more experiments with fog levels

Data integrity Games had large variance in player abilities Players were involved in multiple simultaneous games

May have forgotten strategy Players may have a predefined disposition towards other

players (Social Value Orientation)

ConclusionsFuture Work

Investigate possible equilibrium in collusions versus game length.

  Lag response for social orientation.

Once the strongest player is removed from power, it can take a few rounds for the coalition to change strategies.

As information decreases, agents tend to collude less.  Why? fairness poor assessment of board

Mix socially oriented bots with human players