team assembly and performance in large scale online role ... · team assembly and performance in...

11/27/2014

1

Team Assembly and Performance in Large Scale Online Role Playing Games*

Mengxiao Zhu

(Presented by Noshir Contractor from Northwestern University)

Educational Testing Service

Nov 4, 2014

* This research was supported by grants from the National Science Foundation: DHB Virtual Worlds: An Exploration for Theorizing

and Modeling the Dynamics of Group Behavior (0841583) and Air Force Research Laboratory: The Virtual World Observatory:

Identifying Real World (RW) Characteristics from Virtual Behavior ( FA8650-10-C-7010)

Copyright © 2014 by Educational Testing Service. All rights reserved. Copyright © 2014 by Educational Testing Service. All rights reserved. 2

http://www.nytimes.com/2012/01/08/

jobs/building-the-watson-team-of-

scientists.html

David Ferrucci,

New York Times

1/7/2012

Copyright © 2014 by Educational Testing Service. All rights reserved. Copyright © 2014 by Educational Testing Service. All rights reserved.

Emergence of project teams

• Project teams are “time-limited, draw membersfrom different disciplines and functional units andproduce one-time outputs” (Cohen & Bailey, 1997)

– Built on the basis of common interests or activities

(called foci) (Corman & Scott, 1994; Feld, 1981)

– Often entail voluntary participation (Hahn et al., 2008)

– Collaboration across organizational, cultural and

geographical boundaries (Beyene et al, 2009; Hinds et al, 2011)

– Relatively short life span, lack of bonding relations (Cohen

& Bailey, 1997)

– Embedded in the social context (Granovetter, 1985; Uzzi, 1997)

3


Examples of self-assembled project teams

Inter-disciplinary and multi-institutional scientific collaboration

4

Open source software development teams that collaborate over the Internet

Teams in games, especially in Massively Multiplayer Online Role-playing Games (MMORPGs)

11/27/2014

2


Challenges and opportunities in studying self-assembled project teams

• Theories

– Traditional teams with fixed boundaries vs. dynamics in

self-assembled project teams

• Data

– Difficulty of collecting fine-grained data for organizing

– Availability of big data thanks to the recent development in technological capability

• Methods

– New methods and techniques, such as exponential

random graph modeling or p*

• Computational Infrastructure

– Supercomputing cluster and petascale applications

5


Three Levels of Team Analyses

6

(b) Team as individuals

and relations (e.g. Guimera et al., 2005)

(a) Team as a unit (e.g. Ruef et al., 2003)

(c) Team as

individuals and tasks (Methodology adopted in

this study*)

Individual

Task

Bipartite Network Approach

* Zhu, M., Huang, Y., & Contractor, N. S. (2013). Motivations for self-assembling into project

teams. Social Networks, 35(2), 251–264.


Personal Motivations on Project Team Assembly

• Self interest (Coleman, 1986; Monge & Contractor, 2003)

H1. Low-skilled individuals are more likely to join teams than high-skilled individuals.

• Mutual interest and collective action (Fulk et al., 2004; Feld,

1981,1984)

H2. Individuals are more likely to join teams for more difficult projects.

• Coordination cost (Becker & Murphy, 1992; Gulati & Singh, 1998)

H3. Individuals are less likely to join teams on projects that require a longer duration.

7


Dyadic Motivations on Project Team Assembly

• Exchange and Dependency Theories (Blau, 1964; Homans, 1974)

H4. Individuals are less likely to collaborate with others with the same expertise.

• Homophily and Swift Trust (Carley, 1991; Ibarra & Andrews, 1993;

McPherson & Smith-Lovin, 1987)

H5a,b,c. Individuals are more likely to collaborate with others of

the same gender, similar age, and the same organizational affiliation.

• Homophily and Social Exchange (Blau, 1964; Homans, 1958,

1974)

H6. Individuals are more likely to collaborate with others with

similar skill levels.

• Coevolution (McKelvey, 1997; Guimera et al., 2005)

H7. Individuals who work together in one team are more likely to

join another team together.

8

11/27/2014

3


Structural signatures of hypotheses

9

+

H2: Project difficulty

- H3: Project duration

- H1: Individual skill level

_

H4: Expertise matching

d.

+ H6: Skill level similarity

f.

H7: Coevolution

+ g.

+

H5a,b,c: Gender, age

& affiliation Similarity

e.

Person

Project

Person w/ attributes

Project w/ attributes

a.

b.

c.


Dataset for empirical studies

• Data from a popular Massively Multiplayer Online Role

Playing Game (MMORPG) Everquest II (EQ2)

– 3D fantasy based game

– Server-side records

– Focus on combat teams

– First released on 2004 with 10+ expansions

– 3T+ server-side logs

10

http://everquest2.station.sony.com/screenshots.vm


Data • Dataset:

– Combat logs

• Sep. 5 – Sep. 11, 2006 on Guk Server as major dataset

– Player attributes(a snapshot around 6pm Sep. 4th, 2006)

• Gender, guild, and other character attributes

11


Team Identification and Zone-based Sampling

12

Team Network in Zone Antonica

Time Location Record

ID

Player

ID

XP class eff-lv Group

Level

size level reason

1 6342862 25082 1 27 1 22 22 2 38 5

1 6342862 25083 2 113 2 22 22 2 22 4

2 6342862 25085 1 13 1 22 22 2 38 5

2 6342862 25086 2 57 2 22 22 2 22 3

4 6342868 206247 3 53 1 25 24 5 29 5

4 6342868 206248 4 111 3 25 24 5 25 4

4 6342868 206249 5 106 3 22 24 5 22 4

4 6342868 206250 6 106 3 23 24 5 23 4

4 6342868 206251 7 106 2 22 24 5 22 4

Team 1: Player 1 & 2

Team 2: Player 3,4,5,6 & 7

4,537 project teams assembled by 2,426

unique player characters

11/27/2014

4


Statistical Analysis Methods • Exponential Random Graph Models (ERGMs) or p* models

(Frank & Strauss, 1986;Strauss & Ikeda,1990; Wasserman & Pattison, 1996)

– Dependence in network data

– Multiple levels of analysis

• ERGMs are a class of stochastic models that share the

following general form

where,

– Y is the network realization, similar to a random variable

– y is the observed network

– g(y) is a vector of structural signatures

– θ is a vector of coefficients corresponding to g(y); and k(θ) is a

normalizing factor

• Meta-analyses to aggregate results from 12 zones

13

1( ) exp( ( ))

( )

TP Y y g yk


Summary of results: Structural signatures of hypotheses

19

+

H2: Project difficulty

b.

-

H3: Project duration

c.

-

H1: Individual skill level

a. _

H4: Expertise matching

d.

+ H6: Skill level similarity

f.

H7: Coevolution

+ g.

+

H5a,b,c: Gender, age

& affiliation Similarity

e.

Person

Project

Person w/ attributes

Project w/ attributes


Major findings on Team Assembly

• Individuals are motivated to join ad hoc teams to accomplish difficult projects but not projects with very long duration

• Individuals tend to collaborate with specific teammates who have complementary skills, those who have similar age or skill level, and those who have same organizational affiliation

20


Correspondence analysis of team performance on bipartite networks

• Teams are often overlapping with each other

• Correspondence analysis (Wasserman & Faust, 1994)

can accommodate dependencies among teams

21

192 players 189 teams

Affiliation network of individuals and teams in zone everfrost

with spring embedding layout

11/27/2014

5


Example: Multiple correspondence analysis of guild diversity and team

performance

22

Guild diversity is positively related to short-term performance, and negatively related to long-term performance


Discussions

• Boundary conditions of teams in EverQuest II

– Project teams with limited life span

– Clearly defined goals and membership

– Self-organized

– Diverse skills required

• Insights for assembling better teams

23


What’s Next? Teams & Big Data

24

(b) Team as individuals

and relations (a) Team as a unit (c)Team as individuals

and tasks

Individual

Task

Bipartite Network

(d) Ecosystem of teams

Hypergraph


Mengxiao Zhu

[email protected]

25

Acknowledgements

mailto:[email protected]

http://www.nsf.gov/index.jsp

11/27/2014

6


26


Combat classes and social organizations in Everquest II

27

Fighter Main “tank”, absorbing damage to protect the rest of the group

Priest Healing group members

Mage Attacking using magic powers from a distance

Scout Attacking in close proximity to enemies

Dhona

An In-game Guild Hall

Mengxiao Zhu


28


Partnership

Trade

In-game mail

Instant messaging

Black: male

Red: female

Virtual world social system

11/27/2014

7


Descriptive statistics of variables

30

Player Attributes Mean S.D. Min Median Max

Female 0.34 0 1

Fighter expertise class 0.32 0 1

Priest expertise class 0.25 0 1

Mage expertise class 0.25 0 1

Scout expertise class 0.18 0 1

Age 33.86 9.85 14 33 76

Skill level 40.90 16.41 2 41 70

Prior teaming 6.42 9.80 0 3 88

Team Project Attributes Mean S.D. Min Median Max

Teaming required 0.62 0 1

Project difficulty 39.03 14.81 8 38 72

Project duration 31.17 40.41 0 16 432


Zone name # players # Teams Avg Team Size

Antonica 333 426 3.01

Commonlands 239 318 3.04

Desert of Flames 473 687 3.47

Enchanted 514 653 3.73

Everfrost 192 189 3.66

Feerrott 216 220 3.90

Kingdom of Sky 451 648 3.20

Lavastorm 168 159 3.80

Nektulos 223 244 3.05

Qeynos 80 67 2.39

Thundering Steppes 612 757 3.24

Zek 168 169 2.85

Descriptive Statistics of 12 Zone-based Samples

31


Effect Hypothesis (s.e.)

Density (edge) Control 12 536 -0.85 (2.40)

Active players Control 12 94 -0.006 (0.128)

Big teams Control 12 213 -2.57* (1.29)

Female Control 12 131 -0.25 (0.14)

Age Control 12 96 0.01* (0.002)

Priest class Control 12 54 -0.16* (0.06)

Mage class Control 12 512 -0.14* (0.06)

Scout class Control 12 115 -0.33* (0.06)

Teaming tendency Control 12 778 0.02* (0.002)

Team required Control 12 241 0.51* (0.09)

Skill level H1 12 167 -0.004 (0.006)

Project difficulty H2 12 258 0.06* (0.01)

Project duration H3 12 156 -0.01* (<0.001)

Class matching H4 12 1253 -0.84* (0.08)

Gender matching H5a 12 201 -0.15 (0.10)

Age difference H5b 12 378 -0.01* (0.003)

Guild matching H5c 12 20671 1.44* (0.12)

Skill level difference H6 12 2358 -0.06* (0.008)

Re-teaming H7 12 1367 -0.03 (0.03)

Meta-analysis Results Over 12 Zones

32

* p<0.05


Effect Hypothesis (s.e.) Personal Motivations

Control variables not displayed

Skill level H1

Project difficulty H2

Project duration H3

Class matching H4

Gender matching H5a

Age difference H5b

Guild matching H5c

Skill level difference H6

Re-teaming H7

12 167 -0.004 (0.006)

12 258 0.06* (0.01)

12 156 -0.01* (<0.001)

12 1253 -0.84* (0.08)

12 201 -0.15 (0.10)

12 378 -0.01* (0.003)

12 20671 1.44* (0.12)

12 2358 -0.06* (0.008)

12 1367 -0.03 (0.03)

Meta-analysis results over 12 Zones

33

* p<0.05

11/27/2014

8


Effect Hypothesis (s.e.)

Control variables not displayed

Skill level H1 12 167 -0.004 (0.006)

Project difficulty H2 12 258 0.06* Dyadic (0.01)Motivations

Project duration H3 12 156 -0.01* (<0.001)

Class matching H4 12 1253 -0.84* (0.08)

Gender matching H5a 12 201 -0.15 (0.10)

Age difference H5b 12 378 -0.01* (0.003)

Guild matching H5c 12 20671 1.44* (0.12)

Skill level difference H6 12 2358 -0.06* (0.008)

Re-teaming H7 12 1367 -0.03 (0.03)

* p<0.05

Meta-analysis results over 12 Zones

34

team assembly and performance in large scale online role ... · team assembly and performance in...

Documents