Predicting Leadership Roles in Email Workgroups

Vitor R. Carvalho, Wen Wu and William W. CohenCarnegie Mellon University

CEAS-2007, Aug 2nd 2007

Motivation

Link to leadership studies in hci Importance of email Leadership at a distance,

communication media, etc.[Butler et al, 2002; Fussell et al, 2001]

Overview: Corpus Study

Study: How well we can predict leadership

Textual and “network” features What are the most predictive features

Large and special collection of emails CSPACE corpus (aka GSIA corpus) Leaders (presidents of workgroups) were

previously determined

CSPACE corpus 15,000 messages from 277 students Emails associated with a semester-long project (14

weeks) of Carnegie Mellon MBA students To simulate companies competing for market share

and profit, students were divided in 50 companies/teams (4 to 6 students/team)

Very real: student grades largely based (70%) on company financial performance and external board review

Most communication happened inside group Very rich in task negotiation. Presidents assigned in the beginning of the game. Presidents selected other team members through a

round-robin draft.

Email dataset: network

= President…

Team A Team B

Team C Team D

Evidence from email header: network features

Broadcast messages Sent to all other team

members

Non-Broadcast messages Not sent all team

members

Evidence from email header: network features

Textual Features: Email Acts

Cohen et al., EMNLP-04: classification of email content in terms of having “email speech acts”

Examples: Deliver, Request, Commit, Propose, Meeting, etc.

Ciranda: Java toolkit available online

Textual Features: Email Acts

•We also used the associated ranking features, i.e., first (_1), second (_2), last (_last) and one but last (_butlast)•96 features total

Experiments 1

10-fold cross-validation using SVM with linear kernel

Experiments 2:

Restricting to a single president per group, using All Features: 96% of accuracy F1-measure of 0.882. It correctly predicts the president in 30 out of

34 groups (minimum of 20 messages)

Analysis: Feature Selection with 2 test

Analysis: Feature Selection with 2 test

Analysis Overall, results are interesting

Suggests some types of evidence are correlated with leadership

How “natural” is this dataset? Semester long, 14 weeks No language restriction (flames, arguments, cheering, arrangements,

gossip, etc.) Grade depended on the team financial performance

Leadership & the choice of leader Laboratory setting of work mitigates the reliability of the

conclusions How much self-selection is going on in choosing "President" of

company? Are students behaving according to their expectations of how executives behave?

Presidents were selected on a popular vote: the class as a whole (277 students) elected the 50 presidents.

Related Work

Community structure and leadership roles in email archives of organizations.

[Leuski, 2004][Tyler et al, 2003.]

Very different datasets, methods, validations, etc.

Thank you.