authoritarian and democratic data science in an experimenting society

Authoritarian & Democratic

Data Science in an

Experimenting SocietyMIT CMS/W, Feb 16, 2017

@natematias

natematias.com

civic.mit.edu/users/natematias

J. Nathan Matias

http://civic.mit.edu/users/natematias

McMillen, Andrew. Wikipedia Is Not

Therapy: How the online encyclopedia

manages mental illness and suicide

threats in its volunteer community.

Backchannel. Illustration by Laurent Hrybyk

Goldman, Adam. (2016). The Comet Ping Pong Gunman Answers Our Reporter’s Questions. New

York Times

http://www.nytimes.com/2016/12/07/us/edgar-welch-comet-pizza-fake-news.html

Report to Law Enforcement

Report to reddit Platform

Report to Community Moderators

Up-Vote or Down-Vote

negative feedback leads to significant

behavioral changes that are detrimental to

the community.

Not only do authors of negatively-evaluated

content contribute more, but also their future

posts are of lower quality, and are

perceived by the community as such.

Cheng, J., Danescu-Niculescu-Mizil, C. & Leskovec, J. (2014). How Community Feedback Shapes User

Behavior. ICWSM 2014.

Downvote

Button

No

Downvote

Button

Gerber, A. S., & Green, D. P. (2012). Field experiments: Design, analysis, and interpretation. WW Norton.

Experiments Per Day on bing.com

Kohavi, R., Deng, A., Frasca, B., Walker, T., Xu, Y., & Pohlmann, N. (2013, August). Online controlled

experiments at large scale. In Proceedings of the 19th ACM SIGKDD international conference on

Knowledge discovery and data mining (pp. 1168-1176). ACM.

http://bing.com

Geiger, S. (2015). Does facebook have civil servants? On governmentality and computational social

science. In Workshop on Ethics for Studying Sociotechnical Systems in a Big Data World. Vancouver,

British Columbia, Canada.

academic and industry researchers who

work for institutions that build and operate our

digitally mediated public spaces are either

directly doing governance work themselves

or building systems that have been delegated

governance work.

In this sense, researchers can be said to

form a core part of the elite civil service

and bureaucratic corps of our era

MacKinnon, R. (2012). Consent of the networked: The worldwide struggle for Internet freedom.

Basic Books

Companies act as the new sovereigns of

cyberspace… most companies’ failure to take

responsibility for their power over citizens’

political lives, and their lack of

accountability in the exercise of that

power, corrodes the Internet’s democratic

potential

AUTHORITARIAN

& DEMOCRATIC

DATA SCIENCE

in an Experimenting

Society

MANAGEMENT

&

POLICY

Tiziana Terranova (2000) Free Labor: Producing Culture for the Digital Economy. Social Text

the Internet is about the extraction of value

out of continuous, updateable work

[consumption & production of culture]

[….]

Such means of production need to be

cultivated by encouraging the worker to

participate in a culture of exchange, whose

flows are mainly kept within the company

Prahalad, C. K., and Venkat Ramaswamy. 2004. Co-Creation Experiences: The next Practice in Value

Creation. Journal of Interactive Marketing 18 (3): 5–14.

the market is becoming a forum for

conversations

managers need to invest in building new

infrastructure capabilities, as well as new

functional and governance capabilities

Gillespie, T. (2010). The politics of “platforms.” New Media & Society, 12(3), 347–364.

[platform] choices about what can appear,

how it is organized, how it is monetized, what

can be removed and why, and what the

technical architecture allows and prohibits, are

all real and substantive interventions into

the contours of public discourse.

JoAnne Yates (1989) Control Through Communication: The Rise of System in American Management.

Johns Hopkins University Press

Systematic management attempted to

improve control over–and thus the efficiency

of–managers, workers, materials, and

production processes



Management

Theories for

Scaled

Operations

Growth in

Scale &

Complexity of

Industry

Comm & Info

Technology



Systematicall

y-Defined

Roles

Stopwatch

Frank Bunker Gilbreth and Lillian M. Gilbreth (1910-1924) Original films of Frank & Lillian Gilbreth.

Source: Prelinger Archives, via Wikimedia Commons



Performance

Monitoring

Statistics

Systematicall

y-Defined

Roles

Stopwatch

Chandler Jr, A. D. (1977). The Visible Hand: The Managerial Revolution in American Business.

Harvard University Press.

For the middle and top managers, control

through statistics quickly became both a

science and an art. This need for accurate

information led to the devising of improved

methods for collecting, collating, and

analyzing a wide variety of data generated

by the day-to-day operations of the enterprise.

16" Barbette Carriage Model 1919. Watertown Arsenal. Source: Watertown Free Library

https://www.digitalcommonwealth.org/search/commonwealth:rv042t876

Valentine, R. (1916). The progressive relation between efficiency and consent. Bulletin of Taylor

Society, 2(1)

Valentine, R. (1916). The progressive relation between efficiency and consent. Bulletin of Taylor

Society, 2(1)

A free man—a consenting man— is the more

desirable worker…

organized consent as well as individual

consent is the basis of a more efficient

group.

…build up a finer texture of democracy

through self-training groups, constantly growing

in strength through the consideration of

scientifically-accurate data.

Marshall, Edward. (1913) Industrial Psychologist’ to Prevent Labor Troubles. The New York Times,

April 27, 1913: Magazine Section Part Five, 11

Adolf Hitler delivers a speech at the Kroll Opera House,

Dec 11, 1941. Image source: Wikimedia Commons

https://commons.wikimedia.org/wiki/File:Bundesarchiv_Bild_183-1987-0703-507,_Berlin,_Reichstagssitzung,_Rede_Adolf_Hitler.jpg

Lewin, K. (1944). The dynamics of group action. Educational Leadership, 1(4), 195–200.

Efficient democracy means

organization, but it means

organization and leadership on

different principles than

autocracy.

It is essential that a

democratic commonwealth

and its educational system

apply the rational procedures

of scientific investigation to

its own processes of group

living.

Kurt Lewin. Image source: Wikipedia

Burnes, B. (2007). Kurt Lewin and the Harwood Studies The Foundations of OD. The Journal of Applied

Behavioral Science, 43(2), 213-231.

continue

autocratic

management

Harwood Pajama Factory Experiments

• Increasing Productivity

• Reducing Employee Turnover

workers

discuss &

vote

on

management

changes

Wikimedia CommonsCoch, L., & French, J. (1948). Overcoming resistance to change. Human Relations, 1, 512–532.

https://commons.wikimedia.org/wiki/File:Bundesarchiv_Bild_183-1987-0703-507,_Berlin,_Reichstagssitzung,_Rede_Adolf_Hitler.jpg

Intervention

Design

Goal

Setting &

Variable

Definition

Analysis &

Interpretatio

n

Group

Discussions

Group

Votes

Marrow, A. J. (1977). The Practical Theorist: The Life and Work of Kurt Lewin. Pubn Dev Co.

Adelman, C. (1993). Kurt Lewin and the origins of action research. Educational action research, 1(1),

7-24.

the residents of the affected community

must be involved in the research process

from the beginning

we will build the Great Society. It is a

Society where no child will go unfed,

and no youngster will go unschooled

Johnson, Lyndon. 323 -

Remarks in Athens at Ohio

University. May 7, 1964

Image source: Wikipedia: First Lady

Lady Bird Johnson visits a Head

Start class in 1966

https://en.wikipedia.org/wiki/Head_Start_(program)#/media/File:Lady_Bird_Johnson_Visiting_a_Classroom_for_Project_Head_Start_1966.gif

US National Security Agency System/360 85 Console in 1971. Image source: NSA via Wikimedia Commons

https://commons.wikimedia.org/wiki/File:Supercomputer_NSA-IBM360_85.jpg

Campbell, D. T. (1998). The experimenting society. In The experimenting society: Essays in honor of

Donald T. Campbell (p. 35). New Brunswick: Transaction Publishers.

Can the open society be an

experimenting society?

Popper, K. (1947). The open society and its enemies. Routledge.

Closed Societies

“the learned should rule”

Open Societies

the public evaluates &

criticizes government

“so that bad or

incompetent rulers can

be prevented from doing

too much damage”


the social engineer conceives as the

scientific basis of politics something like a

social technology

the Utopian engineer will have to be deaf to

many complaints ; in fact, it will be part of his

business to suppress unreasonable

objections. But with it, he must invariably

suppress reasonable criticism also


The piecemeal engineer will, accordingly,

adopt the method of searching for, and

fighting against, the greatest and most

urgent evils of society…

There will be a possibility of reaching a

reasonable compromise and therefore of

achieving the improvement by democratic

methods.

Image source: Wikipedia: First Lady

Lady Bird Johnson visits a Head

Start class in 1966

https://en.wikipedia.org/wiki/Head_Start_(program)#/media/File:Lady_Bird_Johnson_Visiting_a_Classroom_for_Project_Head_Start_1966.gif

Williams, W., & Evans, J. W. (1969). The Politics of Evaluation: The Case of Head Start. The ANNALS

of the American Academy of Political and Social Science, 385(1), 118–132

the absolute power of analysis was

oversold

the conflicts in the system between the

analytical staff and the operators of the

programs was underestimated.

Williams, W. (1971). Social Policy Research and Analysis: The Experience in the Federal Social

Agencies. American Elsevier Publishing Company.

Discard

Neutrality

Government Social

Scientists Should

Propose

Policy

Manage

Policies

Advocate

for Policy

RESEARCH is

DESIGN

& we can

REDESIGN our

METHODS to follow

DEMOCRATIC VALUES



Participation in policy experiments is more

akin to participating in democratic political

decision making than to participating in the

psychology laboratory. These restrictions all

have costs in the validity of experimental

inference.

the task of first priority for the methodologists

of the experimenting society is to design

experimental arrangements that obviate

these difficulties



The Contagious Cross-Validation Model for

Local Programs…

national funding would support adoptions that

included locally designed cross-validating

evaluations…



it is those who have situation-specific

information who make the best critics, and

the best judges, of the plausibility of most of

the rival hypotheses…

we must provide these nonprofessional

observers with the self-confidence and

opportunity to publicly disagree with the

conclusions of the professional applied social

scientists.



Management

Theories for

Scaled

Operations

Growth in

Scale &

Complexity

Comm & Info

Technology

Geiger, S. (2015). Does facebook have civil servants? On governmentality and computational social

science. In Workshop on Ethics for Studying Sociotechnical Systems in a Big Data World. Vancouver,

British Columbia, Canada.

In this sense, researchers can be said to

form a core part of the elite civil service

and bureaucratic corps of our era

Most Policy

& Platform

Experiments

Arnstein, Sherry R. 1969. “A Ladder of Citizen Participation.” Journal of the American Institute of

Planners 35 (4): 216–24.

MacKinnon, R. (2012). Consent of the networked: The worldwide struggle for Internet freedom.

Basic Books

Companies act as the new sovereigns of

cyberspace… most companies’ failure to take

responsibility for their power over citizens’

political lives, and their lack of

accountability in the exercise of that

power, corrodes the Internet’s democratic

potential

civilservant.io

Laws Platform

Policies

Community

Policies

Logo images via Wikipedia

conference hosts

moderators

community leaders

administrators

moderators

moderators

admins

enforcement united

52k subreddit

communities

200m monthly

visitors

148k moderator

rolesJuly 2015

Most Platforms

& Users

Online

Communities

Arnstein, Sherry R. 1969. “A Ladder of Citizen Participation.” Journal of the American Institute of

Planners 35 (4): 216–24.

13.5 million subscribers1,200+ moderator roles

989 newcomers/day77 discussions per day

0 1250 2500 3750 5000

Remove Post

Approve Post

Remove Comment

Approve Comment

Ban User

Unban User

Revise Wiki

Recategorize Post

Automated Systems Humans

8,298 Moderation Actions, May 23 - 29, 2016

Does making participants aware of rules

by posting them increase norm-

compliance of first-time commenters?

• Design Experiments

• Coordinate Policy

Interventions

• Monitor Outcomes

• Estimate Experimental Results

Civil ServantCommunity-Led Field Experiments in

Community Governance Online

x Only routine interventions

x No high risk communities

(markets, mental health)

x No groups that organize

to harm others



Open Archive of

Moderation Studies

Community

Experiments



Community

Suggests

Study

Refine

Study

Designs

Deploy Experiment

With Community

Interpret

Results

Debrief

Participants

Debate Policy

Decisions

Publish &

Replicate

Process for Community Experiments

CivilServant.io

New Sticky Comment

Ask-Me-Anything Sticky Comment

Matias, J. N. (2016) Posting Rules in Online Discussions Prevents Problems & Increases

Participation. CivilServant

“Sticking” a Rule Comment to Threads

Increased a Newcomer’s Probability of

Posting a First Comment Within the Rules

Posting the rules increases the incidence

rate of newcomer comments by 38.1% on

average.

If the community adopts sticky comments,

they could prevent 1,838 people a month

from engaging in unacceptable behavior.

They would also gain 9,631 new

commenters per month, on average.



13.7 million subscribers70+ moderator roles

23 tabloid discussions per day

/r/worldnews

[Misleading Title] Bavaria passes new law to

make migrants respect ‘dominant’ local

culture

[Misleading Title | Not Appropriate Subreddit]

Spanish Terror Attack: Gunman enters

supermarket, shouts Allahu Akbar

[Editorialized Title] A last kiss for mama:

Jihadi parents bid young daughters

goodbye… before one walks into a

Damascus police station and is blown up

by remote detonator



Unreliable News

Submitted

Suggest Fact-

Checking

reddit

Algorithms

Notice

Algorithms

Promote

Unreliable

News?

People

Fact-Check

Articles

Can we increase the rate that commenters

question unreliable news without

making unreliable news trend on social

media algorithms?

Pre

dic

ted

In

cid

en

ce o

f C

om

men

ts W

ith

Lin

ks

Encouraging Fact-Checking Causes Unreliable News

To Receive 2x More Comments with Links on Average Tabloid links in r/worldnews receive a 2.01 to 2.03x increase in the number of comments including

links to further evidence when moderators use sticky comments to encourage fact-checking.

Source: J. Nathan Matias, MIT Media Lab. Experiment by r/worldnews, 11/27/2016 – 1/20/2017

n = 840 posts from sites that moderators consider tabloids, 2.4% of submissions on average.

This negative binomial model predicts incidence rates; the effect is larger for more popular posts.

Fact-checking: p = 0.0083. Fact-checking + Voting: p = 0.0073 *** p<0.001, ** p<0.01, * p<0.05

For full details on the findings, which were not yet peer reviewed by Jan 2017, see civilservant.io

1.44**

0.71

1.46**

No Action

Taken

Suggest Fact-

Checking

Suggest Fact-

Checking & Voting

http://civilservant.io

Encouraging Fact-Checking Causes Unreliable News

To Be Promoted Less by reddit’s Algorithms on AverageTabloid links in r/worldnews receive a 2.04x reduction in the scores that shape reddit’s rankings

when moderators encouraged fact-checking, but not when they also suggested voting

Source: J. Nathan Matias, MIT Media Lab. Experiment by r/worldnews, 12/07/2016 – 1/20/2017

n = 696 posts from sites that moderators consider tabloids, 2.4% of submissions on average.

The reddit algorithms use the “score” to determine the ranking of a link. On average, between

links of similar age, the submission with a higher score will be ranked more highly.

This negative binomial model predicts incidence rates; the effect is larger for more popular posts.

Fact-checking intervention p = 0.000562. Voting p = 0.198 *** p<0.001, ** p<0.01, * p<0.05

For full details on the findings, which were not yet peer reviewed by Jan 2017, see civilservant.io

50.56***

103.07

134.37

No Action

Taken

Suggest Fact-

Checking

Suggest Fact-

Checking & Voting

Pre

dic

ted

Sco

re I

ncid

en

ce R

ate

Aft

er

24 h

rs

http://civilservant.io

Community Discussion


Policy Discussion:

What if lack of conflict & increased participation is bad?

Can this cause censorship if taken to an extreme?

How generalizable is this to other subs?

Intervention Design:

I imagine the wording is extremely important.


Personal Stories of Outliers:

I don't think I've ever read any subreddit's rules ever.

Experiment Design & Implications:

I bet that the rules comment increases participation

because it makes it say “(1 comment)” on the forum

index so people click the link to read the comment


Research Ethics:

Did you get the informed consent?

[IRBs] have no authority, legal or ethical, to make

decisions about consent.

you're objecting to this study as an excuse to critique

the moderators

Open Archive of

Moderation Studies

Community

Experiments

CivilServantCommunity-Led Field Experiments in


Data Sampled July 2015

15,300

1,795Eligible Communities

3,000 + comments/month

Moderator Roles

How Far Might Community Experiments

Scale on the reddit Platform?

RESEARCH is

DESIGN

& we can

REDESIGN our

METHODS to follow

DEMOCRATIC VALUES

civilservant.io

Ethan ZuckermanAssociate Professor of the Practice

Massachusetts Institute of Technology

Elizabeth Levy PaluckAssociate Professor, Department of Psychology

Woodrow Wilson School, Princeton University

Tarleton GillespiePrincipal Researcher

Microsoft Research

Merry MouM.Eng Student

Massachusetts Institute of Technology

Elinor OstromAnne Oakley Donna Haraway

Catherine Squires Ellen Swallow Richards

Authoritarian & Democratic

Data Science in an

Experimenting SocietyMIT CMS/W, Feb 16, 2017

@natematias

natematias.com

civic.mit.edu/users/natematias

J. Nathan Matias

http://civic.mit.edu/users/natematias