evolving species—the journal in transition (part 2 of 2), sara bowman

Sara BowmanCenter for Open Science

http://cos.io/ | http://osf.io

Realigning Incentives:Registered Reports

Norms

CommunalityOpen sharing

UniversalismEvaluate research on own merit

DisinterestednessMotivated by knowledge and discovery

Organized skepticismConsider all new evidence, even against one’s prior work

Quality

Counternorms

SecrecyClosed

ParticularlismEvaluate research by reputation

Self-interestednessTreat science as a competition

Organized dogmatismInvest career promoting one’s own theories, findings

Quantity

Anderson, Martinson, & DeVries, 2007

Anderson, Martinson, & DeVries, 2007

Anderson, Martinson, & DeVries, 2007

Incentives for individual success

are focused on getting it

published, not getting it right

Nosek, Spies, & Motyl, 2012

Read more: osf.io/8mpji

ProblemsFlexibility in analysisSelective reporting

Ignoring nullsLack of replication

Sterling, 1959; Cohen, 1962; Lykken, 1968; Tukey, 1969; Greenwald, 1975; Meehl, 1978; Rosenthal, 1979

Design Collect & Analyze Report Publish

PEERREVIE

W

Registered Reports

Design Collect & Analyze Report Publish

PEERREVIE

W

Great… but how?Stage 1 Registered Report:Intro, Methods, Proposed analyses, Pilot Data (if applicable)

Editorial Triage Manuscript Rejected

Stage 1 Reviewers Invited

In-Principle Acceptance

Stage 2 Reviewers Invited

Stage 2 Registered Report:Intro, Methods, Results, Discussion

Manuscript Rejected

Revision Invited

Manuscript Rejected

Revision Invited

Full manuscript acceptance and publication

Authors conduct study Manuscript withdrawn

http://osf.io/8mpji

Community-Driven

Who Publishes Registered Reports?

See the full list and compare features: osf.io/8mpji

Special Issue

Level 3 TOP Guidelines: cos.io/top

For Replications

Registered Reports• AIMS Neuroscience• Attention, Perception, &

Psychophysics• Cognition and Emotion• Comprehensive Results in

Social Psychology• Cortex• Drug and Alcohol

Dependence• eLife• Euro Journal of

Neuroscience• Experimental Psychology

• Journal of Accounting Research

• Journal of Business and Psychology

• Journal of Personnel Psychology

• Journal of Media Psychology• Nutrition and Food Science

Journal• Perspectives on Psych.

Science• Royal Society Open Science• Social Psychology• Stress and Health• Work, Aging, and

Retirement

My authors aren’t ready.reviewers

editors…

http://osf.io/8mpji

Community-Driven

Incremental Step…

The $1,000,000 Preregistration Challenge

Endorse TOP GuidelinesBadges for Open Practices

Registered Reports

Free training on how to make research more

reproduciblehttp://cos.io/stats_consulting

What can you do?

Partner with us to promote incentives that help realign values and

practices

Meet researchers where they are and help them

find immediate benefit in training

Find this presentation:https://osf.io/xmu4h

Questions: sara@cos.io | contact@cos.io

Thank you!Questions?

sara@cos.io

Slides: https://osf.io/t8df9