badges to acknowledge open practices

Badges to Acknowledge Open Practices

Andrew SallansCenter for Open Science

http://cos.io IASSIST

June 2014

Openness is a core value of scientific practice

Challenges: Perceived normsNorms

Communality Open Sharing

Universalism Evaluate research on own merit

Disinterestedness Motivated by knowledge and discovery

Organized skepticism Consider all new evidence, even against

one’s prior work

Quality

Counternorms

Secrecy Closed

Particularlism Evaluate research by reputation

Self-interestedness Treat science as a competition

Organized dogmatism Invest career promoting one’s own

theories, findings

Quantity

Anderson, Martinson & DeVries, 2007

Challenges

● Perceived norms (Anderson, Martinson & DeVries, 2007)

● Motivated reasoning (Kunda, 1990)

● Minimal accountability (Lerner & Tetlock, 1999)

● I am busy (Everyone)

How might this change?

Top-down● Committees● Journal Editors● Federal Agencies● University

administrators

Bottom-up● Exposing users to

good practices● Training sessions● Workshops● Spreading the

message through active publishing

● Going to where the user is now

● Consultants serving as “Shock troops”

Focus on simple interventions that researchers can actually adopt.

How to get a badge...1. Disclosure --- author provides public statement that they’ve met badge criteria

2. Peer Reviewed --- independent review of author’s public statement and meeting of criteria

What are the criteria?● Persistent path to the data, materials, pre

registration, etc.● Sufficient information for an independent

person to reproduce the results● Other more specialized items...

Who endorses these badges?

Which journals are adopting?

Going further…baking badges

What comes next?● Improve the badge issuing workflow● Refine integration into peer review process● Gain adoption of more journals/organizations● Measure effect

Lastly, this is a community effort...Ben B. Blohowiak | Johanna Cohoon | Lee de-Wit | Eric Eich | Frank J. Farach | Roger Giner-Sorolla | Fred Hasselman | Alex O. Holcombe | Macartan Humphreys | Melissa Lewis | Brian A. Nosek | Jonathan Peirce | Andrew Sallans | Jeffrey R. Spies | Chris Seto | Sara Bowman

Note: bold are non-COS

Thanks!Find out more and get involved here:https://osf.io/tvyxz/

Email me: andrew@cos.io Twitter: @asallans