the replication and verification policy at the american ... · 7/11/2016 · in data access,...
TRANSCRIPT
![Page 1: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/1.jpg)
The Replication and Verification Policy at theAmerican Journal of Political Science
William G. JacobyEditor, American Journal of Political Science
Michigan State [email protected]
Prepared for the 2016 DataverseCommunity Meeting, Harvard University,
July 11, 2016
![Page 2: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/2.jpg)
Scientific Objectives
The American Journal of Political Science (AJPS) iscommitted to the general objectives of science
Generating and testing powerful theories of political andsocial phenomena
Rigorous and cumulative research efforts
Science is inherently a social enterprise
Progress within any field can only occur when researchershave ready access to the procedures, tools, data, andresults that were used to produce the existing state ofknowledge within that field.
![Page 3: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/3.jpg)
Scientific Objectives
The American Journal of Political Science (AJPS) iscommitted to the general objectives of science
Generating and testing powerful theories of political andsocial phenomena
Rigorous and cumulative research efforts
Science is inherently a social enterprise
Progress within any field can only occur when researchershave ready access to the procedures, tools, data, andresults that were used to produce the existing state ofknowledge within that field.
![Page 4: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/4.jpg)
Scientific Objectives
The American Journal of Political Science (AJPS) iscommitted to the general objectives of science
Generating and testing powerful theories of political andsocial phenomena
Rigorous and cumulative research efforts
Science is inherently a social enterprise
Progress within any field can only occur when researchershave ready access to the procedures, tools, data, andresults that were used to produce the existing state ofknowledge within that field.
![Page 5: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/5.jpg)
Scientific Objectives
The American Journal of Political Science (AJPS) iscommitted to the general objectives of science
Generating and testing powerful theories of political andsocial phenomena
Rigorous and cumulative research efforts
Science is inherently a social enterprise
Progress within any field can only occur when researchershave ready access to the procedures, tools, data, andresults that were used to produce the existing state ofknowledge within that field.
![Page 6: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/6.jpg)
Scientific Objectives
The American Journal of Political Science (AJPS) iscommitted to the general objectives of science
Generating and testing powerful theories of political andsocial phenomena
Rigorous and cumulative research efforts
Science is inherently a social enterprise
Progress within any field can only occur when researchershave ready access to the procedures, tools, data, andresults that were used to produce the existing state ofknowledge within that field.
![Page 7: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/7.jpg)
Scientific Objectives
The American Journal of Political Science (AJPS) iscommitted to the general objectives of science
Generating and testing powerful theories of political andsocial phenomena
Rigorous and cumulative research efforts
Science is inherently a social enterprise
Progress within any field can only occur when researchershave ready access to the procedures, tools, data, andresults that were used to produce the existing state ofknowledge within that field.
![Page 8: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/8.jpg)
AJPS is Taking a Leadership Role
Establishing a research-conducive environment withinthe political science discipline
Publishing great scientific work!
Making sure that interested researchers have access to the“raw materials” that generated work published in the AJPS
Only feasible way to establish and confirm the qualityof evidence-based claims within the social scienceresearch community
AJPS has made public commitment to best practicesin data access, research transparency andreproducibility of published research
![Page 9: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/9.jpg)
AJPS is Taking a Leadership Role
Establishing a research-conducive environment withinthe political science discipline
Publishing great scientific work!
Making sure that interested researchers have access to the“raw materials” that generated work published in the AJPS
Only feasible way to establish and confirm the qualityof evidence-based claims within the social scienceresearch community
AJPS has made public commitment to best practicesin data access, research transparency andreproducibility of published research
![Page 10: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/10.jpg)
AJPS is Taking a Leadership Role
Establishing a research-conducive environment withinthe political science discipline
Publishing great scientific work!
Making sure that interested researchers have access to the“raw materials” that generated work published in the AJPS
Only feasible way to establish and confirm the qualityof evidence-based claims within the social scienceresearch community
AJPS has made public commitment to best practicesin data access, research transparency andreproducibility of published research
![Page 11: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/11.jpg)
AJPS is Taking a Leadership Role
Establishing a research-conducive environment withinthe political science discipline
Publishing great scientific work!
Making sure that interested researchers have access to the“raw materials” that generated work published in the AJPS
Only feasible way to establish and confirm the qualityof evidence-based claims within the social scienceresearch community
AJPS has made public commitment to best practicesin data access, research transparency andreproducibility of published research
![Page 12: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/12.jpg)
AJPS is Taking a Leadership Role
Establishing a research-conducive environment withinthe political science discipline
Publishing great scientific work!
Making sure that interested researchers have access to the“raw materials” that generated work published in the AJPS
Only feasible way to establish and confirm the qualityof evidence-based claims within the social scienceresearch community
AJPS has made public commitment to best practicesin data access, research transparency andreproducibility of published research
![Page 13: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/13.jpg)
AJPS is Taking a Leadership Role
Establishing a research-conducive environment withinthe political science discipline
Publishing great scientific work!
Making sure that interested researchers have access to the“raw materials” that generated work published in the AJPS
Only feasible way to establish and confirm the qualityof evidence-based claims within the social scienceresearch community
AJPS has made public commitment to best practicesin data access, research transparency andreproducibility of published research
![Page 14: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/14.jpg)
AJPS Has Always Encouraged Replication
For many years, authors of AJPS articles have beenasked to provide their data for replication purposes
Compliance with request was entirely voluntary
Availability and quality of replication materials varied widely
Previous AJPS Editor, Rick Wilson, established a newpolicy in 2013
Authors must upload replication files to the AJPS Dataversewithin the Harvard Dataverse Network (https://dataverse.harvard.edu/dataverse/ajps)
No real guidance about what information and materialsshould be provided
Content of replication files continued to vary across articles
![Page 15: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/15.jpg)
AJPS Has Always Encouraged Replication
For many years, authors of AJPS articles have beenasked to provide their data for replication purposes
Compliance with request was entirely voluntary
Availability and quality of replication materials varied widely
Previous AJPS Editor, Rick Wilson, established a newpolicy in 2013
Authors must upload replication files to the AJPS Dataversewithin the Harvard Dataverse Network (https://dataverse.harvard.edu/dataverse/ajps)
No real guidance about what information and materialsshould be provided
Content of replication files continued to vary across articles
![Page 16: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/16.jpg)
AJPS Has Always Encouraged Replication
For many years, authors of AJPS articles have beenasked to provide their data for replication purposes
Compliance with request was entirely voluntary
Availability and quality of replication materials varied widely
Previous AJPS Editor, Rick Wilson, established a newpolicy in 2013
Authors must upload replication files to the AJPS Dataversewithin the Harvard Dataverse Network (https://dataverse.harvard.edu/dataverse/ajps)
No real guidance about what information and materialsshould be provided
Content of replication files continued to vary across articles
![Page 17: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/17.jpg)
AJPS Has Always Encouraged Replication
For many years, authors of AJPS articles have beenasked to provide their data for replication purposes
Compliance with request was entirely voluntary
Availability and quality of replication materials varied widely
Previous AJPS Editor, Rick Wilson, established a newpolicy in 2013
Authors must upload replication files to the AJPS Dataversewithin the Harvard Dataverse Network (https://dataverse.harvard.edu/dataverse/ajps)
No real guidance about what information and materialsshould be provided
Content of replication files continued to vary across articles
![Page 18: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/18.jpg)
AJPS Has Always Encouraged Replication
For many years, authors of AJPS articles have beenasked to provide their data for replication purposes
Compliance with request was entirely voluntary
Availability and quality of replication materials varied widely
Previous AJPS Editor, Rick Wilson, established a newpolicy in 2013
Authors must upload replication files to the AJPS Dataversewithin the Harvard Dataverse Network (https://dataverse.harvard.edu/dataverse/ajps)
No real guidance about what information and materialsshould be provided
Content of replication files continued to vary across articles
![Page 19: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/19.jpg)
AJPS Has Always Encouraged Replication
For many years, authors of AJPS articles have beenasked to provide their data for replication purposes
Compliance with request was entirely voluntary
Availability and quality of replication materials varied widely
Previous AJPS Editor, Rick Wilson, established a newpolicy in 2013
Authors must upload replication files to the AJPS Dataversewithin the Harvard Dataverse Network (https://dataverse.harvard.edu/dataverse/ajps)
No real guidance about what information and materialsshould be provided
Content of replication files continued to vary across articles
![Page 20: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/20.jpg)
AJPS Has Always Encouraged Replication
For many years, authors of AJPS articles have beenasked to provide their data for replication purposes
Compliance with request was entirely voluntary
Availability and quality of replication materials varied widely
Previous AJPS Editor, Rick Wilson, established a newpolicy in 2013
Authors must upload replication files to the AJPS Dataversewithin the Harvard Dataverse Network (https://dataverse.harvard.edu/dataverse/ajps)
No real guidance about what information and materialsshould be provided
Content of replication files continued to vary across articles
![Page 21: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/21.jpg)
AJPS Has Always Encouraged Replication
For many years, authors of AJPS articles have beenasked to provide their data for replication purposes
Compliance with request was entirely voluntary
Availability and quality of replication materials varied widely
Previous AJPS Editor, Rick Wilson, established a newpolicy in 2013
Authors must upload replication files to the AJPS Dataversewithin the Harvard Dataverse Network (https://dataverse.harvard.edu/dataverse/ajps)
No real guidance about what information and materialsshould be provided
Content of replication files continued to vary across articles
![Page 22: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/22.jpg)
New Policy Initiated in March 2015
Authors of AJPS articles still required to uploadreplication materials to the AJPS Dataverse
Now provide detailed guidance about materials thatneed to be provided
“The American Journal of Political Science Guidelines forPreparing Replication Files”
Originally oriented toward quantitative data, recentlyupdated to include guidelines for qualitative data
Replication materials are verified to make sure thatthey do, in fact, reproduce the analysis results
![Page 23: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/23.jpg)
New Policy Initiated in March 2015
Authors of AJPS articles still required to uploadreplication materials to the AJPS Dataverse
Now provide detailed guidance about materials thatneed to be provided
“The American Journal of Political Science Guidelines forPreparing Replication Files”
Originally oriented toward quantitative data, recentlyupdated to include guidelines for qualitative data
Replication materials are verified to make sure thatthey do, in fact, reproduce the analysis results
![Page 24: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/24.jpg)
New Policy Initiated in March 2015
Authors of AJPS articles still required to uploadreplication materials to the AJPS Dataverse
Now provide detailed guidance about materials thatneed to be provided
“The American Journal of Political Science Guidelines forPreparing Replication Files”
Originally oriented toward quantitative data, recentlyupdated to include guidelines for qualitative data
Replication materials are verified to make sure thatthey do, in fact, reproduce the analysis results
![Page 25: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/25.jpg)
New Policy Initiated in March 2015
Authors of AJPS articles still required to uploadreplication materials to the AJPS Dataverse
Now provide detailed guidance about materials thatneed to be provided
“The American Journal of Political Science Guidelines forPreparing Replication Files”
Originally oriented toward quantitative data, recentlyupdated to include guidelines for qualitative data
Replication materials are verified to make sure thatthey do, in fact, reproduce the analysis results
![Page 26: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/26.jpg)
New Policy Initiated in March 2015
Authors of AJPS articles still required to uploadreplication materials to the AJPS Dataverse
Now provide detailed guidance about materials thatneed to be provided
“The American Journal of Political Science Guidelines forPreparing Replication Files”
Originally oriented toward quantitative data, recentlyupdated to include guidelines for qualitative data
Replication materials are verified to make sure thatthey do, in fact, reproduce the analysis results
![Page 27: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/27.jpg)
New Policy Initiated in March 2015
Authors of AJPS articles still required to uploadreplication materials to the AJPS Dataverse
Now provide detailed guidance about materials thatneed to be provided
“The American Journal of Political Science Guidelines forPreparing Replication Files”
Originally oriented toward quantitative data, recentlyupdated to include guidelines for qualitative data
Replication materials are verified to make sure thatthey do, in fact, reproduce the analysis results
![Page 28: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/28.jpg)
AJPS Guidelines for Replication Files
For quantitative analyses, the Replication Datasetmust contain:
A README file listing and describing all replication filescontained in the article’s Dataset
The analysis dataset and a codebook describing itscontents
Detailed, clear code to reproduce all tables, figures, andexhibits in the AJPS article
Information about the source of the analysis dataset
Instructions for extracting the analysis dataset from thesource data (including all recodes, transformations, detailsabout missing observations, etc.)
![Page 29: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/29.jpg)
AJPS Guidelines for Replication Files
For quantitative analyses, the Replication Datasetmust contain:
A README file listing and describing all replication filescontained in the article’s Dataset
The analysis dataset and a codebook describing itscontents
Detailed, clear code to reproduce all tables, figures, andexhibits in the AJPS article
Information about the source of the analysis dataset
Instructions for extracting the analysis dataset from thesource data (including all recodes, transformations, detailsabout missing observations, etc.)
![Page 30: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/30.jpg)
AJPS Guidelines for Replication Files
For quantitative analyses, the Replication Datasetmust contain:
A README file listing and describing all replication filescontained in the article’s Dataset
The analysis dataset and a codebook describing itscontents
Detailed, clear code to reproduce all tables, figures, andexhibits in the AJPS article
Information about the source of the analysis dataset
Instructions for extracting the analysis dataset from thesource data (including all recodes, transformations, detailsabout missing observations, etc.)
![Page 31: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/31.jpg)
AJPS Guidelines for Replication Files
For quantitative analyses, the Replication Datasetmust contain:
A README file listing and describing all replication filescontained in the article’s Dataset
The analysis dataset and a codebook describing itscontents
Detailed, clear code to reproduce all tables, figures, andexhibits in the AJPS article
Information about the source of the analysis dataset
Instructions for extracting the analysis dataset from thesource data (including all recodes, transformations, detailsabout missing observations, etc.)
![Page 32: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/32.jpg)
AJPS Guidelines for Replication Files
For quantitative analyses, the Replication Datasetmust contain:
A README file listing and describing all replication filescontained in the article’s Dataset
The analysis dataset and a codebook describing itscontents
Detailed, clear code to reproduce all tables, figures, andexhibits in the AJPS article
Information about the source of the analysis dataset
Instructions for extracting the analysis dataset from thesource data (including all recodes, transformations, detailsabout missing observations, etc.)
![Page 33: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/33.jpg)
AJPS Guidelines for Replication Files
For quantitative analyses, the Replication Datasetmust contain:
A README file listing and describing all replication filescontained in the article’s Dataset
The analysis dataset and a codebook describing itscontents
Detailed, clear code to reproduce all tables, figures, andexhibits in the AJPS article
Information about the source of the analysis dataset
Instructions for extracting the analysis dataset from thesource data (including all recodes, transformations, detailsabout missing observations, etc.)
![Page 34: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/34.jpg)
AJPS Guidelines for Replication Files
For quantitative analyses, the Replication Datasetmust contain:
A README file listing and describing all replication filescontained in the article’s Dataset
The analysis dataset and a codebook describing itscontents
Detailed, clear code to reproduce all tables, figures, andexhibits in the AJPS article
Information about the source of the analysis dataset
Instructions for extracting the analysis dataset from thesource data (including all recodes, transformations, detailsabout missing observations, etc.)
![Page 35: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/35.jpg)
AJPS Guidelines for Replication Files
Diversity of qualitative analyses requires moreflexibility in replication requirements
Certain qualitative analyses (e.g., automated contentanalysis or qualitative comparative analysis) requiredata in matrix form, so requirements parallel those forquantitative data
A “granular” approach requires consideration ofspecific individual sources of information and typicallyrequires a “Tracking Appendix” (TRAX)
An overview of research trajectory
Annotations that make connections between informationsources and inferences/interpretations
![Page 36: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/36.jpg)
AJPS Guidelines for Replication Files
Diversity of qualitative analyses requires moreflexibility in replication requirements
Certain qualitative analyses (e.g., automated contentanalysis or qualitative comparative analysis) requiredata in matrix form, so requirements parallel those forquantitative data
A “granular” approach requires consideration ofspecific individual sources of information and typicallyrequires a “Tracking Appendix” (TRAX)
An overview of research trajectory
Annotations that make connections between informationsources and inferences/interpretations
![Page 37: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/37.jpg)
AJPS Guidelines for Replication Files
Diversity of qualitative analyses requires moreflexibility in replication requirements
Certain qualitative analyses (e.g., automated contentanalysis or qualitative comparative analysis) requiredata in matrix form, so requirements parallel those forquantitative data
A “granular” approach requires consideration ofspecific individual sources of information and typicallyrequires a “Tracking Appendix” (TRAX)
An overview of research trajectory
Annotations that make connections between informationsources and inferences/interpretations
![Page 38: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/38.jpg)
AJPS Guidelines for Replication Files
Diversity of qualitative analyses requires moreflexibility in replication requirements
Certain qualitative analyses (e.g., automated contentanalysis or qualitative comparative analysis) requiredata in matrix form, so requirements parallel those forquantitative data
A “granular” approach requires consideration ofspecific individual sources of information and typicallyrequires a “Tracking Appendix” (TRAX)
An overview of research trajectory
Annotations that make connections between informationsources and inferences/interpretations
![Page 39: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/39.jpg)
AJPS Guidelines for Replication Files
Diversity of qualitative analyses requires moreflexibility in replication requirements
Certain qualitative analyses (e.g., automated contentanalysis or qualitative comparative analysis) requiredata in matrix form, so requirements parallel those forquantitative data
A “granular” approach requires consideration ofspecific individual sources of information and typicallyrequires a “Tracking Appendix” (TRAX)
An overview of research trajectory
Annotations that make connections between informationsources and inferences/interpretations
![Page 40: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/40.jpg)
AJPS Guidelines for Replication Files
Diversity of qualitative analyses requires moreflexibility in replication requirements
Certain qualitative analyses (e.g., automated contentanalysis or qualitative comparative analysis) requiredata in matrix form, so requirements parallel those forquantitative data
A “granular” approach requires consideration ofspecific individual sources of information and typicallyrequires a “Tracking Appendix” (TRAX)
An overview of research trajectory
Annotations that make connections between informationsources and inferences/interpretations
![Page 41: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/41.jpg)
AJPS Guidelines for Replication Files
Objective is to establish a general standard for theinformation that must be made available
The Guidelines probably will not cover every possiblesituation.
They should be sufficient for most data analyses that arereported in AJPS articles.
Authors can request exemptions from replicationrequirements in some limited circumstances
Restricted access datasets
Human subjects protection
![Page 42: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/42.jpg)
AJPS Guidelines for Replication Files
Objective is to establish a general standard for theinformation that must be made available
The Guidelines probably will not cover every possiblesituation.
They should be sufficient for most data analyses that arereported in AJPS articles.
Authors can request exemptions from replicationrequirements in some limited circumstances
Restricted access datasets
Human subjects protection
![Page 43: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/43.jpg)
AJPS Guidelines for Replication Files
Objective is to establish a general standard for theinformation that must be made available
The Guidelines probably will not cover every possiblesituation.
They should be sufficient for most data analyses that arereported in AJPS articles.
Authors can request exemptions from replicationrequirements in some limited circumstances
Restricted access datasets
Human subjects protection
![Page 44: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/44.jpg)
AJPS Guidelines for Replication Files
Objective is to establish a general standard for theinformation that must be made available
The Guidelines probably will not cover every possiblesituation.
They should be sufficient for most data analyses that arereported in AJPS articles.
Authors can request exemptions from replicationrequirements in some limited circumstances
Restricted access datasets
Human subjects protection
![Page 45: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/45.jpg)
AJPS Guidelines for Replication Files
Objective is to establish a general standard for theinformation that must be made available
The Guidelines probably will not cover every possiblesituation.
They should be sufficient for most data analyses that arereported in AJPS articles.
Authors can request exemptions from replicationrequirements in some limited circumstances
Restricted access datasets
Human subjects protection
![Page 46: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/46.jpg)
AJPS Guidelines for Replication Files
Objective is to establish a general standard for theinformation that must be made available
The Guidelines probably will not cover every possiblesituation.
They should be sufficient for most data analyses that arereported in AJPS articles.
Authors can request exemptions from replicationrequirements in some limited circumstances
Restricted access datasets
Human subjects protection
![Page 47: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/47.jpg)
AJPS Guidelines for Replication Files
Objective is to establish a general standard for theinformation that must be made available
The Guidelines probably will not cover every possiblesituation.
They should be sufficient for most data analyses that arereported in AJPS articles.
Authors can request exemptions from replicationrequirements in some limited circumstances
Restricted access datasets
Human subjects protection
![Page 48: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/48.jpg)
Verifying Replication Materials
Authors are informed that acceptance of a manuscriptis contingent upon successful replication of theresults.
Verification carried out by third parties
For quantitative analyses, verification is carried out by theArchive Staff at the Odum Institute for Research in SocialScience, University of North Carolina at Chapel Hill.
For qualitative analyses, verification is carried out by thestaff of the Qualitative Data Repository (QDR), SyracuseUniversity.
Support for the verification process is provided by theMidwest Political Science Association
![Page 49: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/49.jpg)
Verifying Replication Materials
Authors are informed that acceptance of a manuscriptis contingent upon successful replication of theresults.
Verification carried out by third parties
For quantitative analyses, verification is carried out by theArchive Staff at the Odum Institute for Research in SocialScience, University of North Carolina at Chapel Hill.
For qualitative analyses, verification is carried out by thestaff of the Qualitative Data Repository (QDR), SyracuseUniversity.
Support for the verification process is provided by theMidwest Political Science Association
![Page 50: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/50.jpg)
Verifying Replication Materials
Authors are informed that acceptance of a manuscriptis contingent upon successful replication of theresults.
Verification carried out by third parties
For quantitative analyses, verification is carried out by theArchive Staff at the Odum Institute for Research in SocialScience, University of North Carolina at Chapel Hill.
For qualitative analyses, verification is carried out by thestaff of the Qualitative Data Repository (QDR), SyracuseUniversity.
Support for the verification process is provided by theMidwest Political Science Association
![Page 51: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/51.jpg)
Verifying Replication Materials
Authors are informed that acceptance of a manuscriptis contingent upon successful replication of theresults.
Verification carried out by third parties
For quantitative analyses, verification is carried out by theArchive Staff at the Odum Institute for Research in SocialScience, University of North Carolina at Chapel Hill.
For qualitative analyses, verification is carried out by thestaff of the Qualitative Data Repository (QDR), SyracuseUniversity.
Support for the verification process is provided by theMidwest Political Science Association
![Page 52: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/52.jpg)
Verifying Replication Materials
Authors are informed that acceptance of a manuscriptis contingent upon successful replication of theresults.
Verification carried out by third parties
For quantitative analyses, verification is carried out by theArchive Staff at the Odum Institute for Research in SocialScience, University of North Carolina at Chapel Hill.
For qualitative analyses, verification is carried out by thestaff of the Qualitative Data Repository (QDR), SyracuseUniversity.
Support for the verification process is provided by theMidwest Political Science Association
![Page 53: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/53.jpg)
Verifying Replication Materials
Authors are informed that acceptance of a manuscriptis contingent upon successful replication of theresults.
Verification carried out by third parties
For quantitative analyses, verification is carried out by theArchive Staff at the Odum Institute for Research in SocialScience, University of North Carolina at Chapel Hill.
For qualitative analyses, verification is carried out by thestaff of the Qualitative Data Repository (QDR), SyracuseUniversity.
Support for the verification process is provided by theMidwest Political Science Association
![Page 54: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/54.jpg)
Verifying Replication Materials
Verification is now a regular component of thepublication workflow.
Author uploads replication files to the AJPS Dataversewhen final draft of the manuscript is submitted.
Odum or QDR staff verify that the full set of replication fileshave been provided and that they do reproduce all analyticresults from the AJPS article.
If problems occur, Odum or QDR staff inform the AJPSEditor, who then informs the author.
Author resolves the issues and resubmits replicationmaterials for rechecking by Odum or QDR staff.
Process continues until the replication is carried outsuccessfully.
Odum or QDR staff release Dataverse Dataset for publicaccess.
![Page 55: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/55.jpg)
Verifying Replication Materials
Verification is now a regular component of thepublication workflow.
Author uploads replication files to the AJPS Dataversewhen final draft of the manuscript is submitted.
Odum or QDR staff verify that the full set of replication fileshave been provided and that they do reproduce all analyticresults from the AJPS article.
If problems occur, Odum or QDR staff inform the AJPSEditor, who then informs the author.
Author resolves the issues and resubmits replicationmaterials for rechecking by Odum or QDR staff.
Process continues until the replication is carried outsuccessfully.
Odum or QDR staff release Dataverse Dataset for publicaccess.
![Page 56: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/56.jpg)
Verifying Replication Materials
Verification is now a regular component of thepublication workflow.
Author uploads replication files to the AJPS Dataversewhen final draft of the manuscript is submitted.
Odum or QDR staff verify that the full set of replication fileshave been provided and that they do reproduce all analyticresults from the AJPS article.
If problems occur, Odum or QDR staff inform the AJPSEditor, who then informs the author.
Author resolves the issues and resubmits replicationmaterials for rechecking by Odum or QDR staff.
Process continues until the replication is carried outsuccessfully.
Odum or QDR staff release Dataverse Dataset for publicaccess.
![Page 57: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/57.jpg)
Verifying Replication Materials
Verification is now a regular component of thepublication workflow.
Author uploads replication files to the AJPS Dataversewhen final draft of the manuscript is submitted.
Odum or QDR staff verify that the full set of replication fileshave been provided and that they do reproduce all analyticresults from the AJPS article.
If problems occur, Odum or QDR staff inform the AJPSEditor, who then informs the author.
Author resolves the issues and resubmits replicationmaterials for rechecking by Odum or QDR staff.
Process continues until the replication is carried outsuccessfully.
Odum or QDR staff release Dataverse Dataset for publicaccess.
![Page 58: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/58.jpg)
Verifying Replication Materials
Verification is now a regular component of thepublication workflow.
Author uploads replication files to the AJPS Dataversewhen final draft of the manuscript is submitted.
Odum or QDR staff verify that the full set of replication fileshave been provided and that they do reproduce all analyticresults from the AJPS article.
If problems occur, Odum or QDR staff inform the AJPSEditor, who then informs the author.
Author resolves the issues and resubmits replicationmaterials for rechecking by Odum or QDR staff.
Process continues until the replication is carried outsuccessfully.
Odum or QDR staff release Dataverse Dataset for publicaccess.
![Page 59: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/59.jpg)
Verifying Replication Materials
Verification is now a regular component of thepublication workflow.
Author uploads replication files to the AJPS Dataversewhen final draft of the manuscript is submitted.
Odum or QDR staff verify that the full set of replication fileshave been provided and that they do reproduce all analyticresults from the AJPS article.
If problems occur, Odum or QDR staff inform the AJPSEditor, who then informs the author.
Author resolves the issues and resubmits replicationmaterials for rechecking by Odum or QDR staff.
Process continues until the replication is carried outsuccessfully.
Odum or QDR staff release Dataverse Dataset for publicaccess.
![Page 60: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/60.jpg)
Verifying Replication Materials
Verification is now a regular component of thepublication workflow.
Author uploads replication files to the AJPS Dataversewhen final draft of the manuscript is submitted.
Odum or QDR staff verify that the full set of replication fileshave been provided and that they do reproduce all analyticresults from the AJPS article.
If problems occur, Odum or QDR staff inform the AJPSEditor, who then informs the author.
Author resolves the issues and resubmits replicationmaterials for rechecking by Odum or QDR staff.
Process continues until the replication is carried outsuccessfully.
Odum or QDR staff release Dataverse Dataset for publicaccess.
![Page 61: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/61.jpg)
Verifying Replication Materials
Verification is now a regular component of thepublication workflow.
Author uploads replication files to the AJPS Dataversewhen final draft of the manuscript is submitted.
Odum or QDR staff verify that the full set of replication fileshave been provided and that they do reproduce all analyticresults from the AJPS article.
If problems occur, Odum or QDR staff inform the AJPSEditor, who then informs the author.
Author resolves the issues and resubmits replicationmaterials for rechecking by Odum or QDR staff.
Process continues until the replication is carried outsuccessfully.
Odum or QDR staff release Dataverse Dataset for publicaccess.
![Page 62: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/62.jpg)
Verifying Replication Materials
Articles that are verified successfully receive anotation to that effect in the Dataverse Dataset
Articles that are verified successfully qualify for “OpenPractices Badges” from the Center for Open Science:
Badges are displayed in the Dataverse Dataset and inthe published article
![Page 63: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/63.jpg)
Verifying Replication Materials
Articles that are verified successfully receive anotation to that effect in the Dataverse Dataset
Articles that are verified successfully qualify for “OpenPractices Badges” from the Center for Open Science:
Badges are displayed in the Dataverse Dataset and inthe published article
![Page 64: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/64.jpg)
Verifying Replication Materials
Articles that are verified successfully receive anotation to that effect in the Dataverse Dataset
Articles that are verified successfully qualify for “OpenPractices Badges” from the Center for Open Science:
Badges are displayed in the Dataverse Dataset and inthe published article
![Page 65: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/65.jpg)
Verifying Replication Materials
Articles that are verified successfully receive anotation to that effect in the Dataverse Dataset
Articles that are verified successfully qualify for “OpenPractices Badges” from the Center for Open Science:
Badges are displayed in the Dataverse Dataset and inthe published article
![Page 66: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/66.jpg)
Verifying Replication Materials
Articles that are verified successfully receive anotation to that effect in the Dataverse Dataset
Articles that are verified successfully qualify for “OpenPractices Badges” from the Center for Open Science:
Badges are displayed in the Dataverse Dataset and inthe published article
![Page 67: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/67.jpg)
How is the AJPS Policy Working?
The AJPS Dataverse was first announced in July 2012
The first Study (i.e., set of replication files for an AJPSarticle) was released for public use in September 2013
The AJPS Dataverse currently contains 235 Datasets(formerly called Studies)
The new AJPS Replication Policy was announced onMarch 26, 2015
As of June 15, 2016, Odum Institute staff have worked withreplication materials for 72 accepted manuscripts.
Of these, 59 have been released and are now available onthe AJPS Dataverse.
![Page 68: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/68.jpg)
How is the AJPS Policy Working?
The AJPS Dataverse was first announced in July 2012
The first Study (i.e., set of replication files for an AJPSarticle) was released for public use in September 2013
The AJPS Dataverse currently contains 235 Datasets(formerly called Studies)
The new AJPS Replication Policy was announced onMarch 26, 2015
As of June 15, 2016, Odum Institute staff have worked withreplication materials for 72 accepted manuscripts.
Of these, 59 have been released and are now available onthe AJPS Dataverse.
![Page 69: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/69.jpg)
How is the AJPS Policy Working?
The AJPS Dataverse was first announced in July 2012
The first Study (i.e., set of replication files for an AJPSarticle) was released for public use in September 2013
The AJPS Dataverse currently contains 235 Datasets(formerly called Studies)
The new AJPS Replication Policy was announced onMarch 26, 2015
As of June 15, 2016, Odum Institute staff have worked withreplication materials for 72 accepted manuscripts.
Of these, 59 have been released and are now available onthe AJPS Dataverse.
![Page 70: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/70.jpg)
How is the AJPS Policy Working?
The AJPS Dataverse was first announced in July 2012
The first Study (i.e., set of replication files for an AJPSarticle) was released for public use in September 2013
The AJPS Dataverse currently contains 235 Datasets(formerly called Studies)
The new AJPS Replication Policy was announced onMarch 26, 2015
As of June 15, 2016, Odum Institute staff have worked withreplication materials for 72 accepted manuscripts.
Of these, 59 have been released and are now available onthe AJPS Dataverse.
![Page 71: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/71.jpg)
How is the AJPS Policy Working?
The AJPS Dataverse was first announced in July 2012
The first Study (i.e., set of replication files for an AJPSarticle) was released for public use in September 2013
The AJPS Dataverse currently contains 235 Datasets(formerly called Studies)
The new AJPS Replication Policy was announced onMarch 26, 2015
As of June 15, 2016, Odum Institute staff have worked withreplication materials for 72 accepted manuscripts.
Of these, 59 have been released and are now available onthe AJPS Dataverse.
![Page 72: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/72.jpg)
How is the AJPS Policy Working?
The AJPS Dataverse was first announced in July 2012
The first Study (i.e., set of replication files for an AJPSarticle) was released for public use in September 2013
The AJPS Dataverse currently contains 235 Datasets(formerly called Studies)
The new AJPS Replication Policy was announced onMarch 26, 2015
As of June 15, 2016, Odum Institute staff have worked withreplication materials for 72 accepted manuscripts.
Of these, 59 have been released and are now available onthe AJPS Dataverse.
![Page 73: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/73.jpg)
How is the AJPS Policy Working?
The AJPS Dataverse was first announced in July 2012
The first Study (i.e., set of replication files for an AJPSarticle) was released for public use in September 2013
The AJPS Dataverse currently contains 235 Datasets(formerly called Studies)
The new AJPS Replication Policy was announced onMarch 26, 2015
As of June 15, 2016, Odum Institute staff have worked withreplication materials for 72 accepted manuscripts.
Of these, 59 have been released and are now available onthe AJPS Dataverse.
![Page 74: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/74.jpg)
How is the Replication Policy Working?
Initial replication files provided by authors usuallyneed additional work
Most authors need to correct and resubmit their replicationfiles one or more times
Mean number of resubmissions is 1.71
Only five cases (out of 68) in which replication filessuccessfully reproduced all analyses on the first try
Reasons for failure to verify usually relatively minor
Missing or inappropriately formatted documentation
Missing software commands for parts of analysis
Small differences in numeric or graphical results
![Page 75: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/75.jpg)
How is the Replication Policy Working?
Initial replication files provided by authors usuallyneed additional work
Most authors need to correct and resubmit their replicationfiles one or more times
Mean number of resubmissions is 1.71
Only five cases (out of 68) in which replication filessuccessfully reproduced all analyses on the first try
Reasons for failure to verify usually relatively minor
Missing or inappropriately formatted documentation
Missing software commands for parts of analysis
Small differences in numeric or graphical results
![Page 76: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/76.jpg)
How is the Replication Policy Working?
Initial replication files provided by authors usuallyneed additional work
Most authors need to correct and resubmit their replicationfiles one or more times
Mean number of resubmissions is 1.71
Only five cases (out of 68) in which replication filessuccessfully reproduced all analyses on the first try
Reasons for failure to verify usually relatively minor
Missing or inappropriately formatted documentation
Missing software commands for parts of analysis
Small differences in numeric or graphical results
![Page 77: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/77.jpg)
How is the Replication Policy Working?
Initial replication files provided by authors usuallyneed additional work
Most authors need to correct and resubmit their replicationfiles one or more times
Mean number of resubmissions is 1.71
Only five cases (out of 68) in which replication filessuccessfully reproduced all analyses on the first try
Reasons for failure to verify usually relatively minor
Missing or inappropriately formatted documentation
Missing software commands for parts of analysis
Small differences in numeric or graphical results
![Page 78: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/78.jpg)
How is the Replication Policy Working?
Initial replication files provided by authors usuallyneed additional work
Most authors need to correct and resubmit their replicationfiles one or more times
Mean number of resubmissions is 1.71
Only five cases (out of 68) in which replication filessuccessfully reproduced all analyses on the first try
Reasons for failure to verify usually relatively minor
Missing or inappropriately formatted documentation
Missing software commands for parts of analysis
Small differences in numeric or graphical results
![Page 79: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/79.jpg)
How is the Replication Policy Working?
Initial replication files provided by authors usuallyneed additional work
Most authors need to correct and resubmit their replicationfiles one or more times
Mean number of resubmissions is 1.71
Only five cases (out of 68) in which replication filessuccessfully reproduced all analyses on the first try
Reasons for failure to verify usually relatively minor
Missing or inappropriately formatted documentation
Missing software commands for parts of analysis
Small differences in numeric or graphical results
![Page 80: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/80.jpg)
How is the Replication Policy Working?
Initial replication files provided by authors usuallyneed additional work
Most authors need to correct and resubmit their replicationfiles one or more times
Mean number of resubmissions is 1.71
Only five cases (out of 68) in which replication filessuccessfully reproduced all analyses on the first try
Reasons for failure to verify usually relatively minor
Missing or inappropriately formatted documentation
Missing software commands for parts of analysis
Small differences in numeric or graphical results
![Page 81: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/81.jpg)
How is the Replication Policy Working?
Initial replication files provided by authors usuallyneed additional work
Most authors need to correct and resubmit their replicationfiles one or more times
Mean number of resubmissions is 1.71
Only five cases (out of 68) in which replication filessuccessfully reproduced all analyses on the first try
Reasons for failure to verify usually relatively minor
Missing or inappropriately formatted documentation
Missing software commands for parts of analysis
Small differences in numeric or graphical results
![Page 82: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/82.jpg)
How is the Replication Policy Working?
Initial replication files provided by authors usuallyneed additional work
Most authors need to correct and resubmit their replicationfiles one or more times
Mean number of resubmissions is 1.71
Only five cases (out of 68) in which replication filessuccessfully reproduced all analyses on the first try
Reasons for failure to verify usually relatively minor
Missing or inappropriately formatted documentation
Missing software commands for parts of analysis
Small differences in numeric or graphical results
![Page 83: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/83.jpg)
How is the Replication Policy Working?
Verification process does add some time to thepublication workflow
Number of days from arrival at the Odum Institute torelease of Dataverse Dataset ranges from 8 to 156
Mean length of verification process is 53.37 days andmedian is 44 days.
Part of the time in the verification process is due toauthor responses
Number of days from arrival at the Odum Institute to theinitial Verification Report ranges from 1 to 63
Mean time until initial Verification Report is 22.48 days, andthe median is 17 days
![Page 84: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/84.jpg)
How is the Replication Policy Working?
Verification process does add some time to thepublication workflow
Number of days from arrival at the Odum Institute torelease of Dataverse Dataset ranges from 8 to 156
Mean length of verification process is 53.37 days andmedian is 44 days.
Part of the time in the verification process is due toauthor responses
Number of days from arrival at the Odum Institute to theinitial Verification Report ranges from 1 to 63
Mean time until initial Verification Report is 22.48 days, andthe median is 17 days
![Page 85: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/85.jpg)
How is the Replication Policy Working?
Verification process does add some time to thepublication workflow
Number of days from arrival at the Odum Institute torelease of Dataverse Dataset ranges from 8 to 156
Mean length of verification process is 53.37 days andmedian is 44 days.
Part of the time in the verification process is due toauthor responses
Number of days from arrival at the Odum Institute to theinitial Verification Report ranges from 1 to 63
Mean time until initial Verification Report is 22.48 days, andthe median is 17 days
![Page 86: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/86.jpg)
How is the Replication Policy Working?
Verification process does add some time to thepublication workflow
Number of days from arrival at the Odum Institute torelease of Dataverse Dataset ranges from 8 to 156
Mean length of verification process is 53.37 days andmedian is 44 days.
Part of the time in the verification process is due toauthor responses
Number of days from arrival at the Odum Institute to theinitial Verification Report ranges from 1 to 63
Mean time until initial Verification Report is 22.48 days, andthe median is 17 days
![Page 87: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/87.jpg)
How is the Replication Policy Working?
Verification process does add some time to thepublication workflow
Number of days from arrival at the Odum Institute torelease of Dataverse Dataset ranges from 8 to 156
Mean length of verification process is 53.37 days andmedian is 44 days.
Part of the time in the verification process is due toauthor responses
Number of days from arrival at the Odum Institute to theinitial Verification Report ranges from 1 to 63
Mean time until initial Verification Report is 22.48 days, andthe median is 17 days
![Page 88: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/88.jpg)
How is the Replication Policy Working?
Verification process does add some time to thepublication workflow
Number of days from arrival at the Odum Institute torelease of Dataverse Dataset ranges from 8 to 156
Mean length of verification process is 53.37 days andmedian is 44 days.
Part of the time in the verification process is due toauthor responses
Number of days from arrival at the Odum Institute to theinitial Verification Report ranges from 1 to 63
Mean time until initial Verification Report is 22.48 days, andthe median is 17 days
![Page 89: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/89.jpg)
How is the Replication Policy Working?
Verification process does add some time to thepublication workflow
Number of days from arrival at the Odum Institute torelease of Dataverse Dataset ranges from 8 to 156
Mean length of verification process is 53.37 days andmedian is 44 days.
Part of the time in the verification process is due toauthor responses
Number of days from arrival at the Odum Institute to theinitial Verification Report ranges from 1 to 63
Mean time until initial Verification Report is 22.48 days, andthe median is 17 days
![Page 90: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/90.jpg)
Conclusions
The AJPS Replication and Verification Policy is onlypossible because of the Harvard Dataverse Network
The AJPS policy makes an important contribution tothe infrastructure of social science research.
Social sciences traditionally have paid lip service tothe standards of research transparency andreplicability of results
The AJPS policy requires us to “practice what wepreach” and adhere to standards in a meaningful way.
![Page 91: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/91.jpg)
Conclusions
The AJPS Replication and Verification Policy is onlypossible because of the Harvard Dataverse Network
The AJPS policy makes an important contribution tothe infrastructure of social science research.
Social sciences traditionally have paid lip service tothe standards of research transparency andreplicability of results
The AJPS policy requires us to “practice what wepreach” and adhere to standards in a meaningful way.
![Page 92: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/92.jpg)
Conclusions
The AJPS Replication and Verification Policy is onlypossible because of the Harvard Dataverse Network
The AJPS policy makes an important contribution tothe infrastructure of social science research.
Social sciences traditionally have paid lip service tothe standards of research transparency andreplicability of results
The AJPS policy requires us to “practice what wepreach” and adhere to standards in a meaningful way.
![Page 93: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/93.jpg)
Conclusions
The AJPS Replication and Verification Policy is onlypossible because of the Harvard Dataverse Network
The AJPS policy makes an important contribution tothe infrastructure of social science research.
Social sciences traditionally have paid lip service tothe standards of research transparency andreplicability of results
The AJPS policy requires us to “practice what wepreach” and adhere to standards in a meaningful way.
![Page 94: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/94.jpg)
Conclusions
The AJPS Replication and Verification Policy is onlypossible because of the Harvard Dataverse Network
The AJPS policy makes an important contribution tothe infrastructure of social science research.
Social sciences traditionally have paid lip service tothe standards of research transparency andreplicability of results
The AJPS policy requires us to “practice what wepreach” and adhere to standards in a meaningful way.
![Page 95: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/95.jpg)
Conclusions
Benefits of replication and verification process faroutweigh any costs to author, Editor, or publisher
Hopefully, the AJPS will serve as a “role model” in away that facilitates transparency and improves theintegrity of political science research more generally
The new AJPS replication standards should contributedirectly to a more rigorous, rational, theory-driven, andcumulative approach to understanding political andsocial phenomena.
![Page 96: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/96.jpg)
Conclusions
Benefits of replication and verification process faroutweigh any costs to author, Editor, or publisher
Hopefully, the AJPS will serve as a “role model” in away that facilitates transparency and improves theintegrity of political science research more generally
The new AJPS replication standards should contributedirectly to a more rigorous, rational, theory-driven, andcumulative approach to understanding political andsocial phenomena.
![Page 97: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/97.jpg)
Conclusions
Benefits of replication and verification process faroutweigh any costs to author, Editor, or publisher
Hopefully, the AJPS will serve as a “role model” in away that facilitates transparency and improves theintegrity of political science research more generally
The new AJPS replication standards should contributedirectly to a more rigorous, rational, theory-driven, andcumulative approach to understanding political andsocial phenomena.
![Page 98: The Replication and Verification Policy at the American ... · 7/11/2016 · in data access, research transparency and reproducibility of published research. AJPS is Taking a Leadership](https://reader033.vdocuments.site/reader033/viewer/2022051814/6039e98be724ba04d42111ec/html5/thumbnails/98.jpg)
Conclusions
Benefits of replication and verification process faroutweigh any costs to author, Editor, or publisher
Hopefully, the AJPS will serve as a “role model” in away that facilitates transparency and improves theintegrity of political science research more generally
The new AJPS replication standards should contributedirectly to a more rigorous, rational, theory-driven, andcumulative approach to understanding political andsocial phenomena.