introduction - cross-cultural survey guidelines · cross-cultural survey guidelines ... do not...

Cross-Cultural Survey Guidelines

© Copyright 2016 Do not distribute or reprint without permission

Introduction i Revised August 2016

Introduction

The number and scope of surveys covering many cultures, languages, nations, or regions have increased significantly over the past decade. This has led to a growing need to provide information on best practices across the multiple phases of multinational, multicultural, or multiregional (“3MC”) survey design and administration to ensure the collection of high quality comparative data. However, there is very little published information on the details of implementing surveys that is specifically designed for comparative research. For example, little has been published on what aspects of 3MC surveys need to be standardized and when local adaptation is appropriate. The aim of the Comparative Survey Design and Implementation (CSDI) Guidelines Initiative was to develop and promote internationally recognized guidelines that highlight best practice for the conduct of comparative survey research across cultures and countries. The intended audience is researchers and survey practitioners planning or engaged in 3MC research. However, we believe that the Guidelines also could benefit researchers and survey practitioners involved in single country surveys.

The goal of the CSDI Initiative has been to develop Cross-Cultural Survey Guidelines (CCSG) as presented here, which cover all aspects of the survey lifecycle. This currently has resulted 18 chapters and 11 sub-chapters. Three additional chapters on study design and organizational structure, survey quality, and ethical considerations are relevant to all processes throughout the survey production lifecycle. Survey quality can be assessed in terms of fitness for intended use, total survey error, and survey production process quality monitoring. This may be affected by survey infrastructure, costs, interviewer and respondent burden, as well as study design specifications. Figure 1 presents a diagram of the survey lifecycle. The 18 chapters and 11 sub-chapters of the CCSG Guidelines are:

Study Design and Organizational Structure

Study Management

Tenders, Bids, & Contracts

Sample Design

Questionnaire Design

Instrument Technical Design

Translation

Overview

Managing and Budgeting

Team

Scheduling

Shared Language Harmonization

Assessment

Tools

Adaptation



Introduction ii Revised August 2016

Pretesting

Interviewer Recruitment, Selection, and Training

Data Collection

General Consideration

Face-to-Face Surveys

Telephone Surveys

Self-Administered Surveys

Paradata and Other Auxiliary Data

Data Harmonization

Data Processing & Statistical Adjustment

Data Dissemination

Statistical Analysis

Survey Quality

Ethical Considerations

Figure 1. The Survey Lifecycle



Introduction iii Revised August 2016

The CCSG Guidelines draw upon and are based on: (1) general good practice survey methodology, as well as cross-cultural and comparative literature on survey methodology; (2) available study-specific manuals and documentation; and (3) the experiences and lessons learned that authors, reviewers, and editors have added through their work on and with numerous comparative surveys. Best practices are dynamic and can be expected to evolve over time. At the present time, the Guidelines relate to not just cross-sectional surveys of households and individuals but also computer-assisted personal interviewing modes and the usage of paradata and statistical analyses. At a later point in time, they may be expanded to include establishment and longitudinal surveys. As more documentation and information about comparative surveys become available, we hope to incorporate the lessons learned from these studies into the CCSG Guidelines. New methodological research will also inform new versions of the CCSG Guidelines. You can greatly help us in these objectives by providing comments and suggestions, or simply alerting us about a topic we need to address. Please contact us at: [email protected]. Citations: Please cite these Guidelines as follows: Survey Research Center. (2016). Guidelines for Best Practice in Cross-Cultural Surveys. Ann Arbor, MI: Survey Research Center, Institute for Social Research, University of Michigan. Retrieved Month, dd, yyyy, from http://www.ccsg.isr.umich.edu/. The CCSG initiative is led by Beth-Ellen Pennell, currently the director of international survey operations at the Survey Research Center, Institute for Social Research at the University of Michigan. Also instrumental in the development and operationalization of the guidelines are Kirsten Alcser and Sue Ellen Hansen of Survey Research Operations, Survey Research Center, Institute for Social Research. The guidelines were initiated at the 2005 meeting of CSDI and have involved more than 70 individuals from more than 35 organizations worldwide. We dedicate these guidelines to Dr. Janet A. Harkness. Dr. Harkness passed away in 2012. She initiated the International Workshop on Comparative Survey Design and Implementation where the development of these Guidelines was launched. Dr. Harkness not only contributed to the overall framework and content of the guidelines but she also authored three of the original key chapters: Questionnaire Design, Adaptation and Translation. She inspired this work through her steadfast conviction that resources must be made available to researchers and survey practitioners if we are to improve comparative survey research methods, dissemination and analysis.

http://www.ccsg.isr.umich.edu/



Introduction iv Revised August 2016

Photo images: Some photo images that appear on this Website are stock photos. Others were provided by the Population and Ecology Laboratory in Nepal (Institute for Social Research, University of Michigan), Professor Jintao Xu at the College of Environmental Sciences and Engineering at Peking University, and Yu-chieh (Jay) Lin and the Institute of Social Science Survey at Peking University.



Introduction 5 Revised August 2016

Contributions to these guidelines have been made by:

Authors

Name Organization

Kirsten Alcser Survey Research Center, University of Michigan (UM-SRC)

Christopher Antoun UM-SRC; now Center for Survey Measurement, U.S. Census Bureau

Dorothée Behr GESIS – Leibniz Institute for the Social Sciences, Survey Design and Methodology

Grant Benson UM-SRC

Ipek Bilgen Survey Research and Methodology Program, University of Nebraska-Lincoln, and UNL Gallup Research Center (UNL-SRAM/GRC); now NORC at the University of Chicago

Emily Blasczyk UM-SRC

Ashley Bowers UM-SRC; now Center for Survey Research, Indiana University Bloomington (IN-CSR)

Margaret Brill UM-SRC; now Bottom Line

Rachel Caspar Research Triangle Institute (RTI) International

AnaLucía Córdova Cazar

UNL-SRAM/GRC

Judi Clemens UM-SRC

Julie de Jong UM-SRC

Brita Dorer GESIS - Leibniz Institute for the Social Sciences, Survey Design and Methodology

Benjamin Duffey UM-SRC; now Orbitz Worldwide

Jennifer Durow UM-SRC

Peter Granda Inter-University Consortium for Political and Social Research, University of Michigan (UM-ICPSR)

Heidi Guyer UM-SRC

Sue Ellen Hansen UM-SRC

Janet Harkness UNL-SRAM/GRC

Kristen Cibelli Hibben UM-SRC

Lisa Holland UM-SRC

Mengyao Hu UM-SRC

Lei Huang UNL-SRAM/GRC

Frost Hubbard UM-SRC; now IMPAQ International

Jennifer Kelley UM-SRC

Hyun Jung Lee UM-SRC

Sunghee Lee UM-SRC

Mingnan Liu Survey Monkey

Rachel Levenstein UM-SRC; now American Institutes for Research

Christina Lien UM-SRC; now Consulting

Yu-chieh (Jay) Lin UM-SRC

An Liu UNL-SRAM/GRC; now GfK

Debbie Miller UNL-SRAM/GRC; now Mixed Methods Consulting

Peter Mohler University of Mannheim

Rachel A. Orlowski UM-SRC

Beth-Ellen Pennell UM-SRC

Emilia Peytcheva RTI International

Lesli Scott UM-SRC

Mathew Stange UNL-SRAM/GRC; now Mathematica Policy Research

Ana Villar UNL-SRAM/GRC; now City University

Ting Yan Westat




Name Organization

Dan Zahs UM-SRC

Editors Name Organization Version

Julie de Jong UM-SRC 2016

Kristen Cibelli Hibben UM-SRC 2016

Lars Lyberg Statistics Sweden; University of Stockholm, now INIZIO

2010

Zeina Mneimneh UM-SRC 2008, 2010

Peter Mohler COMPASS and University of Mannheim 2010

Tom W. Smith NORC, University of Chicago 2016

Reviewers

Name Organization Version

Dorothée Behr GESIS – Leibniz Institute for the Social Sciences, Survey Design and Methodology

2008

Grant Benson UM-SRC 2008, 2010

Bill Blyth TNS-Global 2008

Pam Campanelli Independent consultant 2008

Somnath Chatterji World Health Organization, Multi-Country Studies Measurement and Health Information

2008

Rory Fitzgerald City University, Centre for Comparative Social Surveys

2008

Steven Heeringa UM-SRC 2008

Timothy P. Johnson Survey Research Laboratory, University of Illinois at Chicago

2016

Achim Koch GESIS – Leibniz Institute for the Social Sciences, Survey Design and Methodology

2008

Frauke Kreuter University of Maryland, Joint Program in Survey Methodology

2008

Paul Kussmaul Retired, formerly Johannes Gutenberg University in Mainz, Germany at Germersheim

2008

Kristin Miller National Center for Health Statistics 2008

Meinhard Moschner GESIS – Leibniz Institute for the Social Sciences 2008

José L. Padilla University of Granada, Social Psychology and Methodology

2008

Alisú Schoua-Glusberg Research Support Services 2008

Eleanor Singer UM-SRC 2008

Tom W. Smith NORC, University of Chicago 2008

Jaré Struwig Human Sciences Research Council, South Africa 2008

Rick Valliant University of Maryland, Joint Program in Survey Methodology; UM-SRC

2008

Gordon Willis National Cancer Institute 2008

Christine Wilson Heriot-Watt University, School of Management and Languages

2008

Christof Wolf GESIS – Leibniz Institute for the Social Sciences 2008

Formatting & Copy-Editing Name Organization Version

Gail Arnold UM-SRC 2008, 2010, 2016




Judi Clemens UM-SRC 2010, 2011, 2016

Shaw Hubbard Independent consultant 2008

Graphic Design Name Organization Version

Larry LaFerte UM-SRC 2008, 2016

Ruth Shamraj UM-ICPSR 2008

Website Programming Name Organization Version

Jamal Ali UM-SRC 2016

Tricia Blanchard UM-SRC 2008, 2010, 2011

Deborah Wilson UM-SRC 2008, 2010

introduction - cross-cultural survey guidelines · cross-cultural survey guidelines ... do not...

Documents