Mixed Methods WorkshopUNC -- The Odum Institute

February 12-13, 2015

Lisa Pearce (ldpearce@unc.edu)

Chitwan Valley Family Study (CVFS), 1995-2015perl.psc.isr.umich.edu

• Survey methods• Hybrid calendar methods• Environmental measures• Observation• Semi-structured interview methods

National Study of Youth and Religion (NSYR), 2002-2013youthandreligion.org

WAVE 1 (2002/3: Ages 13-17)

• RDD Telephone Survey, National Sample (n=3290)

• In-Person, Semi-Structured Interviews

WAVE 2 (2005: Ages 16-21)

WAVE 3 (2007/8: Ages 18-24)

WAVE 4 (2012/3: Ages 23-30)

• Follow-up Telephone Survey

• Follow-up In-Person, Semi-Structured Interviews

Workshop Plan

• Defining Mixed Methods Research• Standard Definitions• Paradigms• Mixed Research Approach

• MMR & The Research Process• Research Questions• Research Design & Data Collection• Data Analysis & Interpretation• Presentation of Results• Publishing• Research Funding

• Being a Mixed Methods Researcher

Standard Definitions (Johnson

et al. 2007)

• Burke Johnson and Anthony Onwuegbuzie:• Mixed methods research is the class of research where the researcher

mixes or combines quantitative and qualitative research techniques, methods, approaches, concepts or language into a single study or set of related studies.

• Jennifer Greene:• Mixed method inquiry is an approach to investigating the social world

that ideally involves more than one methodological tradition and thus more than one way of knowing, along with more than one kind of technique for gathering, analyzing, and representing human phenomena, all for the purpose of better understanding.

(Johnson et al. 2007)

Paradigms

“Qualitative”Research

“Quantitative”Research

OntologyWhat can we know?

EpistemologyHow do we know it?

AxiologyThe role of values

MethodologyTechnical approaches

The Use of Methods Through Time (Pearce 2012)

1900s 1930s 1950s 1960s -70s-80s 2000s

ArchivalLife HistoryObservation

Sample Surveys/Quantitative/Positivist

Qualitative/Interpretive/

Constructivist

Pragmatism/Mixed

Methods

Thomas & ZnanieckiE. Franklin Frazier

Middletown Studies

Union Democracy Boys in White

Triangulation

TABLE 2 A Pragmatic Alternative to the Key Issues

in Social Science Research Methodology

Qualitative Approach

Quantitative Approach

Pragmatic Approach

Connection of Theory and Data

Relationship to Research Process

Inference from Data

Induction

Subjectivity

Context

Deduction

Objectivity

Generality

Abduction

Intersubjectivity

Transferability

Morgan (2007)

Mixed Research

Philosophical PositionsEpistemology•Subjectivity•Objectivity• Intersubjectivity

Ontology•Relativism•Absolutism•Critical Realism

Axiology•Value-Bound•Value-Free•Value-Conscious

Practical Positions

Logic•Research Problem•Research Question/s•Project Design

Locations on Axes of Inquiry•Subjective to Objective• Inductive to Deductive• Idiographic to Nomothetic•High to Low Reflexivity

Methods•Data Collection•Analysis• Interpretation

Figure 1. Facets of Social Inquiry (Pearce Forthcoming)

• Formulate research problem/objective• Develop purpose/question/hypothesis• Select a research design/method• Collect data• Analyze data• Interpret/validate data• Communicate findings

Research Process (Onwuegbuzie & Leech 2005)

Mixed Methods Research

• …is that which integrates the collection and/or analysis of two or more types of data to answer a set of interrelated research questions

Principles Behind Good Mixed Methods Research (Axinn & Pearce 2006)

• Method counterbalance• Strengths and weaknesses• Investigator involvement• Flexibility in design and application

• Comprehensive empirical record• Multiple kinds of evidence

• Reproduce findings with multiple methods• Discover mechanisms• Highlight temporal ordering

MIXED METHODS RESEARCH & THE RESEARCH PROCESSResearch Questions--Research Design & Data Collection--Data Analysis & Interpretation--Presentation of Results—Publishing--Research Funding

Research Questions (Alford 1998)

Research Problem

RESEARCH QUESTION/S

Theoretical

Empirical

Action Agenda

Mixed Methods Research Questions(Clark & Badiee 2010)

• Level of Integration• Separate questions• Broad general questions• Hybrid question• Procedural/mixing question• Combination

• Relationship Between Questions• Independent/Dependent

• Relationship of Questions to the Process• Predetermined/Emergent

Surveys Semi-Structured Interviews Focus Groups

ObservationHistorical/Archival/

MaterialsExperimental

Components of Research Design & Data Collection

Figure 1. Comparison of Structure, Interviewer Involvement, and Researcher Involvement among Data Collection Methods (Axinn & Pearce 2006)

Data Collection Method Level of Structure

Interviewer Involvement

Researcher Involvement with Study Population

Surveys High Usually Low

Semi-Structured Interviews

Low Always High

Focus Groups Low Always Medium

Observation Low Usually High

Historical or Archival Methods

Out of researcher’s

control

Out of researcher’s

controlLow

Figure 2. Comparing Sample Size and Coding among Data Collection Methods (Axinn & Pearce 2006)

Data Collection Method

Data on Large Numbers of People Could Be Coded As

NumbersCould Be Analyzed As

Text

Surveys Usual approach X X

Semi-Structured Interviews

Possible X X

Focus Groups Possible X X

Observation Possible X X

Historical or Archival Methods

Possible X X

Features of Design

• Method balance• Equal or unequal

• Method order• Sequential• Parallel/simultaneous/concurrent

• Method integration• Data collection

• Sampling (Teddlie & Yu 2007)• Choice of methods• Design of instruments

• Data analysis, inference, and interpretation• One set of findings guides another analysis• Two sets of findings compared/contrasted

• Multiple points of integration

Focus Groups Survey RESULTS

Less structured methods used to help develop survey measures and

instruments

EXAMPLE:Zeller, Richard A. 1993. “Combining Qualitative and Quantitative Techniques to Develop Culturally Sensitive Measures.” in David G. Ostrow and Ronald C. Kessler (Eds.) Methodological Issues in AIDS Behavioral Research. New York: Plenum Press, pp. 95-116.

SurveySemi-Structured

InterviewsRESULTS

Survey methods used to set the stage for less

structured investigation

EXAMPLE:David J. Harding. 2009. “Violence, Older Peers, and the Socialization of Adolescent Boys in Disadvantaged Neighborhoods.” American Sociological Review 74(3): 445-464.

Semi-StructuredInterviews &Observations

Survey RESULTS

Survey data used to test findings from a less

structured method

EXAMPLE:Small ML, Jacobs EM, and Massengill RP. 2008. "Why Organizational Ties Matter for Neighborhood Effects: A Study of Resource Access through Childcare Centers." Social Forces. 87(1).

Experiment/Audit Study

Observation &Fieldnotes

Results

Less structured investigation used to help

explain quantitative findings

EXAMPLE:Pager, Devah, Bruce Western, and Bart Bonikowski. 2009. “Discrimination in a Low-Wage Labor Market: A Field Experiment.” American Sociological Review 74: 777-799.

Semi-StructuredInterviews with

Children & Mothers

Survey of Teachers

RESULTS

Parallel use and equal balance of methods

EXAMPLE: Timberlake (1994) from Tashakori & Teddlie (1998)

Systematic Anomalous Case Analysis (Pearce 2002)

• QUANT• Theory-informed survey analyses• Regression diagnostics to identify anomalous (and matched) cases• Sampling

• QUAL• Fieldwork• Findings spark new & revised hypotheses

• QUANT• Additional survey analyses

• Latent Class Analysis • Types of Religiosity

• Predicted by survey data• Using semi-structured interviews to confirm and/or flesh out

types

Typology Development(Pearce & Denton 2011)

Five Latent Classes of Religiosity

Hybrid Methods

• Event history calendars• Axinn & Pearce 2006

• Life histories from survey data & records• Singer, Burton H., Carol D. Ryff, Deborah Carr, and William

Magee. 1998. “Linking Life Histories and Mental Health: A Person-Centered Strategy.” Sociological Methodology 28:1-51.

• Q-sort methods• Shemmings, D. 2006. '"Quantifying” qualitative data: an

illustrative example of the use of Q methodology in psychosocial research’. Qualitative Research in Psychology 3 (2). pp. 147-165.

• Grid design• Columns = time periods (years, months, etc)• Rows = domains of interest

• Visual, cooperative process• Allows for memory cues• Allows flexibility

• Question ordering• Probing

Calendar Methods

Life History Calendar

Neighborhood History Calendar

Questionnaire Calendar

• Gives a visual structure to interviewing• A tool for reflection

Using LHCs in SS Interviews