1

Planning a Mixed Methods Study

by John W. Creswell, Ph.D.,Department of Educational Psychology, University of Nebraska-LincolnCo-editor, Journal of Mixed Methods Research, and Co-Director, Office of Qualitative and Mixed Methods Research

Presentation made to Social Work-Nursing Graduate Students, University of Michigan, February 5, 2008

© Please do not duplicate or use these slides without the express permission of the author.

2

“Products” of Planning

Dissertation or masters thesis Proposal for external funding

Planning the steps that will be undertaken in the research Focus on a dissertation or masters thesis

Let’s not get stuck – value in seeing the entire planning process

3

Our Agenda:

View typical contents for a proposal for mixed methods research Select steps in the process of planning a study Design a mixed methods study as a group using one example in a

“guided design” approach Assemble the final plan in order of standard presentation of a plan Individuals can plan their own studies as we go through “guided

design” Share the final plans at the end of the session

4

Outline of the Structure of a Proposal for a Dissertation or Thesis (Creswell & Plano Clark, 2007, p. 155) Title (foreshadows mixed methods study and type of design)

Introduction The research problem Past research on the problem Deficiencies in past research and one deficiency related to a need to collect both quantitative and qualitative data The audiences that will profit from the study

Purpose The purpose or study aim of the project (use the scripts) and reasons for mixed methods The research questions

Quantitative, qualitative, mixed methods research questions/hypotheses

Philosophical foundations for using mixed methods and the type of design (optional) Literature Review (include quantitative, qualitative, and mixed methods studies, if they are available)

Methods A definition of mixed methods research The type of design used, and its definition Challenges in using this design Examples of using the type of design (in your field, if possible) Reference and inclusion of a visual diagram Quantitative data collection and analysis Qualitative data collection and analysis Mixed methods data analysis procedures Validity approaches in both quantitative and qualitative research

Researcher’s Resources and Skills Potential Ethical Issues Timeline for Completing the Study References/Appendices with instruments/protocols

5

Components of a Plan that We will Design Title (foreshadows mixed methods study and type of design)

Introduction The research problem Past research on the problem Deficiencies in past research and one deficiency related to a need to collect both quantitative and qualitative data The audiences that will profit from the study

Purpose The purpose or study aim of the project and reasons for mixed methods The research questions/hypotheses

Quantitative, qualitative, mixed methods research questions/hypotheses (create separate sections for each set of questions

Philosophical foundations for using mixed methods and the type of design (optional) Literature Review (include quantitative, qualitative, and mixed methods studies, if they are available)

Methods A definition of mixed methods research The type of design used, and its definition Challenges in using this design Examples of using the type of design (in your field, if possible) Reference and inclusion of a visual diagram Quantitative data collection and analysis Qualitative data collection and analysis Mixed methods data analysis procedures Validity approaches in both quantitative and qualitative research

Researcher’s Resources and Skills Potential Ethical Issues Timeline for Completing the Study References/Appendices with instruments/protocols

6

Components That We Will Design (in order that we will address them )

Title The research problem –

Content intent Content issue(s)

Reasons for mixed methods Researcher’s resources and skills Definition of mixed methods Qualitative data collection and analysis Quantitative data collection and analysis Research questions (qualitative, quantitative, mixed) Type of design Visual diagram Philosophical foundation Purpose statement

7

Topic Writing the title

Short Identify participants and site Consider a neutral title, neither quantitative or qualitative in

wording Include the words “mixed methods”

Designing a title

8

Title

9

The Research Problem

Intent of the study What are you trying to accomplish in this study?

One sentence Narrow down to one overriding goal

Problem What content issue are you addressing?

Real life problem Deficiency in the literature Practice needs of audience All of the above

10

Intent of the Study

11

Research Problem (Issue)

12

Reasons for Mixed Methods

Why are you planning to use mixed methods in your study? What purpose (research objective) will it accomplish? Why do you plan on gathering both quan and qual data? Think about this from a research methods perspective.

13

Reasons for Conducting a Mixed Methods Study (Bryman, Qualitative Research, 2006)

Validity – to corroborate quan and qual data Offset – offset weaknesses of quan and qual and draw on strengths Completeness – more comprehensive account that qual/qual alone Process – quan provides outcomes; qual, the processes Different question – quan and qual answer different questions Explanation – qual can explain quan results or vice-versa Unexpected results – surprising results from one, other explains Instr development – qual employed to design instr, then instr tested Sampling – one approach facilitates sampling from other approach Credibility – both approaches enhance integrity of findings Context – qual provides context; quan provides general. Illustration – qual data helps develop “depth” for quan data Utility – more useful to practitioners Confirm – quan tests qual generated hypotheses Diversity of views – relationship and meaning; researcher/participant views Enhancement – augmenting or building on one form of data with the other

14

Reasons for Conducting Mixed Methods

15

Research Skills and Resources

Are you qualified to conduct a mixed methods study? Quantitative coursework Qualitative coursework Personal experience conducting a mixed methods study Support from your committee/support from your field Time and resources

16

Research Skills and Resources

17

Definition of Mixed Methods

How would you describe this approach? Focus on methods? Focus on methodology? Focus on paradigm/worldview? Focus on application to traditional design?

Aspects I would include: Definition Citations to studies that have employed it Major characteristics of the design

18

Methodology – Definition of Mixed Methods Research

Mixed methods research is a design for collecting, analyzing, and mixing both quantitative and qualitative research (or data) in a single study or series of studies to understand a research problem. (adapted from Creswell and Plano Clark, 2007)

19

Sites/Individuals to be Studied

Site Quantitative sample Qualitative sample

Sampling Quantitative sample Qualitative sample

20

Types of Quantitative and Qualitative Data to be Collected

Quantitative data Instruments Checklists Records

Qualitative data Interviews Observations Documents Audio-visual materials

21

Data Collection

Site to be studied

Participants to be studied

Forms of data collection Quan Qual

22

Quantitative and Qualitative Data Analysis

Qualitative analysis Use text and

images, For coding For theme

development For relating themes

Quantitative analysis Use statistical

analysis, For description For comparing

groups For relating variables

23

Data Analysis

Qualitative Quantitative

24

Research Questions

Quantitative questions Describing variables Relating variables Comparing variables Think in terms of i.v., d.v. at least (narrowed, closed-ended) Theory

Qualitative questions Exploring a single phenomenon General, open-ended

Mixed methods questions What will you learn by “mixing” Combine quan and qual and what you hope to achieve by it (reasons)

25

Research Questions

Quantitative questions

Qualitative questions

Mixed method question

26

Type of Mixed Methods Design

Will the quan and qual data be collected: Timing

Sequentially? – one builds on the other Concurrently? – both are collected at the same time

Emphasis Quan emphasized? Qual emphasized?

Mixing Merging? Connecting? Embedding?

27

QUANData & Results

QUANData & Results InterpretationInterpretation

QUALData & Results

QUALData & Results

QUANPre-test Data & Results

QUANPre-test Data & Results

QUANPost-test

Data & Results

QUANPost-test

Data & Results

Intervention

qualProcess

qualProcess

InterpretationInterpretation

Triangulation Design

Embedded Design

Concurrent Mixed Methods Designs

28

Sequential Designs Mixed Methods Designs

QUANData & Results

QUANData & Results

InterpretationInterpretation

qualData & Results

qualData & Results

Following up

QUALData & Results

QUALData & Results

quanData & Results

quanData & Results InterpretationInterpretation

Building to

Before-intervention

qual

Before-intervention

qual

QUANIntervention

Trial

QUANIntervention

Trial

After-

intervention qual

After-

intervention qual InterpretationInterpretation

Exploratory Design

Explanatory Design

Sequential Embedded Design

29

PostpositivismDeterminationReductionismEmpirical observation and measurementTheory verification

ConstructivismUnderstandingMultiple participant meaningsSocial and historical constructionTheory generation

Advocacy/ParticipatoryPoliticalEmpowerment issue-orientedCollaborativeChange-oriented

PragmatismConsequences of actionsProblem-centeredPluralisticReal-world practice oriented

Worldview or Paradigm for Study

30

Purpose Statement

Most important statement in your project plan Follow one of my scripts to get started (can adapt later) Elements of scripts:

Intent (from earlier) Include qualitative purpose statement (central phenomenon, data

collection, participants, site) Include quantitative purpose statement (variables, data collection,

participants, site) Reason for a mixed methods study

31

Explanatory Sequential QUAN

The intent of this two-phase, sequential mixed methods study will be to ______________________ (mention content objective of the study). In the first phase, quantitative research questions or hypotheses will address the ___________ relationship or comparison of ___________ (independent) and ___________ (dependent) variables with ___________ (participants) at ______________ (the research site). Information from this first phase will be explored further in a second qualitative phase. In the second phase, qualitative interviews or observations will be used to probe significant _______________ (quantitative results) by exploring aspects of the _____________ (central phenomenon) with ______________ (a few participants) at ____________ (research site). The reason for following up with qualitative research in the second phase is to __________________ (e.g., better understand and explain the quantitative results).

qual

32

Exploratory Sequential QUAL

The intent of this two-phase, sequential mixed methods study will be to _______________ (mention content objective of the study). The first phase will be a qualitative exploration of a ____________ (central phenomenon) by collecting ___________ (types of data) from __________ (participants) at ___________ (research site). Findings from this qualitative phase will then be used to test ________ (a theory, research questions, or hypotheses) that ____________ (relate, compare) ____________ (independent variable) with _____________ (dependent variable) for _________ (sample of population) at ___________ (research site). The reason for collecting qualitative data initially is that ______________ (e.g., instruments are inadequate or not available, variables are not known, there is little guiding theory, or few taxonomies).

quan

33

Concurrent QUAL + QUAN

The intent of this concurrent mixed methods study is to __________________ (content objective of the study). In the study, ___________ (quantitative instruments) will be used to measure the relationship between _____________ (independent variable) and ____________ (dependent variable). At the same time, the _____________ (central phenomenon) will be explored using _______________ (qualitative interviews or observations) with ___________ (participants) at _______________ (the research site). The reason for combining both quantitative and qualitative data is to better understand a research problem by converging both quantitative (broad numeric trends) and qualitative (detailed views) data.

34

Embedded QUAN(qual) or QUAL(quan)

This mixed methods study will address _________________ (overall content aim of the study). An embedded mixed method design will be used, and it is a design in which one data set provides a supportive, secondary role in a study based primarily on the other data set. The primary purpose of this study will use ________________ (quantitative instruments) to test the theory of _____________ (the theory) that predicts that __________________ (independent variables) will influence ________________ (positively, negatively) the __________ (dependent variables or outcomes) for ____________ (participants) at __________ (the research site). A secondary purpose will be to gather qualitative data _______________ (type of qualitative data, such as interviews) that will explore ____________ (the central phenomenon) for _________________ (participants) at _____________(site). The reason for collecting the secondary database is ________________ (e.g., to address different question, to provide support for the primary purpose).

35

Planning a Mixed Methods Study

by John W. Creswell, Ph.D.,Department of Educational Psychology, University of Nebraska-LincolnCo-editor, Journal of Mixed Methods Research, and Co-Director, Office of Qualitative and Mixed Methods Research

Presentation made to Social Work-Nursing Graduate Students, University of Michigan, February 5, 2008

© Please do not duplicate or use these slides without the express permission of the author.