Working with Qualitative Data

Christine Maidl PribbenowWisconsin Center for Education Research

cmpribbenow@wisc.edu

Session Outline

• General discussion about educational research, assumptions and misconceptions

• Contrast educational research with research in the sciences

• Define common qualitative analysis terms• Provide example using ATLAS.ti–

qualitative analysis software program • Code some text

Free Association…

DATA

QUALITATIVE

Qualitative Data: Oxymoron or inherent tensions?

• Hard vs. soft (mushy)

• Rigor

• Validity and reliability

• Objective vs. subjective

• Numbers vs. text

• What is The Truth?

What are some of the assumptions that you have about educational

research?

How are they helping or hindering the development of your study?

Research in the sciences vs. research in education

• “Soft” knowledge• Findings based in specific

contexts• Difficult to replicate• Cannot make causal claims

due to willful human action• Short-term effort of

intellectual accumulation– “village huts”

• Oriented toward practical application in specific contexts

• “Hard” knowledge• Produce findings that are

replicable • Validated and accepted as

definitive (i.e., what we know)

• Knowledge builds upon itself– “skyscrapers of knowledge”

• Oriented toward the construction and refinement of theory

Strongly Agree

Agree Unsure Disagree Strongly Disagree

Educational research is rigorous. 4 (31 %) 8 (62 %) 1 (8 %) 0 (0 %) 0 (0 %)

I have read at least ten articles

published in educational research

journals before attending this Institute.

11 (85 %) 2 (15 %) 0 (0 %) 0 (0 %) 0 (0 %)

Educational research is more difficult

than my scientific research.

1 (8 %) 2 (15 %) 6 (46 %) 3 (23 %) 1 (8 %)

I regularly collect qualitative data

in my classes for assessment purposes.

1 (8 %) 5 (38 %) 2 (15 %) 4 (31 %) 1 (8 %)

I need a control or comparison group

to conduct educational research.

2 (15 %) 3 (23 %) 1 (8 %) 5 (38 %) 2 (15 %)

Assessment data gleaned from students

(i.e., "self report") are valuable.

2 (15 %) 9 (69 %) 2 (15 %) 0 (0 %) 0 (0 %)

I have analyzed qualitative data in the past.

1 (8 %) 3 (23 %) 1 (8 %) 4 (31 %) 4 (31 %)

Strongly Agree

Agree Unsure Disagree Strongly Disagree

Qualitative data can meet "reliability" standards.

2 (15 %) 5 (38 %) 6 (46 %) 0 (0 %) 0 (0 %)

Qualitative data can meet "validity" standards.

2 (15 %) 5 (38 %) 6 (46 %) 0 (0 %) 0 (0 %)

If I collect learning assessment data from my students and the analyzed results are "not significant" it proves that students did not learn what I intended.

0 (0 %) 0 (0 %) 1 (8 %) 5 (38 %) 7 (54 %)

If I conduct classroom research and the results are "not significant",

the study was a waste of my time.

0 (0 %) 0 (0 %) 1 (8 %) 3 (23 %) 9 (69 %)

I need human subjects approval to

conduct and publish research

about my students.

7 (54 %) 1 (8 %) 5 (38 %) 0 (0 %) 0 (0 %)

I want to conduct research

in my classroom so that I can teach better.

11 (85 %) 2 (15 %) 0 (0 %) 0 (0 %) 0 (0 %)

I want to conduct research in my classroom so that my students learn more or better.

13 (100 %) 0 (0 %) 0 (0 %) 0 (0 %) 0 (0 %)

What are some sources of qualitative data?

• Lab notebooks

• Open-ended exam questions

• Papers

• Journal entries

• On-line discussions

• Email

• Notes from observations

Qualitative Data Analysis

Qualitative analysis is the

“interplay between researchers and data.”

Researcher and analysis are

“inextricably linked.”

Qualitative Data Analysis

• Inductive process– Grounded Theory

• Unsure of what you’re looking for, what you’ll find• No assumptions• No literature review at the beginning• Constant comparative method

• Deductive process– Theory driven

• Know the categories or themes using rubric, taxonomy• Looking for confirming and disconfirming evidence• Question and analysis informed by the literature

Example Research Questions

Why do faculty leave UW-Madison?

Do UW-Madison faculty leave

due to climate issues?

Definitions: Coding and Themes

• Coding process: – Conceptualizing, reducing, elaborating and

relating text– words, phrases, sentences, paragraphs.

• Building themes:– Codes are categorized thematically to describe

or explain phenomenon.

Let’s Code #1

Read through the reflection paper written by the student from an Ecology class and highlight words, parts of sentences, and/or whole sentences with some “code” attached and identified to those sections.

What did you highlight?

Why?

Let’s Code #2

Read through this reflection paper and code based on this question:

What were the student’s assumptions or misconceptions before taking this course?

What did you highlight?

Why?

Let’s Code #3

Read through this reflection paper and code based on this question:

What did the student learn in the course?

What did you highlight?

Why?

Can we say that the students learned something in the course

using reflection papers?

Why or why not?

Ensuring “validity” and “reliability” in your research

• Use mixed methods, multiple sources.• Triangulate your data whenever possible.• Ask others to review your design methodology,

observations, data, analysis, and interpretations (e.g., inter-rater reliability).

• Rely on your study participants to “member check” your findings.

• Note limitations of your study whenever possible.

C oncept m aps o f contentfound in journa l a rticles (Both)

P re-post examof concepts (Q uantita tive)

R eflection Paper(Q ualita tive)

3 Sourcesof D ata

Does the redesign of an ecology course to include concept maps derived from

current journal articles help students to gain a more current and realistic view of

ecological issues?

Questions?

The plural of anecdote is data.

-Donna Shalala