qualitative analysis educational skills workshop 2011-2012 arianne teherani, phd bridget o’brien,...
TRANSCRIPT
Qualitative Analysis
Educational Skills Workshop
2011-2012
Arianne Teherani, PhD
Bridget O’Brien, PhD
Creative Commons License
Attribution-NonCommercial-Share Alike 3.0 Unported
You are free:• to Share — to copy, distribute and transmit the work • to Remix — to adapt the work Under the following conditions:• Attribution. You must give the original authors credit (but not in any
way that suggests that they endorse you or your use of the work).• Noncommercial. You may not use this work for commercial purposes. • Share Alike. If you alter, transform, or build upon this work, you may
distribute the resulting work only under a license identical to this one.
See http://creativecommons.org/licenses/by-nc-sa/3.0/ for full license.
Overview
Evaluating qualitative research
Overview of qualitative research
Qualitative data collection methods
Analyzing qualitative data
Combining qualitative and quantitative
Exercise 1: Judging Qualitative Research (Analyses)Do you have a clear sense of the context of the
study and the role of the researcher(s)?
How well do you feel the methodology used in this study captured the research question? Explain some of the strengths and/or limitations.
Are there sufficient examples / paraphrasing to support the findings / argument?
Has the author convinced you that his/her interpretation is plausible? How?
Has the author discussed alternate interpretations?
Qualitative Research
Research about lives, behavior, organizational functioning, interactional relationships
Exploratory
Open-ended
Data = Words
Analytic procedures are interpretivist and concurrent with and post data collection
Research
Multiple Purposes of Qualitative Methods
Curriculum Development
Instrument Development
Evaluation
Qualitative Research Questions
• What are the qualities of an ideal mentor?
• How are the professional identities of students shaped during clerkship X?
• How do clerkship students characterize the struggles they face when transitioning to the clerkships? How do clinical teachers characterize students’ struggles?
• How does participation in a teamwork & communication skills curriculum improve small group process in Year 1 of medical school?
Qualitative Paradigm
Sets of beliefs that guide action
Qualitative research takes place within a vast spectrum of paradigms:
–Positivism
–Post-positivism
–Constructivism
–Critical Theory
–Post-modernismSee: Bunniss & Kelly. Research paradigms in medical
education. Medical Education, 2010, 44: 358-366.
Qualitative Approaches
Explore social processes through interpretation of data
Three Most Common in Medical Education:
• Ethnography
• Grounded theory
• Case study
• Phenomenology
• Hermeneutics
• Narrative research
• Action research
Sampling
• Purposeful
– Seeking the best sources of information about the phenomenon of interest
• Theoretical
– Seeking the best sources of information to confirm / disconfirm your developing explanatory model
“Participants are not recruited on a representative basis, but rather because of their expert knowledge of the phenomenon under inquiry” (Green & Thorogood, 2005)
Qualitative Data Collection
Observation
Interview
Focus groups
Open-ended prompts
Artifacts (documents, photos)
Observation
Spend time with a group of people as they carry out their daily activities, to understand their way of life and how they make sense of the world in which they live.
– To understand experiences of group members, their activities, interactions, discussions
– To understand how contextual factors influence the activities and decisions of group members
Observation
Types of observation
– Field Notes
– Sketches
– Relation diagrams
Continuum of roles that range from complete participation to complete observation
Type of role adopted depends on research question
Interviews
Types of Interviews
– Structured—verbal questionnaire, scripted questions
– Semi-structured—outline of topics guides questions
– Informal--conversational, questions arise in context
– Retrospective—focuses on past events
Focus Group
• Small groups with moderator
• Concentrated data on topic of interest in short time span
• Group interview with insight into interactions on topic (compared to interviews)
• Important that topic of interest would be easy for participants to discuss in group
Data Collection Decisions
Methods
Types of Data
Focus groups
Interviews
Open ended prompts
Observations Collect Artifacts
How Interactive? Researcher’s Role?
Audio Recording
Notes
Transcripts
Written response
Typed response
Notes
Video
Documents
Photos / Visual
Video
Example: How does participation in a teamwork & communication skills curriculum improve small group process in Year 1 of medical school?
• Observation of small groups pre & post
• Interviews with small group leaders (retrospective comparison)
• In-depth interviews with students
• Focus group with students
• Written evaluations of group process by students and faculty
Exercise 2: Analysis
A researcher conducted 5 focus groups at different medical schools. Each group had 3-7 third-year medical students and the students were at least 2 months into the clerkship year. The focus groups were recorded and transcribed. This researcher approaches you for help coding the focus group data and developing a report of the findings. You will work in groups. Read the passages individually and in your group, start the initial analysis discussion.
Please document the steps you take to analyze the data and use quote/paraphrases to support your argument.
Qualitative Data Analysis
Two purposes
To identify and explain themes
To pull loosely related themes together in order to understand and explain relationships
Coding: Step 1
The process of breaking down, examining, comparing, conceptualizing, and categorizing data.
1. Conceptualize and label
2. Categorize
3. Name/define your category
4. Validate against examples
Coding: Step 2
Putting the data back together in new ways by making connections between categories and sub-categories.
– Link sub-categories to categories
– Define set of conditions under which categories work
– Verify statements against data
– Define differences/similarities between categories
– Propose and track relationships against data
Qualitative Analysis Continuum
Content Analysis
(quasi-qual)
Qualitative Content Analysis
Thematic Analysis
Grounded Theory
Data analysis
Count the frequency of
words or phrases
Generate a list of categories – a coding scheme
Code, organize categories into
constructs / themes
Collect, code, analyze data
simultaneously
Constant Comparative
Reporting Frequencies Frequencies Discuss themes (sometimes
counts)
Describe a model to explain a
phenomena
Rigor Inter-rater reliability
Multiple coders; Check for
consistency & reconcile
differences
Multiple coders
Review codes & themes, revise, &
reconcile differences
Seek out confirming / disconfirming info
Multiple coders & reviewers
Analysis Considerations
• Techniques to ensure rigor (credibility or trustworthiness)
• Interrater agreement
•Triangulation of data - cross checking
•Triangulation of sources – member checking
•Transparency
•Reproducibility (agreement over time, events or settings)
Exercise 3: Writing up Findings
You have finished analyzing the focus group data you collected on students’ struggles in the third year. Now you are ready to write up your findings. Please practice writing up one of your findings, making sure to consider the following:
– How will you organize the results section?
– How will you provide evidence to support your findings?
– Are there any techniques you might use to enhance trustworthiness or credibility of your results?
– What limitations would be important to mention?
Tools for Coding & Data Management
Demo of Nvivo
Other options: Atlas.ti, HyperRESEARCH, SPSS text analysis
Teaching Material References
Dr. Carol Kamin, UIC, Department of Medical Education
Lofland & Lofland. Analyzing social settings. Rubin, H.J. & Rubin, I.S. (1995). Qualitative
interviewing: The art of hearing data. Thousand Oaks, CA: Sage Publications
Strauss, A. & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park: Sage Publications
Cote, L., & Turgeon, J. (2005). Appraising qualitative research articles in medicine and medical education. Medical Teacher, 27(1), 71-75.
Also, bibliography will be emailed