![Page 1: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/1.jpg)
1
Using Qualitative Methods to Develop Culturally Adequate Surveys
Anna Nápoles-Springer, Ph.D.University of California San Francisco
Center for Aging in Diverse Communities
![Page 2: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/2.jpg)
2
Overview
Integrating quantitative and qualitative methods to develop surveys
Focus groups Cognitive interview methods Using results of cognitive interviews to make
decisions about item revisions/deletions
![Page 3: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/3.jpg)
3
Integrating Quantitative and Qualitative Methods
In diverse populations, qualitative work is necessary in addition to more traditional quantitative studies
Prior to quantitative: to develop concepts, items appropriate to culture
After quantitative: – to help identify reasons for items not performing well
quantitatively– to explore possible explanations for unexpected results
![Page 4: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/4.jpg)
4
What are Qualitative Methods?
Data consist of words, not numbers Richly descriptive, open-ended Focus on inductive analytic approaches Many types: ethnography, participant-
observation, direct observation, interviews, focus groups
![Page 5: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/5.jpg)
5
When are Qualitative Methods Useful?
To understand the meaning of participants’ events, situations, and actions
To understand contextual influences on participants’ actions
To identify unanticipated phenomena and influences (e.g., exploratory studies to design questionnaires and identify variables for study)
To understand the processes underlying observed relationships between variables
![Page 6: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/6.jpg)
6
Combining Qualitative and Quantitative Methods
Qualitative Methods to Develop Quantitative Tools
Qualitative Methods to Explain Quantitative Results
Qualitative ResultsQuantitative
Quantitative Results
Qualitative
![Page 7: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/7.jpg)
7
Combining Qualitative and Quantitative Methods cont.
Quantitative Methods to Expand on Qualitative Study
Quantitative and Qualitative Methods Equal
Qualitative Results
Quantitative
Qualitative Results Quantitative
(Tasjakkori A. Sage Publications, Inc. 1998)
![Page 8: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/8.jpg)
8
Focus Groups can Identify Right Questions to Ask in Expanded Surveys
1996 US Census Bureau –little difference between rates of work force participation of low-income families of children with special needs and those with no special needs
Might conclude child care systems adequately meeting needs of special needs families-able to work
Focus groups-enormous strain, scarce programs, working nights, losing multiple jobs
(Sunhua L. 2002)
![Page 9: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/9.jpg)
9
From Focus Groups to InterventionThe Subjective Culture of Smoking
Purposeful sampling stratified by comparison groups (by ethnicity)
Examine similarities and differences in attitudes, beliefs and behaviors (Triandis)
Antecedents and consequences of smoking
![Page 10: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/10.jpg)
10
Focus Groups to Identify Themes
Themes Extracted from Focus Groups Family concerns - 2nd hand smoke Heightened health issues Importance of appearance and
interpersonal relations - simpatía Habitual use less important Addiction less of a concern
![Page 11: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/11.jpg)
11
Integrate Results to Develop Intervention: Programa Latino Para Dejar de Fumar
Guia Para dejar de Fumar--self-help guide
Electronic media in Spanish
Emphasis on family and collective
orientation of culture
Immediate effects of smoking
Health effects in different contexts
![Page 12: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/12.jpg)
12
Develop Structured Questionnaire
Scales developed based on themes/lit/theory 17 antecedents to smoking (habitual activities, social
activities, emotional states) 15 reasons to quit (family, appearance, health) 3 reasons to keep smoking (weight, nervousness,
concentration)
![Page 13: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/13.jpg)
13
Triangulation of Focus Group Themes: Latinos’ Reasons to Quit
OR 95% CI
Criticized by family1.93 (1.26, 2.98)
Burn clothes 1.57 (1.02, 2.42)
Children's' health 1.67 (1.08, 2.57)
Bad breath 2.07 (1.40, 3.06)
Family pressure 1.69 (1.10, 2.60)
Good example 1.83 (1.21, 2.76)
Not to gain weight 0.38 (0.24, 0.59)
![Page 14: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/14.jpg)
14
Integrating Qualitative and Quantitative Methods to Develop Surveys
Goal: develop survey items that mean the same thing across groups
Quantitative methods (descriptive statistics, reliability, validity, missing data) identify ethnic differences in response patterns
Qualitative methods uncover cognitive processes respondents use to answer questions; disclose reasons for ethnic difference in response patterns
Iterative quantitative and qualitative research methods
![Page 15: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/15.jpg)
15
Integrating Qualitative and Quantitative Methods to Develop Surveys
Example: – Item: In the past 12 months, how many times have
you been to see a doctor?– Test-retest reliability was .56-.65 for Chinese and
Vietnamese compared to .90 in Whites– Interviewer debriefing revealed problems with
interpretation of what constituted a visit to a doctor among Chinese and Vietnamese
Pasick RJ, et al. 2001 Public Health Rep;116:223-43
![Page 16: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/16.jpg)
16
Open-ended Interviews to Develop Structured Questionnaire
Example: Constructing attitudinal scales Use open-ended interviews (individual or
focus groups) to obtain statements on topic of interest to identify relevant concepts
Organize statements into “item pool” Pretest closed-ended items Modify items based on pretest and administer
in structured interview
![Page 17: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/17.jpg)
17
Goals of Purposeful Sampling
Achieve representative, typical settings, individuals, or activities
Adequately capture heterogeneity, sample for broadest range of variation rather than typical individuals
Set up contrasts to examine differences between settings or individuals
(Maxwell JA. Sage Publications, Inc., 1996)
![Page 18: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/18.jpg)
18
Sampling in Qualitative Research
Quantitative research-2 types of sampling:
– Probability
– Convenience Qualitative research-3rd type of sampling:
– Purposeful»Deliberately select settings, persons or events
to best answer research questions
![Page 19: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/19.jpg)
19
Qualitative Methods to Develop Surveys
Focus Groups– Open-ended guided group discussion with probing of
responses
Cognitive interviews– Open-ended probes to assess how items are interpreted and
adequacy of response choices
– Respondent is expert
Expert panels provide input on relevant concepts Interviewer debriefing
![Page 20: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/20.jpg)
20
Focus Groups
Group discussion led by experienced moderator usually 1.5 - 2 hours
Purposeful sampling of 6-10 homogenous participants per group
Use open-ended questions with follow-up probes for clarification, usually covering 10-12 topics
Participants stimulate comments of others Audio-record and transcribe discussion
![Page 21: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/21.jpg)
21
Focus Groups-Logistics
Moderator skills: listening, communication, negotiation, cultural similarity to participants
Costs of group: $600 - $1000 per group (incentives, audio-taping, transcription, translation, food)
Convenient and hospitable community setting In-person recruitment with telephone and mail
follow-up
![Page 22: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/22.jpg)
22
Example: The Interpersonal Processes of Care
Examined similarities and differences in attitudes, beliefs and behaviors
Sampling stratified by comparison groups: Whites, African Americans and Latinos
Probed on aspects of communication, decision making and interpersonal style of physician
Used information to design structured questionnaire
![Page 23: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/23.jpg)
23
Identify Themes: How Interpersonal Processes of Care Might Affect Health Outcomes
Poor communication - doctors ...– Use unclear language, jargon
– don’t listen carefully
– don’t make sure that the patient understands Decision-making did not involve patients Cultural insensitivity
– patients’ beliefs, preferences not taken into account
![Page 24: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/24.jpg)
24
Identify Domains of Communication
General clarity Elicitation of and responsiveness to patient
problems and concerns Explanations and information
– about condition, prognosis
– about processes of care
– about what to do for themselves
![Page 25: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/25.jpg)
25
Define Domains: Definition of General Clarity in Communication
Communicates clearly even when patient does not speak English well– Uses interpreters if needed
Easy to understand Speaks clearly and slowly enough Explains things well Makes sure patients understand
![Page 26: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/26.jpg)
26
Develop Items Based on Definition: General Clarity
Over the past 12 months, how often did doctors…
• use words that were hard to understand• speak too fast• explain things well• asked if you understood what they told you
![Page 27: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/27.jpg)
27
Develop Item Pool and Pretest
Develop 5 – 6 items per domain or content area of interest
Will end up with 3 – 4 that work Use pre-testing to identify best candidate items for
structured survey Types of pretests
– General debriefing on comprehension, flow
– Cognitive interviews
![Page 28: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/28.jpg)
28
Cognitive Interviews
Derived from social and cognitive psychology to explore processes respondents use to answer survey questions
Diagnostic tool for pretesting survey questions
![Page 29: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/29.jpg)
29
Cognitive Interviews Examine 4 Steps in Answering Questions
Interpret and understand the question– as intended by the researchers
Retrieve the information – various schemas used to access memory
Judgment formation - formulate an answer– calculate or judge the correct information
Edit response - decide what to report– is answer embarrassing, socially undesirable?
![Page 30: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/30.jpg)
30
Purpose of Cognitive Interviews
To learn .. .. if respondents understand words and phrases as
intended (meaning) .. about the process of answering the questions .. whether items are unacceptable .. about the usefulness of response choices
– whether response choices are adequate– how they use the response choices
![Page 31: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/31.jpg)
31
Two Types of Cognitive Interviews
Think aloud interviews– Respondent asked to think aloud as they answer
question Probe interviews
– Interviewer asks specific questions to elicit how respondent answered question
Think alouds - greater respondent burden
![Page 32: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/32.jpg)
32
Steps Involved in Cognitive Interview Pretesting
Decide on a final item pool Develop interview “probes” for a subset of items Translate interview and probe questions Recruit sample for cognitive interviews Conduct cognitive interviews Analyze results Revise items based on results
![Page 33: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/33.jpg)
33
Reducing Item Pool
Reduce items from item pool (subjective process among research team)
Criteria:– maintain breadth of concept, multiple items/concept– reduce redundancy (but OK to test alternate
versions of items)– eliminate items that are unclear, complex, require
high verbal skills, lack face validity, or will not translate well
![Page 34: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/34.jpg)
34
Writing Probe Questions
From reduced set of items, select potentially problematic items for pretesting
Write open-ended probe questions– worded to reveal if suspected problem with
item is present
![Page 35: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/35.jpg)
35
Types of Cognitive Probes
General– Tell me what you were thinking when you
answered that question– How easy or difficult was it to answer that
question? Why? Explore meaning of word or phrase
– I asked you how often doctors take a genuine interest in you. What does the phrase “genuine interest” mean to you?
![Page 36: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/36.jpg)
36
Types of Cognitive Probes (cont.)
Retrieval– How did you remember that?
Judgment– Why did you pick that number for your answer?
Response– Do you think that most people answer this question
honestly?Collins D. Quality of Life Research 2003. 12:229-38.
![Page 37: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/37.jpg)
37
Types of Cognitive Probes (cont.)
Redundancy– How is the phrase “give you advice about your diet
and exercise” different from the phrase “talk to you about your diet and exercise”?
Acceptability– When I asked you how often you felt discriminated
against by doctors because of your race or ethnicity, you answered (read answer given). Were you offended by this question?
![Page 38: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/38.jpg)
38
Types of Cognitive Probes (cont.)
Cultural appropriateness– I asked you how often doctors asked you about your
health beliefs? What does the term ‘health beliefs’ mean to you?
![Page 39: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/39.jpg)
39
Sampling for Cognitive Interviews
Aim is not representative samples Include respondents from major segments of
population to be sampled for main survey Approximately 5-30 interviews/group
![Page 40: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/40.jpg)
40
Recruiting for Cognitive Interviews
Explain how their help fits into larger study, process of creating questions
Explain their role clearly: – “help us learn how to ask better questions”– “help us make questions clearer for others” – “help us to identify problems with questions”
Pay subjects – interview is demanding If survey is long, pretest different sections on
different subjects
![Page 41: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/41.jpg)
41
Conducting Cognitive Interviews
Individual face-to-face, in-depth interviews Standard administration of closed-ended
items Administer probe questions at the end Typically 1 to 1.5 hr interview Each interview audiotaped and transcribed
![Page 42: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/42.jpg)
42
Analysis of Cognitive Interviews
Data preparation– Data are transcripts– Translate transcripts if necessary back to
English Two approaches
– Behavioral coding– Use qualitative data analysis software to
perform content analysis
![Page 43: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/43.jpg)
43
Behavioral Coding
Review transcripts to identify problems with items– respondent and interviewer “problem” behaviors
Assign “problem behavior” codes to each item using pre-determined categories
Summarize results for each item:– proportion of interviews with each problematic
behavior for each item» e.g., 7/48 respondents requested clarification of
item 10
![Page 44: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/44.jpg)
44
Examples of Behavioral Codes
Interviewer behavior Hard to read - interviewer experiences
difficulty reading question
Respondent behavior Repeat question - respondent asks to have the
question repeated
![Page 45: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/45.jpg)
45
Analysis of Cognitive Interviews (cont.)
Using qualitative analysis software, review all dialogue that ensued during standard administration of closed-ended items and open-ended probes– can reveal source of problems– can help in deciding whether to keep, modify or
drop items Allows you to examine dialogue for each item
– within groups– across groups/languages
![Page 46: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/46.jpg)
46
Example of Results: Interpreting the Question
Original item: How satisfied are you with the amount of stress or worries in your life?
Probe: Did you answer this question in terms of stress, worries, or both?
AA - tended to answer in terms of worries No term in Spanish for “stress” Item revised
– How satisfied are you with the amount of worries in your life?Warnecke RB, etal. JNCI Monographs No. 20, 1996; 29-38.
![Page 47: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/47.jpg)
47
Example of Results: Unclear Phrase
Original item: Have you had any medical tests or procedures in past year?– 26% of respondents asked for clarification
Probe: What did you include as medical tests or procedures?
Medical test or procedures unclear (e.g., asked if it included dental or cosmetic procedures)
Item revised to include examples:– Have you had any medical tests or procedures, such as blood
tests, x-rays, or cancer screening tests?
![Page 48: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/48.jpg)
48
Example of Results: Information Retrieval
Original item: How satisfied are you with your ability to travel on vacations?
Probe: What do you think we meant by vacation? Neither concept of “vacation” nor “travel for pleasure”
had relevance to lifestyle of AA and MA; travel for family reasons
Dropped itemWarnecke RB, et al. JNCI Monographs No. 20, 1996; 29-38.
![Page 49: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/49.jpg)
49
Example of Results: Response Sets
Original scale: “Very unimportant to Very Important” on 0-100 scale (bipolar)
Problem: No direct Spanish translation for “unimportant”– bilinguals understood translation “sin importancia” as
equivalent to unimportant– Spanish monolinguals did not understand it as the
negative pole of the scale Solution: Changed English to unipolar scale “not at all
important” to “very important” to correspond to best Spanish
Warnecke RB, et al. JNCI Monographs No. 20, 1996; 29-38
![Page 50: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/50.jpg)
50
Example of Results: Redundancy of Items
Original items:– How often did doctors explain what was causing
your health problem?– How often did doctors explain your diagnosis?
Probe: What do the words health problem and diagnosis mean to you?
Respondents viewed them as the same Some respondents did not know meaning of diagnosis Dropped item with word diagnosis
![Page 51: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/51.jpg)
51
Summary
Greatest problems are with question interpretation– usually due to need to write at lower level of verbal
comprehension
Sometimes English concepts not meaningful in other languages or are irrelevant for certain groups
Need to pretest response sets– Ethnic groups may use them differently
![Page 52: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/52.jpg)
52
Advantages of Cognitive Interviews
Complement other survey field test methods (e.g. where problems are identified by missing or truncated answers)
Identifies where responses might be affected by cultural or group experiences
Suggests ways to revise items, response sets Improves validity of questions
![Page 53: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/53.jpg)
53
Disadvantages of Cognitive Interviews
Flags problems, but significance of the problem remains matter of subjective judgment– at which point need to revise or drop items
Based on small number of respondents Time and labor intensive
![Page 54: 1 Using Qualitative Methods to Develop Culturally Adequate Surveys Anna Nápoles-Springer, Ph.D. University of California San Francisco Center for Aging](https://reader035.vdocuments.site/reader035/viewer/2022081603/56649e235503460f94b1050a/html5/thumbnails/54.jpg)
54
Conclusions:Integrating Quantitative and Qualitative Methods
Focus groups, cognitive interviews complement and are necessary component of research in diverse groups
To identify relevant themes, constructs, items In context of developing theory (research
questions), structured surveys, interventions