cross cultural surveys and linkert scales
Post on 10-Feb-2017
126 Views
Preview:
TRANSCRIPT
Cultural Biases in Surveys
Sarah MillerMarch 12, 2013
Our QuestionsJapan - Are they always going to say “yes”?Brazil - Are Brazilians just happy?France - Is anything good enough for the French?Germany - Are the Germans just not fun?
Why are your survey responses for Germany & France so different from Brazil & Japan? Is your
survey adapted for culture?
How are they responding currently?
Japan (-)Brazil (+)France (negative)Germany (negative)
Maybe they have one or more of these biases
Acquiescence Response Style (ARS)Where is it a problem?
Agree/Disagree questions
Middle Response Style (MRS)Extreme Response Style (ERS)Response style bias: is there a tendency to
respond to survey questions in a particular way, regardless of content?
ARSAcquiescence Response Style (ARS)
Where is it Prevalent?Cultures where being polite is importantCan we pinpoint the issue?Agree/Disagree questions are tough
Some cultures interpret these questions as a double negatives (agreeing with something negative is doubly negative)
They may not fully disagree or agree
MRSMiddle Response Style (MRS)
Where are MRS prevalent? Cultures where extremes are discouraged: Asian CulturesCan we pinpoint the issue?
Confucious: No one answer is absoluteCollectivism: Avoid making others look bad
ERSExtreme Response StylesWhere are ERS prevalent?
In cultures that value individualism, are masculine, and have a low tolerance for ambiguity
Can we pinpoint the issue?
Individualistic, masculine cultures put a value on assertiveness and decisiveness
Geert wants to remind us…
Power Distance (PDI)Individualism (IDV)Uncertainty Avoidance (UA)Masculinity (MAS)Long Term Orientation (LTO)
Response biases are related to all of these orientations
Brazil: High Acquiescence Highest acquiescence cultures…and may have a bit of SDRWhy?
High UA & Low IDVCollectivism (low IDV)
Expected behaviors More collectivist than Japan!
Aversion to confrontation Politeness
UA - 71IDV - 38
Japan is a Middle Response CultureWhy?
Collectivism Personal opinions are not
strongly voiced Value harmony
Dialectical Thinkers No one, absolute truth
Long Term Orientation LTO societies believe that
truth depends on the situation
Germans are well balanced, but can be a
little negativeWhy?
DRS or “disacquiescence” is associated with low power distance and high individualism. Germany is be a prime example of this type of culture.
High masculinity and UA are also associated negatively to acquiescence.
PDI: 35 IDV: 67 MAS: 66 UA: 65
The French are Extremists!
The French are the 3rd most extreme culture in Europe.
Why?Rules (UA)Societies higher in UA have many rules and have little tolerance for ambiguity.Direct Communication (IDV) Individualistic societies are
characterized by self-expression, independence, and personal opinions.
Individual contributions expected, speaking up appreciated – disagreement is normal.
Individualistic societies have absolute truths
UA: 86 IDV: 71
"You Can't Judge a Book by its Cover", Arun Joshi, Sagar Tamang, Himanshu Vashishtha, Research World, January 2009 International Marketing Research By C. Samuel Craig, Susan P. Douglas
SolutionsLinkert 5 point scales are not the best solution across these response styles & cultures
How to balance acquiescent culturesWhat’s the Solution?
Give various levels of agreement and disagreementReverse order points of scaleEmploy negatively worded statements vs positive
Or don’t use obvious highs/lows or negatives/positives
Use visual stimuli/pictures vs words or numbersTest for acquiescence using a behavioral variable Joshi-Tamang-Vashishtha Scale
Acquiescence is closely related to Social Desireability – lower class people will give responses that reflects the values of the interviewer in face to face interviews
Solutions for MRS CulturesWhat’s the Solution?Kill the middle response!Use a multi-item scale
Solutions for the Extreme Response Styles
What’s the solution?
Survey DevelopmentSurvey TypeItem GenerationSurvey Languages
Survey TypeNot all surveys are transferableScale development is important
Measurement scales need to be investigated
Item GenerationDoes the survey rely on commonalities or distinctions?
Emic or Etic? Applicable universal concepts, values, truths
Frame of reference - not just language - should be conceptually equivalent
Suggestion
Qualitative data (interviews) can be helpful, especially if taking surveys from multiple countries at once
The Role of LanguageEnglish language by default elicits middle responses
Opposites in one language may not be opposite in another language
Double negatives!
Intensity levels or words could be misinterpretedDirect translation isn’t always the key
Scale anchors may not translate well
Harzing, 2006
The Role of LanguageLanguage impacts surveys!
Cultural accommodation Symbolic effect
Concepts and terms Interpretation – culture specific connotations
Researcher needs cultural knowledge as well as language knowledge
Use clear, simple, and familiar wording
Use natives in country where surveying Careful translation + pilot testing with native speakers
Other SolutionsMixture of Positive & Negative Statements
Mitigates acquiescence and disaquiescence
Linkert scales, use more than 5! Reduces response and language bias Scale anchors that have mutually exclusive constructs rather
than agreement level also reduce biases*
Do not compare loyalty scores across countries, instead compare loyalty scores across companies in the same country!
Less complex than statistical adjustments and probably more accurate
NOT Good SolutionsSome organizations try to eliminate the influence of
response bias by calculating a cross-cultural adjustment factor for each country or region in which they do business.
The adjustment factor is calculated as the ratio of the average score for a comparison country over the average score for a base country, which is typically the organization’s largest, most important market.
In practice, however, this method is ill-advised unless the adjustment procedure truly isolates opinion differences from differences due to cultural response bias
RememberThe researcher needs to take culture into consideration in the overall design, language, and item generation of the survey.Benchmarks should be within the same countryMore points on the Linkert scaleCross-country comparison helps to identify
survey issuesNative/local speakers can help with design
top related