trusting human beings to accurately report on their own characteristics
TRANSCRIPT
![Page 1: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/1.jpg)
SURVEYSTrusting human beings to
accurately report on their own characteristics
![Page 2: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/2.jpg)
WHY ARE THESE COMING UP IN THIS CLASS? Surveys (along with observations,
discussed later) are a very common technique for collecting nonexperimental data
Surveys are a systematic way of asking people to volunteer information about their attitudes, behaviors, opinions and beliefs
The success of survey research rests on how closely the answers that people give to survey questions matches reality (i.e., how people really think and act)
![Page 3: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/3.jpg)
PROBLEMS TO ADDRESS The first problem that a survey researcher has
to tackle is how to design the survey so that it gets the right information. Is this survey necessary? Is the purpose of the survey to evaluate
people or programs? Can the data be obtained by other means? What level of detail is required?
The second problem is how accurate does the survey have to be? Is this a one-time survey or can the
researcher repeat the survey on different occasions and in different settings?
How will the results be used? How easy is it to do the survey?
![Page 4: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/4.jpg)
WHEN TO DO A SURVEY The survey is an appropriate means of
gathering information under three conditions:
when the goals of the research call for quantitative and qualitative data
when the information sought is specific and familiar to the respondents and
the researcher has prior knowledge of the responses likely to emerge
![Page 5: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/5.jpg)
TYPES OF SURVEY DATA Categorical data: numbers or words are
used to group thingsEX: gender, race, religion, food
group, or place of residence. Ordinal data: When the numbers are used
to order a list of thingsEX: The ranking of football or
basketball teams, A list of things to do, color of medal in the Olympics
Interval data: responses represent actual quantityEX: height, weight, age
![Page 6: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/6.jpg)
MORE SPECIFIC TYPES OF SURVEY DATA There are six basic types of data that you might collect: Attitudes Opinions Beliefs Behavior Attributes (demographic characteristics)
Preferences
![Page 7: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/7.jpg)
SETTING UP SURVEY QUESTIONS The way a question or statement is
worded and the response options offered determine the nature of the data received.
Types of survey questions include:Open-ended responseClosed response Semantic differential scales Agreement and rating scales Ranking scales Checklists
![Page 8: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/8.jpg)
OPEN-ENDED RESPONSERespondent writes response in own wordsConsiderations for using open-ended
questions: Need to enter data by hand Develop a coding scheme for responses Content analysis?
Frequently used in exploratory studies to facilitate better understanding of a concept
Suggestion: it’s a good idea to always include an open-ended question giving the respondent the opportunity to add any additional comments.
![Page 9: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/9.jpg)
OPEN-ENDED RESPONSE – CON’T
Advantages: Allows the respondent to answer the question
with few limitations Report more information than with discrete
answers Disadvantage:
Need qualitative methods or coding system to analyze the responses
Require subjective judgements Example:
What habits increase a person’s risk for being overweight?
Describe the pain you experience with walking?
![Page 10: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/10.jpg)
CLOSED RESPONSEThese are the "multiple-choice" variety
where a person has to choose among several possible answers.
There are two types of closed response questions: Ordered answer choices represent points
along a continuum. Pain on a scale of 0 (none) to 10 (worst
pain ever) Unordered answer choices with each
choice is an independent answer. Examples: ethnicity and marital status
![Page 11: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/11.jpg)
CLOSED RESPONSE – CON’T
Advantages: Quicker and easier to answer Easier to tabulate and analyze List of possible responses helps participant
understand the meaning of the question Suitable to multi-item scales designed to
provide a single scoreDisadvantages:
Do not allow participants to express their own answers
Set of answers may not be exhaustive Must be clear about selection of items, one
or as many as applicable
![Page 12: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/12.jpg)
PARTIALLY OPEN RESPONSE
Require specific, short answers that do not encourage free expression.
Are a compromise between closed response and open response forms.
Provide an “Other” category where a person can provide additional information.
Example: blank spaces provided for the questions on racial background and persons living with you.
![Page 13: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/13.jpg)
SEMANTIC DIFFERENTIAL QUESTIONS
Use a five to seven-point rating scale with each end of the scale having an adjective or phrase.
These adjectives, called bipolar adjectives, are direct opposites. Semantic differential scales have three
common factors: an Evaluative factor covering such dimensions as good-bad, pleasant-unpleasant and positive-negative;
A Potency factor representing the dimensions of strong-weak, hard-soft and heavy-light, and
An Activity factor with such scales as fast-slow, active-passive and excitable-calm
![Page 14: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/14.jpg)
AGREEMENT SCALESEnd points are identified by adjectives or phrases
all of the steps may have an adjective or phrase associated with it
Example:A five point scale with steps labeled Strongly Agree, Agree, Neutral, Disagree, and Strongly Disagree (“Likert” scale)
![Page 15: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/15.jpg)
MODES OF SURVEY ADMINISTRATION – FACE TO FACE ADVANTAGES:
Generally yields highest cooperation and lowest refusal rates
Allows for longer, more complex interviews High response quality Takes advantage of interviewer presence Multi-method data collection
DISADVANTAGES: Most costly mode of administration Longer data collection period Interviewer concerns
![Page 16: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/16.jpg)
MODES OF SURVEY ADMINISTRATION - TELEPHONE ADVANTAGES:
Less expensive than personal interviews RDD samples of general population Shorter data collection period than personal interviews Interviewer administration (vs. mail) Better control and supervision of interviewers (vs.
personal) Better response rate than mail for list samples
DISADVANTAGES: Biased against households without telephones, unlisted
numbers Nonresponse Questionnaire constraints Difficult to administer questionnaires on sensitive or
complex topics
![Page 17: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/17.jpg)
MODES OF SURVEY ADMINISTRATION - MAIL ADVANTAGES:
Generally lowest cost Can be administered by smaller team of
people (no field staff) Access to otherwise difficult to locate, busy
populations Respondents can look up information or
consult with others DISADVANTAGES:
Most difficult to obtain cooperation No interviewer involved in collection of data Need good sample More likely to need an incentive for
respondents Slower data collection period than telephone
![Page 18: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/18.jpg)
MODES OF SURVEY ADMINISTRATION - INTERNET ADVANTAGES:
Lower cost (no paper, postage, mailing, data entry costs) Can reach international populations Time required for implementation reduced Complex skip patterns can be programmed Sample size can be greater
DISADVANTAGES: Approximately 77% homes have computer (2010 data) –
what about the other 23%? Representative samples difficult - cannot generate random
samples of general population Differences in capabilities of people's computers and
software for accessing Web surveys Different ISPs/line speeds limits extent of graphics that can
be used
![Page 19: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/19.jpg)
WRITING “GOOD” SURVEY QUESTIONS
Are clear and use simple languageAre conciseAre specificAre possible to answerAre relevant to the respondentDo not use negativesAvoid bias termsHave only one part (not two parted
question)
![Page 20: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/20.jpg)
READING LEVEL So that every respondent will
understand a question, it is important to keep the reading level at or below the average reading level of the population. Complex words may be replaced by simpler ones or ones more easily understood. If you are giving a survey to a particular group, you would want to use words that are common to the group.
![Page 21: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/21.jpg)
RESPONSE OPTIONS Should reflect concepts you are
trying to measure, and fit with the wording of the questionAvoid simple “yes” or “no” answers
and attempt to measure intensity if possible
Mutually exclusive (select only one answer)
Exhaustive (all possible answers are listed, including other or not applicable or don’t know)
![Page 22: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/22.jpg)
QUESTION ORDER Be attentive:
Initial questions affect answers to subsequent ones
Start with easy, salient, non-threatening questions near the end
Cluster questions addressing the same topic or concept together.
Avoid redundancy
![Page 23: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/23.jpg)
RELIABILITY AND VALIDITY
![Page 24: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/24.jpg)
WHAT ARE THEY? Reliability:
Expect to obtain the same information time after time. Assessed by correlation coefficient The concept of reliability can be applied to
sampling If we repeatedly draw random samples of equal
size from a population, we can expect to get the same sample values each time (plus or minus a certain amount due to sampling error).
Validity: Measures the concept intended to measure
Instrument is presented or used in the way for which it was intended
An IQ test is valid only if it is used to measure intelligence it is not valid if it used to assign individuals to groups.
A psychological test that is a valid measure of anxiety is not a valid measure of stress.
![Page 25: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/25.jpg)
TYPES OF VALIDITY Face validity
Information collected appears to be what was expected (Face value)
A question that asked “Do you smoke?” would appear to have face validity as a measure of smoking behavior
Content validity A question adequately reflects the underlying
behavior or body of knowledge Content validity is established by having a panel of
experts evaluate and agree on the relevance of the test items
Concurrent validity One instrument or question is comparable to
another that has been shown to validly measure the same content or construct
Concurrent validity is established by correlating one question with another that has previously been validated
![Page 26: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/26.jpg)
EVEN MORE TYPES OF VALIDITY Discriminant validity
A question or survey that is able to discriminate between group differences.
Example Depressions scales have discriminate validity if
individuals who are depressed score differently from those who are not clinically depressed.
Predictive validity A question can be used to predict behavior. Example:
Can you walk 5 blocks? Construct validity
A construct is a theoretical dimension like self-esteem that is measured by having several questions that all relate to how people view themselves.
Self-esteem does not exist by itself but is represented by how people respond to these questions.
In this example, construct validity measures the extent to which these responses can be called self-esteem.
![Page 27: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/27.jpg)
TYPES OF RELIABILITY Test-retest reliability
Obtained by administering the same test on two or more successive occasions and then correlating the scores
Statistic that reflects reliability is correlation coefficient, higher is better
Internal consistencyObtained by correlating the scores on
several questions that pertain to the same content to the sum total of the scores
The average item-total correlation is a measure of how consistently people respond to related items on a test
![Page 28: Trusting human beings to accurately report on their own characteristics](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649c8f5503460f94947dc2/html5/thumbnails/28.jpg)
TYPES OF RELIABILITY – CON’T
Stability How much variation exists in scores upon
repeated administrations of the instrument.Stable measures will reproduce the same
score on repeated administrations of the instrument.
This concept is similar to test-retest except that in test-retest situations there is no assumption that the absolute value of each persons test score will stay the same.
Considerations: Time between administration, if too close
may be remembering answers Learning effect on repeated administrations