![Page 1: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/1.jpg)
Survey Research
AD700 College of Advancing Studies
13 October, 2004 Brendan Rapple
![Page 2: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/2.jpg)
This presentation owes much to the American Statistical Association brochure series on survey research:
http://www.amstat.org/sections/srms/whatsurvey.html
![Page 3: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/3.jpg)
Little Cards on Restaurant Tables:Was the service good?
Telephone: Is president doing a good job?The most popular programs on public radio?
Census:How many bathrooms do you have?
Magazines:How is your romantic life?
Market Researchers: Brand X or Brand Y?How many drinks last week?
![Page 4: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/4.jpg)
Surveys Provide Important Knowledge
Economists, psychologists, health professionals, political scientists, and sociologists conduct surveys to study such topics as:
Income and expenditure patterns among households; Roots of ethnic or racial prejudice; Implications of health problems on people’s lives; Voting behavior; Effects on family life of women working outside the home, etc.
![Page 5: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/5.jpg)
Specific Purpose Essential
Objectives of a survey should be as Specific Clear-cut Unambiguous as possible
"Men's Health Practices" is a very nebulous topic.
Better: How often do African-American males aged 40-49 visit the
dentist?
![Page 6: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/6.jpg)
Steps in Conducting A Survey
Define precise purpose
Specify population
Specify appropriate sample
How to administer survey?
Draft of survey instrument
Pretest it
Revise it
Administer survey to sample
Analyze, write it up, and communicate the results
Use results meaningfully
![Page 7: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/7.jpg)
Decide on Mode of Data Collection
Telephone
In Person Interview
Computer
![Page 8: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/8.jpg)
Pre-Testing
Critical for identifying questionnaire problems.
Main problems revolve about:
Question content, e.g. confusion with overall meaning of question as well as misinterpretation of individual terms or concepts
Formatting, e.g. problems with how to skip or navigate from question to question may result in missing data and frustration for both interviewers and respondents.
![Page 9: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/9.jpg)
Population
Individuals
Larger units, e.g. families
![Page 10: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/10.jpg)
Sometimes Difficult to Specify Population
e.g. female faculty members at BC: do we include part-time profs?
![Page 11: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/11.jpg)
Samples
Must be representative of population.
Are the distributions of attributes, opinions, and beliefs in the sample the same as in the population?
You want to be able to make inferences about the population as a whole based on what you find to be true of the sample.
Often difficult to find representative sample.
Always a danger of sampling error or bias.
![Page 12: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/12.jpg)
Quality of Sample Important
The quality of the sample – whether it is up-to-date and complete – is probably the dominant feature for ensuring adequate coverage of the desired population to be surveyed.
![Page 13: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/13.jpg)
Variability
Variability is large, then sample should be large
Converse also true
![Page 14: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/14.jpg)
2 Barrels of Apples
Barrel A (low variability) -- all apples about 3 ins. in diameter (range 3.1 to 2.9 ins.)
Barrel B (high variability) -- apples range from 2 to 5 ins. in diameter
Picking 3 apples from Barrel B might give result well below (above) average.
![Page 15: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/15.jpg)
Size of Sample Isn't Everything
Large numbers do not, in and of themselves, increase the representativeness of a sample.
Most professional survey conductors hold that a moderate sample size is enough statistically and operationally.
![Page 16: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/16.jpg)
Whole Population and Sample
Sometimes the Same
Example:– Small companies in the paper recycling industry in LA.
– Unit of Analysis: a company
– You define "small company" as a private co. with turnover of less than $2,000,000 per annum
– Research shows that there are 34 relevant companies
– Therefore, manageable to use ALL in sample
![Page 17: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/17.jpg)
N.B.Results will only relate to small paper recycling companies in LA -- difficult to generalize about other types of company in other parts of the country.
![Page 18: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/18.jpg)
Population Often Not Feasible Due to Size
• Welfare Recipients
• Mentally Ill
• Prison Inmates
Often essential to survey a REPRESENTATIVE sample.
![Page 19: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/19.jpg)
Early Studies of Gay Men
Sampling frame composed of men, patients of
therapists participating in research
But most gay men were not patients of therapists
![Page 20: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/20.jpg)
Representative Sample
EXAMPLE--Success of unwed teenage mothers in raising children?
![Page 21: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/21.jpg)
To be representative, sample must contain same proportion of unwed teenage mothers at
--each age level
--each educational level
--each socio-economic statusin community
![Page 22: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/22.jpg)
Lists May Be Very Exclusive
EXAMPLEUndocumented Aliens
--We know that many live in LA
--But relying on Govt. lists may be useless
![Page 23: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/23.jpg)
Suppose You Have a “Population,”e.g.
all registered voters in your county
all Mercedes owners in the state
all soccer players in your school district who drive green mopeds
THEN YOU SELECT SAMPLE IN SUCH A WAY THAT EVERY NAME ON THE LIST HAS AN EQUAL CHANCE OF BEING INCLUDED IN THE SAMPLE
![Page 24: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/24.jpg)
Random Sample
Random =Purposeful & methodical
Not reflect biases of researcher
Everyone has equal & independent chance of being selected
![Page 25: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/25.jpg)
Random Sample
Once selected it cannot be chosen again (like lottery winners)
Example: 500 part-time students in Advancing Studies
Sample of 20% is required
Assign each student a number from 1 to 500
Randomly select 100 numbers (by computer or by table of random numbers)
![Page 26: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/26.jpg)
Systematic Random Sampling
Example 1. 2,000 in sampling frame and you want a sample of 200, then you
might select every 10th name
Example 2. 500 part-time students in Advancing Studies Sample of 20% is required
--Randomly Select a Number from 1 to 5
--Select Every 5th Person
--002, 007, 012. 017, 022, and up to 497.
![Page 27: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/27.jpg)
Possible Problem: Staff in govt. agency may be listed unit by unit
Each unit has 9 line-level workers and 1 supervisor.
The supervisor is the 10th person on the list.
It’s a survey of 20% -- every 5th person is selected.
If first no. selected is 1, 2, 3, or 4 then no supervisor will be selected, though they comprise 10% of population.
If first number selected is 5, then supervisors will be greatly overrepresented.
Thus, possibility of bias due to periodicity or patterns.
![Page 28: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/28.jpg)
Stratified Sampling
Population: 2,000 (800 females; 1,200 males)
Sample required: 200
If gender is an important variable in your survey, then both females and males should be included in appropriate numbers, that is, in proportions that correspond to their presence in the population.
![Page 29: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/29.jpg)
Strategy:
Treat both sexes as separate populations and take 10% sample from each.
OR
Make sure that all females are listed first and then take every tenth name.
Either way you will end up with 80 females and 120 males
![Page 30: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/30.jpg)
Convenience Sampling
![Page 31: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/31.jpg)
Cluster Sampling
Often difficult to list all members of target population and select the sample from among them
e.g. 1) Population of American high school students
2) Population of U.S. postal delivery workers
3) Adult population of Atlanta
![Page 32: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/32.jpg)
Possible Strategies
Population of American high school students
choose 50 schools randomly from entire list and include all students in those schools in the sample.
Population of U.S. postal delivery workers
choose 100 post offices randomly from all 50 States and include all
deliverers in those post offices in the sample.
Adult population of Atlanta
Randomly choose sample of 50 blocks from a city map and then poll all adults living on those blocks.
![Page 33: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/33.jpg)
Potential Problems
Confidentiality Confidentiality of data supplied by respondents is of prime
concern to all reputable survey organizations.
Strategies: Using only number codes to link the respondent to a questionnaire. Refusing to divulge names and addresses of survey respondents. Omitting the names and addresses of survey respondents from
computer files used for analysis Presenting statistical tabulations by broad enough categories so that
individual respondents cannot be singled out.
![Page 34: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/34.jpg)
Reporting
Important that individual respondents are not identified in reporting survey findings.
All of the survey’s results should be presented in totally anonymous summaries, such as statistical tables and charts.
![Page 35: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/35.jpg)
Volunteers Volunteers usually have characteristics that differentiate
them from the larger population. The fact that they volunteer makes them different from
persons in the population who do not volunteer. They tend (but not in all circumstances):
– to be better educated– have higher social class– to be more intelligent– have greater need for social approval– to be more sociable– more unconventional (especially when volunteering for
studies of sex behavior)– less authoritarian– less conforming
![Page 36: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/36.jpg)
Volunteers (Cont.)
Jews more likely than Protestants.
Protestants more likely than Catholics.
Females more likely than males.
![Page 37: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/37.jpg)
Volunteers -- Example
TV programs asking viewers to vote.
– people call who are most committed to issue.
– “stuffing of ballots” by multiple calls.
– Time of day is important – who’s available?
![Page 38: Survey Research AD700College of Advancing Studies 13 October, 2004Brendan Rapple](https://reader030.vdocuments.site/reader030/viewer/2022032704/56649d775503460f94a59f9d/html5/thumbnails/38.jpg)
Margin of Error
Error margin of 1,000 randomly chosen individuals is said to be 3.1%.
Thus, if a random sample of 1,000 indicates that 59% will vote for Bush, the actual number could range from 55.9% to 62.1%.
In the 1984 election, the Gallup Survey (using 3,456 individual responses) missed by just +0.2 of 1% when it predicted that Ronald Reagan would win by 59.0%.