Survey Research

By

Dr Rojnath Pande

Survey Research

Longitudinal Surveys

Cross-Sectional Surveys

Characteristics of Survey Research

In survey research, the researcher selects a sample of respondents and administers a questionnaire or conducts interviews to gather data, which are then used to describe the population

Surveys are used to learn about attitudes, beliefs, opinions, behaviors, to name a few

Surveys are often used because researchers can gather accurate information about a large number of people using a small sample

Thus we have both surveys as a research tool and survey research as a research design.

Examples of Survey Research Topics

Descriptive Studies– What is the average length of time teachers use to prepare

lessons?

Relationship Studies– Is there a relationship between teacher attitudes toward

discipline and student satisfaction with the class?

Explanatory Studies– Why are students in one school achieving better than similar

students in another school?

Advantages of Survey Research

Versatility Efficiency Generalizability Cost effectiveness Small samples provide for reliable

extrapolation of results

Longitudinal Surveys

Studies in which the same group of subjects are studied over time

Trend studies use the same population across time but use different samples from that population each time

Cohort studies examine a specific group (same population) over time

Panel studies are cohort studies that use the same sample each time

Cross-Sectional Surveys

Attempt to simulate longitudinal data in a shorter time frame

Data are gathered from multiple samples of the same population simultaneously

May be used to study a phenomenon at one time or to gather data from multiple groups at the same time

A Note on Surveys…

Need to know, in advance, how the data will be used rather than “fishing” for whatever’s out there

Data collection methods may vary but they must be standardized

Instructions should be clear Both the instructions and survey should be pilot

tested Because a typical response rate to a mailed survey is

only 40%-60%, follow ups should be conducted Nonrespondents may introduce bias

Questions to Ask Before Doing Survey Research

Do you have a clear hypothesis? Do your questions focus on that hypothesis? Will participants answers provide accurate

answers to your questions?* To whom will your results apply?*

Advantages and Disadvantages of Survey Research:Conclusions

Easy way to get a lot of information However, that information:

– Will not have internal validity– May not have construct validity because of self-report

problems– May not have external validity because of poor sampling or

because of nonresponse bias– May not answer research question because survey

questions weren’t focused on hypotheses

The Advantages and Disadvantages of Different Survey Instruments

Written Instruments * Interviews*

Written Instruments

Self-administered questionnaires– Cheap, easy to distribute to large sample--but nonresponse bias is

big problem– Anonymous which may promote honest responses

Investigator-administered questionnaires– Higher response rates– May hurt sense of anonymity and thus decrease honesty of

responses Note: A highly refined version of the investigator-

administered questionnaire is the psychological test*

Interviews

May be worth added expense if– It increases response rate– Need ability to clarify questions, follow up on responses

May not be worth added expense if construct validity is harmed by

– Interviewer bias– Participant trying to impress interviewer

Telephone interviews may be ideal solution*

Planning a Survey

Deciding on a research question Choosing the format of your questions* Choosing the format of your interview--if you use an

interview* Editing your questions* Sequencing your questions* Refining your survey instrument* Choosing a sampling strategy*

Choosing the Format of Your Questions

Fixed alternative– Yes/No

Reliable Not powerful

– Likert

Open-ended– May not be properly answered– May be difficult to score

Choosing the Format of Your Interview

Unstructured– Interviewer bias is a serious problem– Data may not be hard to analyze

Semi-structured– Follow-up questions allowed– Probably best for pilot studies

Structured– Standardized, reducing interviewer bias

Editing Questions: Nine Mistakes to Avoid

1. Avoid leading questions

2. Avoid questions that invite the social desirability bias

3. Avoid double-barreled questions

4. Avoid long questions

5. Avoid negations

6. Avoid irrelevant questions

7. Avoid poorly worded response options

8. Avoid big words

9. Avoid ambiguous words & phrases

Sequencing Questions

To boost response rate, put innocuous questions first, personal questions last

To avoid wasting time, qualify early To increase accuracy, keep similar questions

together To boost response rate, put demographic

questions last

Putting the Final Touches on Your Survey Instrument

Professional appearance Proof reading Pilot testing Practice coding responses--may lead to

refining questionnaire so that it is easier to code responses

Choosing a Sampling Strategy

Random sampling– Proportionate stratified random sampling

Convenience sampling– Quota sampling

Conclusions– Only random sampling will be representative– Nonresponse bias may ruin your sample

Administering the Survey

Informed consent Clear instructions Debriefing Confidentiality

Analyzing Survey Data

Summarizing data Inferential statistics

Summarizing Data

Interval or ratio data– Mean– Correlation coefficients– Tables of means

Nominal data– Frequencies, percentages– Phi coefficient– Tables of frequencies

Using Inferential Statistics

Interval or ratio data– Looking at relationships between pairs of variables

If have two groups, could use t-test between means If not, should use test to see whether the correlation between

two variables was significant Be aware that if you do numerous statistical tests, you may be

setting yourself up for a Type 1 error

– To look at more than two variables at once, you can do ANOVA Multivariate analysis of variance, multiple regression, factor

analysis, and other sophisticated tests

Using Inferential Statistics

Nominal data– Chi-Square test– Be aware that if you do numerous statistical tests,

you may be setting yourself up for Type 1 errors

Concluding Remarks

Survey research is the most used research method

Survey research is the most misused research method

You know how to use rather than abuse survey research

The End