Getting knowledge about people’s opinions, behaviour etc.(a) Survey Research <http://www.sfu.ca/~sreiblin/CED/401_final/node4.html>

Survey research is one of the most common forms of research that we encounter everyday. We receive surveys about our shopping habits by mail and surveys about our political beliefs by phone, and surveyors also come door to door to ask us for our opinions.

Survey research is a form of research used when researchers and/or the companies that hire them need to know more about a population. “The main goal of survey research is to produce an accurate picture of the population from which the sample is drawn''  [RICH],p.172.This means that anyone who is interested in the behaviors, attitudes, beliefs or just about any other demographic or psycho-graphic characteristic of a population can use a survey to find the answers to their questions.

Surveys can be used in may instances. Some of these include:

public opinion polls and market research studies analyses of social influence measuring behaviors, attitudes, values and beliefs gathering information to create policy determining financial trends in consumer and business habits determining how to best reach a potential audience

“The survey is an appropriate means of gathering information under three conditions: when the goals of the research call for quantitative and qualitative data, when the information sought is specific and familiar to the respondents, and the researcher has prior knowledge of the responses likely to emerge.'' 

"Although survey research seems to be a simple proposition consisting of asking people questions and then counting up their responses, social science has formalized this process to legitimize it as a scientifically valid research method. The debate around the validity of survey research continues to rage on in academic circles."

&&&&&&&&&&&&&&&&&&&

(b) Questionnaire Reliability <http://www.decpoint.com/reliability.html>

Here are nine things to listen for or ask questions about:

Are there any statements or guidelines about "sample size", or the minimum number of respondents necessary for good results.  If there is no concern about minimum sample sizes, there won't be any discussion about reliability.

If they seem to know what reliability is, do they measure it?  And how do they measure it?  There are

different approved ways — listen for words like "test-retest", "equivalent forms", "split-half", or "Cronbach's Alpha".  You don't have to know what those reliability terms mean, but your provider certainly should.

Ask if they will measure the reliability of your questionnaire's results.  This is far better than a test of reliability performed at another time and place.  It will give you concrete evidence of whether your results can be trusted.

Ask them if their questionnaire controls for different response patterns of the respondents.  Did you ever notice that some teachers give mostly C's and B's with few A's, while other teachers give mostly A's and only a few B's and C's?  People fill out questionnaires the same way — some are "black and white" using lots of high and low responses, while others use two or three boxes only.  Just like an A grade from an

easy teacher is different than an A from a hard teacher, a reliable questionnaire knows that a "3" from one respondent is not be the same as a "3" from another.

Ask them if their questionnaire controls for the effect of a issue's position in the survey.  It is known that people spend more time thinking about their answers at the beginning and get tired toward the end of a survey.  Does their questionnaire present different versions so each issue appears both early and late in the survey?

Ask if they are able to detect collusion — when a group of respondents agree in advance how they will answer the questionnaire, to make the results come out a particular way.

Ask if they measure the variability of each question or issue, to know on which issues people agree and on which they disagree.

Ask if they can identify respondents who have difficulty understanding the questions or instructions because of literacy or language issues.

What is Reliability, and Why Does It Matter?If you buy an unreliable questionnaire, it's like trying to measure the length of something with a rubber tape measure.  You can mark off inches or centimeters very precisely, but the tape must be a material that is inflexible in the direction of measurement.  Steel or fiberglass work wonderfully for this purpose.  They coil or fold up, but in the direction of measurement, they are rigid.  Not so if the material is rubber.  You could make your marks at precise intervals, but the flexibility of the material would destroy its reliability.  Questionnaires that use rating scales to record people's opinions are like rubber tape measures.  For more on this, please read What's Wrong With Rating Scales?.

Non-technically speaking, a reliable questionnaire is one that that would give the same results if you used it repeatedly with the same group.  That may sound funny because most organizations don't administer a questionnaire to the same group twice.  But if they did, they would learn how reliable their questionnaire is, because a reliable survey will give the same results on Tuesday as it did the previous Monday.  Instead of doing it twice, statisticians have devised tests of reliability for questionnaires.  These tests let us know whether the results are meaningful.

Reliability is a property of the measuring instrument.  If you are like many people, you probably get on your bathroom scale in the morning, look at the weight displayed, then step off, and do it again.  You have learned that what is displayed by a bathroom scale the first time is not always exactly the same as the second, but it is usually very close.

What if one morning you weighed yourself, then a second time, and the second weight displayed was 5 lbs. heavier than the first?  You would probably step off, then weigh yourself a third time.  What if it was now 4 lbs. lighter than the first?  Would you still be concerned about your weight?  Or would you be more concerned about finding out what's "wrong" with the scale?  What's wrong is that your scale has become unreliable.  You can see unreliability by repeatedly measuring the same thing.  And when you know the scale is unreliable, you don't even try to measure your weight, you concentrate on fixing the scale first.

Only when you know your questionnaire is reliable, can you begin to discuss validity (see the site for more problems related to surveys).