TYPES OF BIASSection 4.1A

Remember….

• Population • Consists of all objects that I wish to describe• Census – survey of the population

• Sample• Subset of the population• Used to predict the population

Sample Survey

We often draw conclusions about a whole population on the basis of a sample.

Choosing a sample from a large, varied population is not that easy.

Choose a sample Survey

1. Define the population we want to describe.

2. Say exactly what we want to measure.*A “sample survey” is a study that uses an organized plan to choose a sample that represents some specific population

3. Decide how to choose a sample from the population

Current Population Survey• Contacts 60,000 households each month

• Produces monthly unemployment rate and a lot of other economic and social information

• Population is defined as all U.S. residents (legal or not) 16 years of age and over who are civilians and are not in an institution such as prison.

• Unemployed – if you are available for work and if you actually looked for work in the last 4 weeks.

Bad Sampling….

• Sample that does not represent the population.

Convenience Sampling

• Choosing individuals who are easiest to reach.• Example: Questioning the 1st 100 people to come to the store

• Example: Mall interview

• What problems can you see?

Studies that use convenience sampling generally have results that are suspect because the selection of individuals is not random.The results should be looked on with extreme skepticism.

Voluntary Response Sample

• Consists of people who choose themselves by responding to a general appeal.

• Those most likely to respond are the people with strong opinions, especially negative opinions.

• Television call in polls• FACT: Only about 15% of the public has ever

responded to a call-in poll. That is not a representative sample of the population as a whole!

Example• The American Family Association (AFA) is a conservative

group that stands for “traditional family values. It had a poll on its Web site – about same sex marriage in 2004. They had 60% of 850,000 people responded that they favored same-sex marriage. This did not support AFA’s position. What do you think happened?

What type of Sampling• A farmer brings a juice company several crates of oranges

each week. A company inspector looks at 10 oranges from the top of each crate before deciding whether to buy all the oranges.

Convenience Sampling:This could lead him to think that the oranges are of better quality than they really are, if the farmer puts the best oranges on the top.

What type of Sampling• The ABC program Nightline once asked whether the

United Nations should continue to have its headquarters in the United States. Viewers were invited to call one telephone number to respond “Yes” and another for “No”. There was a charge for calling either number. More than 186,000 callers responded and 67% said “No”.

Voluntary Response Sample:In this case, those who are happy that the United Nations has its headquarters in the U.S. already have what they want and so are less likely to worry about responding to the question.

Activity #1• Guess the length of my string.

Activity #2 - M&M Rectangles• I want to know the average size of the rectangles on this

page.• Pick 10 rectangles that you believe are representative of

all of the on the page.• Find the area of each of the 10 you chose.• Find the average area of all 10

Bias

The sampling method is biased if it systematically favors certain outcomes.

Exam Tip: Always tell which way it is biased.

Ex: “Explain how using a convenience sample of students in your stats class to estimate the proportion of all high school students who own a GDC could result in bias.” You might respond: “This sample would probably include a much higher proportion of students with a GDC than in the population at large because a GDC is required for the stats class.” In other words, this method would probably lead to an overestimate of the actual population proportion.

3 Sources of Bias

• Selection Bias• Nonresponse Bias• Response Bias

Selection Bias• The method of selecting the sample systematically

excludes some part of the population of interest.

• Example – the M&Ms lab

voluntary response

Measurement / Response• Method of observation tends to produce values that

systematically differ from the true value in some way.

• Measurement – improperly calibrated scale, the string activity

• Response – Improperly worded questions

Example: “Should illegal immigrants be prosecuted and deported for being in the U.S. illegally, or shouldn’t they?” 69% favored deportation.Vs.“Should illegal immigrants who have worked in the U.S. for two years be given a chance to keep their jobs and eventually apply for legal status?” 62% favored allowing them to stay.

Response Bias - More• Response bias occurs when the answers on a survey do not reflect the true feelings of the respondent. It can occur in many different ways.

• Interviewer Error• Misrepresented

Answers• Wording of Questions• Ordering of Questions

or Words• Type of Question• Data-Entry Error

Interview ErrorAn interviewer should be trained to be able to get truthful responses from people. If people don’t feel they can trust the interviewer they will give questionable answers.

Also, be aware of interviewers who have a vested interest in the results of the survey.

Ex: Would you trust a survey conducted by a car dealer that reports 90% of customers say they would buy another car from the dealer?

Misrepresented AnswersSome survey questions result in responses that misrepresent facts or are flat-out lies.

Ex: A survey of recent college graduates may find their self-reported salaries are inflated.

Also, people may overestimate their abilities.

Ex: Ask people how many pushups they can do in 1 minute and then ask them to do the pushups. How accurate were they?

Wording of Questions

Balanced

Questions should be asked in a balanced form to prevent bias.

Ex: The yes/no question: “Do you oppose the reduction of estate taxes?”

Should be changed to

“Do you favor or oppose the reduction of estate taxes?”

Not Too Vague

Another consideration in wording a question is not to be vague.

Ex: “How much do you study?” is too vague.

Should be changed to

“How many hours do you study statistics each week?”

Ordering of Questions or WordsMany surveys will rearrange the order of the questions or words within a questionnaire so that responses are not affected by prior questions.

Ex: The Gallup organization routinely asks the following question of 1017 adults aged 18 years or older:

Do you (rotated) approve or disapprove of the job Barack Obama is doing as president?

The words approve and disapprove are rotated to remove the effect that may occur by writing the word approve first in the question.

Nonresponse Bias• Responses are not actually obtained from all individuals

selected for inclusion in the sample.

• Example – Failure to return polls

Nonresponse occurs when an individual chosen for the sample can’t be contacted or refuses to participate.

All surveys suffer from nonresponse biasNon response bias can be controlled by 1) using callbacks 2) using rewards a) Cash for completing survey b) Incentives that state responses have an impact on future policy.

What type of Bias• Bill is assigned by his editor to determine what most

Americans think about a new law that will place a federal tax on all modems and computers purchased. The revenues from the tax will be used to enforce new online decency laws. Bill, being technically inclined, decides to use an email poll. In his poll, 95% of those surveyed opposed the tax. Bill was quite surprised when 65% of all Americans voted for the taxes.

SELECTION BIAS:Excluded those people NOT technically inclined!

What type of Bias

• The United Pacifists of America decide to run a poll to determine what Americans think about guns and gun control. Jane is assigned the task of setting up the study. To save mailing costs, she includes the survey form in the group's newsletter mailing. She is very pleased to find out that 95% of those surveyed favor gun control laws and she tells her friends that the vast majority of Americans favor gun control laws.

SELECTION BIAS:Pacifists will most likely favor gun control laws.A proportion of the population was left out!

What type of Bias• Large scale polls were taken in Florida, California, and

Maine and it was found that an average of 55% of those polled spent at least fourteen days a year near the ocean. So, it can be safely concluded that 55% of all Americans spend at least fourteen days near the ocean each year

Selection Bias:The states chosen for the polls had easier access to the ocean. A large part of the population was left out.

Simple Random Sample.• Good sampling technique.

• A simple random sample (SRS) of size n consists of n individuals from the population chosen in such a way that every set of n individuals has an equal chance to be the sample actually selected.

Let’s look at those M&M rectangles again!

• Let’s randomly pick 10 rectangles.

• Press • Math• Prb• RandInt (1,100,10)

• Find the area of the ten rectangles that match the numbers you generated.

Choosing a SRS…using a Random Number Chart

• Be specific about how you select.

Ex: I’m going to start with line 100 and pick

two digits going across the row. The number will

represent the group number I will sample.

• Indicate the stopping rule.

Ex: I will stop this process when I have found 10

samples.

• Tell whether you sample with or without replacement.

Ex: I do not want to repeat numbers because I need 10

distinct groups therefore, I will sample without

replacement.

• Use labels to identify subjects selected to be in the sample.Ex:

Sam

pling and Surveys

• How to Choose an SRS

Step 1: Label. Give each member of the population a numerical label of the same length.Step 2: Table. Read consecutive groups of digits of the appropriate length from Table D.Your sample contains the individuals whose labels you find.

How to Choose an SRS Using Table D

Definition:A table of random digits is a long string of the digits 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 with these properties:

• Each entry in the table is equally likely to be any of the 10 digits 0 - 9.• The entries are independent of each other. That is, knowledge of one part of the table gives no information about any other part.

We are planning an article on family-friendly places to stay over spring break at a nearby beach town. The editors intend to call 4 randomly chosen hotels to ask about their amenities for families with children. They have an alphabetized list of all 28 hotels in the town.

01 Aloha Kai 08 Captiva 15 Palm Tree 22 Sea Shell

02 Anchor Down 09 Casa del Mar 16 Radisson 23 Silver Beach

03 Banana Bay 10 Coconuts 17 Ramada 24 Sunset Beach

04 Banyan Tree 11 Diplomat 18 Sandpiper 25 Tradewinds

05 Beach Castle 12 Holiday Inn 19 Sea Castle 26Tropical Breeze

06 Best Western 13 Lime Tree 20 Sea Club 27Tropical Shores

07 Cabana 14 Outrigger 21 Sea Grape 28 Veranda

69051 64817 87174 09517 84534 06489 87201 97245 88221 22356 77183 88725

Sam

pling and Surveys

69051 64817 87174 09517 84534 06489 87201 97245

69 05 16 48 17 87 17 40 95 17 84 53 40 64 89 87 20

Our SRS of 4 hotels for the editors to contact is: 05 Beach Castle, 16 Radisson, 17 Ramada, and 20 Sea Club.

Activity – Table T5

• Which rectangle would you choose if you used the random number table? (Table D – back of the book)

• How would you do it?

Homework• Page 226 (1-11) odd• Worksheet – Bias