PRE-ELECTION POLLS IN 2012

Clyde Tucker

Outline

• Review of the Polls

• Issues in Pre-election Polls

• Electoral Choice in Survey Research

MoE Obama Romney Spread

Final Result 50.9 47.3 +3.6 D

RCP Average 10/31-11/5 48.8 48.1 +0.7 D

Politico/GWU 3.1 47 47 Tie

Rasmussen 3.0 48 49 +1.0 R

IBD/TIPP 3.7 50 49 +1.0 D

CNN/OR 3.5 49 49 Tie

Gallup 2.0 49 50 +1.0 R

ABC/Wash. Post 2.5 50 47 +3.0 D

Monmouth/Braun 2.6 48 48 Tie

NBC/Wall St J 2.6 48 47 +1.0 D

Pew Research 2.2 50 47 +3.0 D

State RCP Average Spread Final Result Spread

Obama Romney Obama Romney

OH 50.0 47.1 +2.9 D 50.7 47.7 +3.0 D

FL 48.2 49.7 +1.5 R 50.0 49.1 +0.9 D

VA 48.0 47.7 +0.3 D 51.2 47.3 +3.9 D

NH 49.9 47.9 +2.0 D 52.0 46.4 +5.6 D

NC 46.2 49.2 +3.0 R 48.4 50.4 +2.0 R

MI 49.5 45.5 +4.0 D 54.2 44.7 +9.5 D

WI 50.4 46.2 +4.2 D 52.8 45.9 +6.9 D

PA 49.4 45.6 +3.8 D 52.0 46.6 +5.4 D

IA 48.7 46.3 +2.4 D 52.0 46.2 +5.8 D

CO 48.8 47.3 +1.5 D 51.9 46.1 +5.8 D

NV 50.2 47.4 +2.8 D 52.4 45.7 +6.7 D

Kaine Allen Spread

VA Final Result 52.5 47.5 +5.0 D

RCP Avg. 48.6 46.8 +1.8 D

Warren Brown Spread

MA Final Result 53.7 46.3 +7.4 D

RCP Avg. 50.0 47.0 +3. 0 D

Heitkamp Berg Spread

ND Final Result 50.5 49.5 +1.0 D

RCP Avg, 43.3 49.0 +5.7 R

Tester Rehberg Spread

MT Final Result 48.7 44.8 +3.9 D

RCP Avg. 47.3 47.7 +0.4 R

Issues in Pre-election Polls

• Cell Phone Only Households– At least a third of households are cell only

– Increases to about half when “cell mostly”

– Excluding cell-only households restricts access to young voters

• 22% of adults in the Census are 18-29 years old

• 0nly 6% of respondents are 18-29 in Pew landline samples

• Increasing Nonresponse– Over the last two decades response rates n

telephone surveys have been in rapid decline

– Contributing to this decline have been the following

• A growing number of single-person households

• Changing technologies—answering machines, caller ID and the decline in the number of households with a landline

8www.people-press.org

Steep Decline in Response Rates Over the Past 15 Years

• Rates calculated using AAPOR’s CON2, COOP3 and RR3. Rates are typical for Pew Research surveys conducted in each year. From “Reflections on Election Polling and Forecasting from Inside the Boiler Room,” Prepared for the CNSTAT public seminar by Scott Keeter.

• ”

– Peter Miller, a former President of AAPOR, said trying to weight away nonresponse can create unknown errors

– Even if the demographics of telephone respondents are similar to the voting age population as a whole, there can still be substantial differential nonresponse

Pew Research Standard

surveyGovernme

ntsurveys

In the past year… % %

Volunteered for an organization

55 27

Contacted a public official 31 10

Talked with neighbors weekly or more

58 41

10 From “Reflections on Election Polling and Forecasting from Inside the Boiler Room,” Prepared for the CNSTAT public

seminar by Scott Keeter.

www.people-press.org

But Sizeable Differences in Civic and Political Engagement

– The biggest problem, of course, would come from differential nonresponse by vote choice

– There has been evidence of this in exit polls

– However, one saving grace may be that nonrespondents are less likely to vote

– This is probably true given the lack of civic engagement

• Determining Likely Voters– Likely voter screens are usually based on

answers to a series of questions, including political interest, past voting history, length of time in the community, and likelihood of voting

– The likely voter screen is used either to exclude respondents with a low score or to assign a probability of voting to each respondent, often aided by information from voter validation studies.

– But, what is the result?

– In his recent presentation at NAS, Scott Keeter provided evidence that using a likely voter screen resulted in a difference between the candidates that was closer to the final result in the 2008 Presidential Election

– Also, as we already have seen, the difference between the candidates among likely voters in the final 2012 Pew poll was very close to the actual result

– Yet, this is not always the case

– Registered voter numbers should exhibit less variance than likely voter numbers

– Begin with a baseline for both registered voters and likely voters shortly before the conventions

– Track both series up until election day

– Examine the variance of the likely voter series, determining which screener questions contribute most to the variance

– Carry out this exercise over several elections

• Allocating Undecideds– There are essentially three ways to allocate the

undecideds• Evenly divide them between the candidates (most

often the two leading ones)

• Assign undecideds proportionately based upon the proportions for each candidate among the decided voters

• Assign an undecided voter based on the characteristics of the voter (models will differ depending on the analyst)

– Again, the correct allocation is probably dependent on the specific election context

An Example from Nate SilverPresidential Vote in Virginia

Obama Romney

Polling Avg. 48.2 46.9

Adj. Polling Avg. 48.8 46.7

State Fundamentals 48.2 47.4

Now Cast 48.8 46.8

Overall Avg. 48.5 47.4

Projected Vote 50.7 48.7

Allocated Undecideds 2.2 1.3

• House effects– Political bias—incorporated at any stage

– Methodological sophistication

– Funding

– Mode • Interviewer conducted (dual frame or only landline)

• Robo calls

• Internet polls

The Tyranny of the Electoral Choice in Survey Research

• The importance of electoral choice– Very important behavior--the best known product of survey

research (unemployment rate probably in second place)

– A gold standard is available

• The problem with electoral choice– Only one of many products of survey research

– Binary choice

– Constrained distribution• At least 40% (probably more now) can be initially assigned to

each party

• In poll with 1000 respondents, margin of error (+/- 3%) covers a good part of the remaining difference between the candidates

An Interesting Alternative to a National Presidential Poll

• Using state-based models to predict the electoral vote margin

• Two methods offered by Nate Silver at the New York Times and Drew Linzer at Emory University

• Both correctly predicted the electoral vote margin relying, in part, on averages of polls in individual states

• Silver’s method combines an overall measure of economic performance, state polls, and recent state electoral history to forecast each state race

• He then uses these estimates to determine the most likely electoral vote margin

• Although the economic measure and past state voting history have a great effect early in the campaign season, they decline in importance relative to the polls as the election draws near

• Linzer’s method starts with a prior based on a modified structural forecast and then relies on only the state polls and the recent state electoral history in a process he calls Dynamic Bayesian Forecasting

• This is a hierarchical model that borrows strength from trends in the frequently polled states to make estimates of states with little polling that also incorporates random-walk priors

• As in Silver’s model, electoral history takes a back seat to the polls in the model as the election approaches

• Both methods rely on simulations of model predictions of the electoral outcome to establish a probability distribution for determining the chance that each candidate has of winning