election polls in the 21 st century: can they be trusted in the era of cell phones? league of women...
TRANSCRIPT
![Page 1: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/1.jpg)
ELECTION POLLS IN THE 21ST CENTURY:
CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES?
League of Women Voters Lawrence Township
![Page 2: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/2.jpg)
![Page 3: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/3.jpg)
First Statistical Associations
• Statistical Society of London (1834)
• American Statistical Association (1839)
• International Statistical Institute (1885)
Sir Ronald Fisher and Prasanta Mahalanobis
![Page 4: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/4.jpg)
Anders Kiaer and the Norwegian Census of 1891
Source: Kiaer, Anders N. 1897 “The Representative Method Of Statistical Surveys”, p. 44.
![Page 5: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/5.jpg)
Fisher’s Agricultural Research
• Use of repeated measurements to estimate crop yields (1919)
• Development of randomized experiments, hypothesis testing and tests of statistical significance
• Publishes Statistical Methods for Research Workers in 1925
![Page 6: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/6.jpg)
Gini, Galvani and the Italian Census of 1921
• Need to make room for the 1930 census• Chose 29 of 216 census districts• Averages computed across districts
don’t match population numbers• Both purposive and random sampling
considered inferior to full census tabulations
Source: Feinberg, Stephen E. and Judith M. Tanur. 1995. “Reconsidering Neyman On Experimentation And Sampling: Controversies And Fundamental Contributions.” Probability and Mathematical Statistics Vol. 15, pp. 47-60.
![Page 7: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/7.jpg)
The Impact of the Great Depression• Unemployment clearly a massive problem
• Questions on work developed from 1880 through 1930 censuses
• Population measurements needed between decennial censuses
• Trial Census of Unemployment 1933-34• Enumerative Check Census of 1937• Leads to the Current Population Survey
Source: Card, David. 2011. “Origins of the Unemployment Rate: The Lasting Legacy of Measurement Without Theory” http://davidcard.berkeley.edu/papers/origins-of-unemployment.pdf
![Page 8: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/8.jpg)
1936 Presidential Election• Literary Digest Poll
– 2 million responses– forecasts Republican
victory• Gallup Poll
– 300,000 responses– forecasts
Democratic victory• Roosevelt gets 67% of
the popular vote, 98% of electoral votes
![Page 9: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/9.jpg)
1948 Presidential Election• Gallup Poll
– Use of quota sampling
– 3,250 responses– forecasts Dewey
victory• Truman gets 49.5% of
the popular vote, 62% of electoral votes
• End of quota sampling• Campaign dynamics
![Page 10: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/10.jpg)
American Association for Public Opinion Research
• Founded in 1947• More than 2,200 public
opinion scientists, survey researchers and statisticians are members
• Code of Professional Ethics• Transparency Initiative
![Page 11: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/11.jpg)
Average Errors in Presidential Polls 1948-2008Year # of Polls # of
CandidatesAverage Error
(%)
2008 23 2 1.4
2004 19 2 1.7
2000 19 3 1.7
1996 9 3 1.7
1992 6 3 2.2
1988 5 2 1.5
1984 6 2 2.4
1980 4 3 3.0
1976 3 3 1.5
1972 3 2 2.0
1968 2 3 1.3
1964 2 2 2.7
1960 1 2 1.0
1956 1 2 1.8
1956-2008 (Avg) 1.8
1948 3 3 4.9
Source: Traugott, Michael. 2005. “The Accuracy of the National Preelection Polls” Public Opinion Quarterly 69(5): 649
![Page 12: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/12.jpg)
The 2000 Election: What Went Wrong?
• “If we say somebody’s carried a state, you can pretty much take it to the bank.” – Dan Rather
• Competitive pressures on networks to call the outcome in Florida first
• Voter News Service has problems with following procedure for sampling and interviewing exiting voters and with computer models processing incoming results
![Page 13: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/13.jpg)
Accuracy of the 2008 Polls
Source: http://electoralmap.net/pollsters/popular.txt
![Page 14: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/14.jpg)
Identifying the Likely Voter
![Page 15: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/15.jpg)
Likely Voter Questions
How much thought have you given to the coming presidential election?
How closely have you been following news about the candidates?
Do you plan to vote in the presidential election? How certain are you that you will vote?
Rate your chance of voting in November on a scale of 10 to 1, with 10 being "definitely will vote" and 1 "definitely will not vote.“
How often do you follow what's going on in government and public affairs?
Have you ever voted in your precinct or election district?
How often would you say you vote?
Do you happen to know where people in your neighborhood go to vote?
In the last election, did things come up that kept you from voting or did you vote?
Source: Ask the Expert: Determining Who Is a 'Likely Voter' Scott Keeter, Director of Survey Research August 29, 2012
![Page 16: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/16.jpg)
Problems with Polling in 2012
Source: http://electoralmap.net/pollsters/popular.txt
• Declining Response Rates• Growth of the “Cell-only” population• Shifting attention to state polls after 2000• Increasing popularity of survey by “robo-
call”
![Page 17: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/17.jpg)
Declining Response Rates
Source: http://www.people-press.org/files/legacy-pdf/Assessing%20the%20Representativeness%20of%20Public%20Opinion%20Surveys.pdf
![Page 18: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/18.jpg)
The Growing “Cell-Only” Population
Source: http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201206.pdf
![Page 19: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/19.jpg)
Dual Frame Estimation
Source: http://pewresearch.org/pubs/1601/assessing-cell-phone-challenge-in-public-opinion-surveys
Summary of Differences Between Dual Frame and Landline Estimates
![Page 20: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/20.jpg)
Surveys by Robo-call
• Understanding the math behind a surveySample
size CostMargin of
errorFixed cost 5000
200 $10,000 6.9%Unit cost 25
400 $15,000 4.9%
600 $20,000 4.0%
800 $25,000 3.5%
1,000 $30,000 3.1%
1,200 $35,000 2.8%
1,400 $40,000 2.6%
1,600 $45,000 2.5%
1,800 $50,000 2.3%
2,000 $55,000 2.2%
3,000 $80,000 1.8%
5,000 $130,00
0 1.4%
![Page 21: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/21.jpg)
Rasmussen’s Methodology
“While we do not currently call cell phones directly when conducting our surveys, to reach those who have abandoned traditional landline telephones, Rasmussen Reports uses an online survey tool to interview a demographically diverse panel. Just like our telephone surveys, respondents for this approach are selected on a random basis to insure the reliability of the sample. As a result, you cannot sign up to participate. “
![Page 22: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/22.jpg)
Aggregating State Polls
• Nate Silver’s 538• RealClear Politics• Princeton Election Consortium• PollTracker• Dave Leip's Atlas of U.S. Presidential Elections
![Page 23: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/23.jpg)
RealClear Average
![Page 24: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/24.jpg)
Nate Silver/ NY Times
![Page 25: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/25.jpg)
Princeton Election Consortium
![Page 26: ELECTION POLLS IN THE 21 ST CENTURY: CAN THEY BE TRUSTED IN THE ERA OF CELL PHONES? League of Women Voters Lawrence Township](https://reader038.vdocuments.site/reader038/viewer/2022103121/56649c8b5503460f949452de/html5/thumbnails/26.jpg)
Important Trends
• Early voting• Voter ID movement/Voter fraud issue
– Demographic changes in the US• Increasing automation/improving robo-call
technology• More web-based surveying