electionaudits: a django app for good election auditing
DESCRIPTION
ElectionAudits: a Django App for Good Election Auditing. Neal McBurnett OSCON July 22 2009. Boulder County used open source code to audit its 2008 election!. Share the story, share the code, and get you all involved where you live. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/1.jpg)
ElectionAudits: a Django App for Good Election Auditing
Neal McBurnett OSCON July 22 2009
![Page 2: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/2.jpg)
Boulder County used open source code to audit its 2008 election!
![Page 3: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/3.jpg)
![Page 4: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/4.jpg)
Share the story, share the code, and get you all involved where you live.
![Page 5: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/5.jpg)
Questions
Clarifications? Anytime
“But what about...”? At the end
![Page 6: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/6.jpg)
Why audit?
![Page 7: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/7.jpg)
Elections can inspire us! South Africa 1994
Paper ballots, hand counted
![Page 8: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/8.jpg)
Or serve as a warning
Iran 2009
![Page 9: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/9.jpg)
US problems with elections:
black box voting systems
![Page 10: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/10.jpg)
Not just a problem with touch screen devices (DRE)
![Page 11: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/11.jpg)
Humboldt County 2008Paper ballots, optical scan
197 ballots deleted by Diebold/PremierWithout a trace
![Page 12: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/12.jpg)
Certified....
![Page 13: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/13.jpg)
Discovered later by Humboldt County Election Transparency Project audit
“Ballot Browser”(also open source Python code)
Kudos to Mitch Trachtenberg, brave Registrar of Elections Carolyn Cernich, and
colleagues
![Page 14: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/14.jpg)
Surprise!
Computers make mistakes
Sometimes whoppers!
![Page 15: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/15.jpg)
Growing movement to require paper ballots
Not doing well at looking at them....
Not often required or well done
![Page 16: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/16.jpg)
Goal:
software independence(Rivest & Wack)
via auditable paper records,good audits
![Page 17: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/17.jpg)
Open Source voting systems
Important!
![Page 18: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/18.jpg)
Good audits and clean chain of custody
Necessary
![Page 19: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/19.jpg)
Election Quality
FSF
Statue of liberty with floppy disk in her hand
![Page 20: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/20.jpg)
Election Integrity
Computer Scientists for Social Responsibility
Question Technology
![Page 21: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/21.jpg)
Boris Bazhanov's Memoirs of Stalin's Former Secretary - quote from Stalin
Loosely translated:
"I consider it completely unimportant who in the party will vote, or how; but what is
extraordinarily important is this — who will count the votes, and how."
![Page 22: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/22.jpg)
But what is an audit anyway?
![Page 23: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/23.jpg)
Compare system's reported results with the evidence
Take sample of detailed results by batchand compare to hand counts of those
batches
![Page 24: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/24.jpg)
Auditing steps`
Produce report by audit unit (precinct or batch)
Reconcile number of ballots in each unit Randomly select audit units to audit Count audit units by hand Compare results Escalate audit if statistical evidence isn't
good enough
![Page 25: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/25.jpg)
Report of Vote Counts by Audit Unit
Audit Unit Susan Nelson Under Over Total100 137 144 2 0 383101 77 68 1 0 247102 122 87 3 2 316103 98 102 1 1 305104 22 18 2 0 146105 103 140 1 0 349
Total 559 559 10 3 1746
![Page 26: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/26.jpg)
Optical scanners arrived
Era of trusting computers too much
Few audits
![Page 27: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/27.jpg)
Audit DREs without voter verified paper trail?
Can't do it....
Pushback – Can't Trust Computers!
![Page 28: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/28.jpg)
“But we're doing audits”
![Page 29: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/29.jpg)
Announcing “random selection” before results come out
![Page 30: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/30.jpg)
Using software to select random numbers
No more software independence....
![Page 31: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/31.jpg)
Wasting time auditing contestswith a single candidate
![Page 32: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/32.jpg)
Reports are by precinct
But often piles of paper aren't
- Mail in- Early voting- Provisional
![Page 33: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/33.jpg)
Colorado rescans and recounts just the selected batches of mail-in ballots
Not an audit
Just a tiny post-election test that is unrelated to the actual election results
![Page 34: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/34.jpg)
![Page 35: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/35.jpg)
But no state yet does an efficient, “best practices” audit
![Page 36: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/36.jpg)
Principles and Best Practices forPost-Election Audits (2008)
http://electionaudits.org/principles
League of Women VotersElection Audit Report (2009)
![Page 37: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/37.jpg)
Fixed percentage vs Risk-limiting audits
![Page 38: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/38.jpg)
Fixed percentage:
Wasteful focus on landslide contests
Little confidence for tight contests
![Page 39: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/39.jpg)
Significance of the results driven by how many batches you audit, not how many
ballots you count
Looking for incorrect vote counts
Don't care about total vote count for the sample
![Page 40: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/40.jpg)
20 samples out of 1000 batches much better than
2 samples out of 100 batches (2%)and easier than
20 samples out of 100 batches
More samples = more statistical significance
More audit units = smaller samples, less counting
![Page 41: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/41.jpg)
“Risk-limiting” audit chooses more audit units when margin of victory is small
Has a pre-specified minimum chance of requiring a full hand count whenever the
apparent outcome of the contest is wrong
![Page 42: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/42.jpg)
Trying to audit in Boulder since 2002
Obstacles, cluelessness, politics
![Page 43: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/43.jpg)
Elect new Clerk, Hillary Hall
3 good audits in 2008
The hard part - getting good data
![Page 44: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/44.jpg)
Hart InterCivic system
Precinct reports
But only 15% cast in precinct
70% mail-in, 15% early voting
![Page 45: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/45.jpg)
Solution: run cumulative report 500 times
Once for each batch
Subtract each report from the previous report to get batch sub-totals
![Page 46: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/46.jpg)
Like snapshots of election-night reporting through the night
But we want lots of them....
![Page 47: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/47.jpg)
Report of Cumulative Vote Countsby Audit Unit
Audit Unit Susan Nelson Under Over Total100 137 144 2 0 383101 214 212 3 0 630102 336 299 6 2 946103 434 401 7 3 1251104 456 419 9 3 1397105 559 559 10 3 1746
Total 559 559 10 3 1746
![Page 48: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/48.jpg)
Report of Vote Counts by Audit Unit
Audit Unit Susan Nelson Under Over Total100 137 144 2 0 383101 77 68 1 0 247102 122 87 3 2 316103 98 102 1 1 305104 22 18 2 0 146105 103 140 1 0 349
Total 559 559 10 3 1746
![Page 49: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/49.jpg)
ElectionAudits supports the Best Practices
Automates many steps of the audit
Enter the data, publish with statisticsRoll 15 dice, publish all the selections
To do: automate discrepancies, escalation
![Page 50: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/50.jpg)
Time to look at some real data
In a real audit
ElectionAudits in action!
One of the first and most extensiveBest Practices audits
![Page 51: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/51.jpg)
Features of ElectionAudits Imports standard election report files Produces auditable reports for the public. Protects voter anonymity by merging small audit
units Doesn't require that paper ballots be sorted into
piles by precinct Can produce batch reports from sequence of
cumulative reports Facilitates risk-limiting audits Verifiably pseudo-random dice + "Sum of Square
Roots"
![Page 52: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/52.jpg)
Future Plans Add Stark's proper statistical methods for risk-
limiting audits: deciding when discrepancies require escalation
Automate more steps Support more vendors: Sequoia, etc. Read and write Election Markup Language data Hopefully use it in Denver and elsewhere in 2009 Hopefully use it for Colorado's audit pilots in 2010
![Page 53: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/53.jpg)
Selecting batches to audit
Rivest's “Negexp”Probability proportional to size
![Page 54: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/54.jpg)
Rivest's Sum of Square Roots pseudo random number generator
Public can verify unpredictability of selections
![Page 55: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/55.jpg)
Django
Python
SQLite
lxml
![Page 56: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/56.jpg)
Rivest's varsize.py
![Page 57: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/57.jpg)
Ubuntu LinuxWindows
Mac OS X?etc.
![Page 58: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/58.jpg)
MIT license
![Page 59: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/59.jpg)
Hosted at Launchpad
RSS feed of announcements
Bug tracking
Team mailing list
Bzr
Blueprints that I can mentor
![Page 60: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/60.jpg)
Tip:
Django's Debug_toolbar for great debugging over the web
![Page 61: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/61.jpg)
Help wanted!
![Page 62: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/62.jpg)
Web presentation: css, layout
![Page 63: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/63.jpg)
Logo
![Page 64: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/64.jpg)
XML expertise, e.g. for reading and writing (and improving) Election Markup Language
![Page 65: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/65.jpg)
Database design (pretty simple!)
![Page 66: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/66.jpg)
Django/python insights
![Page 67: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/67.jpg)
Implement features
![Page 68: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/68.jpg)
Windows testing, installation, eggs and Django, etc
![Page 69: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/69.jpg)
Ask for 2008 precinct data in your county
Send it to me!
Try to parse it
Audit 2009 election
![Page 70: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/70.jpg)
Help getting auditing laws passed
![Page 71: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/71.jpg)
Biggest challenge: getting useful data out of election systems
![Page 72: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/72.jpg)
We're experts in interplay between security, privacy, transparency, and freedom
Remember Christine Peterson's challenge - pitch in!
![Page 73: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/73.jpg)
Many thanks to Philip Stark, John McCarthy, Mark Lindeman, Mark Halvorson, Ron Rivest, Crystal Christman, Hillary Hall,
Aaron D. Gerber, Mary Eberle, Holly Lewis, and the many other colleagues and friends
that helped.
![Page 74: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/74.jpg)
1:45 today: Open Source and Democracy - Creating transparent, trustworthy voting
systems
5:20 today: Hacking the Open Government
Sunlight Labs Hackathon 9-5 Tue-Thu, Room N
![Page 75: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/75.jpg)
Code is Law
Write our own procedures into practice by providing the code!
Gov 2.0
DIY => DIO
![Page 76: ElectionAudits: a Django App for Good Election Auditing](https://reader035.vdocuments.site/reader035/viewer/2022070418/568159a3550346895dc6fa98/html5/thumbnails/76.jpg)
Remember
Must look at our ballots! Audits not done much, or right Open Source audits seen in the wild! Lots of room to improve Open Source folks have great insights Please help out! http://launchpad.net/electionaudits