government legislative data, the other great white fail whale & how to avoid it oscon 2011
TRANSCRIPT
![Page 1: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/1.jpg)
Government Legislative Data, The Other Great White Fail Whale & How To Avoid It
http://vimeo.com/3841614
OSCON 2011
![Page 2: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/2.jpg)
Presenters
Jared WilliamsNew York State Senate
Graylin KimNew York State Senate
Noel Hidalgo, aka @noneck
OSCON 2011
![Page 3: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/3.jpg)
History of Open Government at NY State Senate
OSCON 2011
![Page 4: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/4.jpg)
OSCON 2011
![Page 5: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/5.jpg)
OSCON 2011
![Page 6: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/6.jpg)
OSCON 2011
![Page 7: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/7.jpg)
OSCON 2011
![Page 8: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/8.jpg)
OSCON 2011
![Page 9: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/9.jpg)
OpenLeg "Yahoo Edition"
OSCON 2011
![Page 10: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/10.jpg)
OpenLeg Mobile Edition
OSCON 2011
![Page 11: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/11.jpg)
OpenLeg v2 prototype
OSCON 2011
![Page 12: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/12.jpg)
Open Legislation v2 "Google Edition"
OSCON 2011
![Page 13: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/13.jpg)
OSCON 2011
![Page 14: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/14.jpg)
OSCON 2011
![Page 15: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/15.jpg)
How does it work?
OSCON 2011
![Page 16: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/16.jpg)
OSCON 2011
![Page 17: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/17.jpg)
Pull - How we get the data
2011S00607B402/01/11 ADVANCED TO THIRD READING
OSCON 2011
![Page 18: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/18.jpg)
Bill Structure
Process
OSCON 2011
![Page 19: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/19.jpg)
Process
We needed a buffer between pulling and pushing:• data quality• accountability
Flat files are convenient:• backups• human read/editable
It also opens the door for a few good ideas:• firehose• version control
OSCON 2011
![Page 20: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/20.jpg)
Push
OSCON 2011
![Page 21: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/21.jpg)
Publish
OSCON 2011
![Page 22: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/22.jpg)
OSCON 2011
![Page 23: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/23.jpg)
Story Time
OSCON 2011
![Page 24: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/24.jpg)
Infinite Beta
Data Quality issues cropped up in many places
Users noticed this (and we fight for the users)• internally steps towards greater senate integration were
minimized until openleg could be proven acurate• externally users praised the idea but were clearly frustrated
with the quality of the service
We're serving NYS legislative data• bad data during initial development wasn't great but seen as
acceptable• 1.5 years after it's initial release people were getting worried
OSCON 2011
![Page 25: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/25.jpg)
The Road Home
We had to be able to guarantee accuracy and rebuild our integrity• LBDC is the expert with this data and they've been a great
resourceo meet on a regular basiso set up a biweekly, automated diff process
• Created a simple data quality dashboard• Mutual gain from these mechanisms• In the beginning: nearly 1000 documents with issues• Currently we're hovering at ~20 known issues
o out of ~18K bills that isn't too bad!
OSCON 2011
![Page 26: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/26.jpg)
???
OSCON 2011
![Page 27: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/27.jpg)
Profit!
Just kidding. Its an Open Source project for the public good.
Seriously though. Where does that leave us?
To start the public profits from the service being:• free• near-real time• linkable• a primary source of information
Integrates with internal tools and has potential for use in external tools
OSCON 2011
![Page 28: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/28.jpg)
A Place to Disqus
We use Disqus for comments• People trust this: commenters are knowledgable, passionate
and dedicated• 3613 comments• Popular bills:
o S7234 - 136 commentso S2994 - 99 commentso A9529 - 97 comments
(as of 7/22/11)
Created an aggregate service: BillBuzz• Follow member's legislation• receive updates with new comments• informative for constituents, GREAT for senators/staffers
OSCON 2011
![Page 29: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/29.jpg)
Powerful Search API
Voting Correlations:
aye:(senX AND senY) AND nay:(senZ)
Bills Passed
actions:signed AND sponsor:senX
Bills Abandoned
stricken:true AND full:term
Null values
otype:bill AND NOT summary:[A* TO Z*]
OSCON 2011
![Page 30: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/30.jpg)
Open Legislation on the Wild
• Increasing Usage
• Google likes it (#1 source of traffic)
• Linked from various different placeso twittero facebooko NYTimes
OSCON 2011
![Page 31: Government Legislative Data, The Other Great White Fail Whale & How To Avoid It OSCON 2011](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513ffd8550346d8488b4850/html5/thumbnails/31.jpg)
Thank you...
Now, Questions and Answers...
OSCON 2011