The value of parliamentary data

The UK Parliaments data drive in 2016Friday lunchtime lecture @ ODI22nd January 2016Zeid HadiSo who is Zeid ?Zeid Hadi- Development manager / delivery manager / Scrum MasterWorking with Parliament since June 2013 and data.parliament since November 2013Responsible for Hansard re-development as well as other applicationsReed Elsevier data Subscription Services, CTO of my own software House, and now going back to my roots in data publishing

The TeamCross functional multi-disciplined teamC# developers Wojciech, James, FaisalTesters Thulasi and Rodrick Business Analyst Alan & AlexProject Managers (Sayed / Aidan)

What is data.parliamentA collection of services enabling Parliament to share its data both within and outside of Parliament.Was created in response to a growing demand for open dataMost of the data is already public, but the platform is about ensuring that it is exposing this data usefully.In time, we hope that all published Parliamentary information will be available on data.parliament

Why is it important ?We hope that this will encourage public understanding of the work of Parliament and enable citizens to find out more easily about the work Members of both Houses of Parliament have been doing on their behalf.

Why is it important ?Allows the public to search and download data from Parliament.Gives the ability to analyse, link, query the dataIs used byJournalistsEducatorsDevelopersSocial BusinessesLobbyistsUs. Parliament itself re-using the data to power parliament.uk.

data.parliament servicesData Directory data.parliament.ukCKAN same technology as data.govContains a listing of our datasets and links to resourcesSearch explore.data.parliament.ukDirectly query data in data.parliamentDownload dataAPI lda.data.parliament.ukbuild highly functional data-driven applicationsAnalyse and interpret the dataLink data to other open data sources such as DBpedia.Open DataAll data is available as uniquely identifiably URIs

data.parliamentDEMOhttp://data.parliament.ukhttp://explore.data.parliament.uk

How we built data.parliament

http://data.Parliament.uk http://explore.data.parliament.ukhttp://lda.data.parliament.uk

ELDA APIGraph DB TriplestoreTransformation Engine > RDFNo SQL Repository(xml, json, images, text, pdf, html) AppAppAppAppAppAppProcessOver 50 business centric applications produce data.Pull, publish or upload data into data.parliamentTransform the data into RDF and link that data to other datasets.Load the data into the triplestore.Expose the data from the triplestore using Linked Data API.

Lets see some demonstratorsDemo of Busy Bees and Petitions

What about real world applications?A growing number of applications use data.parliament.Written Questions & Answers ApplicationResearch BriefingsPapers Application (Statutory Instruments)Lords Amendments SystemData.parliament is the integration platform for parliamentary data.ChallengesSpreading the message of open data throughout ParliamentContinuous trainingPut in place several basic principles e.g. all projects will use or publish open data by default.

ChallengesWhat does this data mean?Meaning is locked in the applicationIdentifying data owners and processesCorrecting dataPutting in place a model by which data can be reviewed, maintained and governed.ChallengesCreating our Semantic OntologyWhere do you start ?Borrow from other ParliamentsBig Bang Upfront / as you load in new data

ChallengesTechnologyRelatively niche technologiesSPARQL - eh?Performance of our triplestoreHardwareChallengesDeliveryWe managed to load / convert and transform over 50 datasets over 2 yearsBut have only delivered under half of these into a live production environment Live DatasetsCommons Oral Questions and Question TimesQ&A Daily Report (Commons)ThesaurusAnswering BodiesPapers LaidCommons DivisionsCommons MembersEarly Day MotionsLords MembersSessions DataCommons Written StatementsLords Written QuestionsLords Written StatementsCommons Written QuestionsConstituenciesBriefing PapersElectionsLords DivisionsBills (Metadata)Lords Registered InterestsQ&A Daily Report (Lords)Datasets waiting to go liveCommons Select CommitteesTerms EnhancementCommittee Written EvidenceAv Live LoggingCommittee Oral Evidencee-petitionsDeposited PapersHansard Lords DebatesLords AttendanceHansard Commons DebatesLords Items of BusinessHansard Commons Written Ministerial StatementsLords Oral QuestionsHansard Lords Written Ministerial StatementsMembers Name DBPedia EnhancementHansard Commons General CommitteesPrivate Bills WitnessesHansard Lords Grand CommitteesPublic PetitionsHansard Commons Westminster HallEuropean ScrutinyMPs Registered Interests

Hansard Commons PetitionsHansard Commons CorrectionsMPs Registered Interests

Strategy going forwardOpen by default from the very startContinue to work with our colleagues to educate on open dataContinue to work with the public to understand what and how to deliver dataEat our own dog food with the redesign of our own website using data from data.parliament.Strategy going forwardContinue to work on delivering the data in useful formats for non-technical audiences. Bridging the gap between search and dataExpose the value of our data that is currently hidden in the publication of reportsComplete the release of all our remaining datasets.

Questions? Useful linkshttp://data.parliament.ukhttp://blog.data.parliament.ukhttp://explore.data.parliament.ukhttp://lda.data.parliament.uk