first they have to find it: getting open government data discovered and used
Upload: tetherless-world-constellation-rensselaer-polytechnic-institute
Post on 21-Jun-2015
531 views
TRANSCRIPT
![Page 1: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/1.jpg)
First they have to find it: Getting Government Data Discovered and Used
John S. Erickson, Ph.D.Tetherless World ConstellationRensselaer Polytechnic InstituteTroy, New York, USA
Twitter: @olyerickson #TWCRPI
Panel: The Art & Science of Data Visualization#IOGDC
![Page 2: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/2.jpg)
Open Government Data Around the World
2
Starting with efforts in the US and UK, governments around the world have recognized the need to publish their critical data
Percent of total collection (from 1M+ datasets)
![Page 3: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/3.jpg)
Diverse Approaches to Open Gov't Data
3
Government data initiatives have taken many forms
GovData portals are widely varied in how they help users discover and use relevant datasets
Percent of total catalogs(from 192 catalogs)
![Page 4: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/4.jpg)
Federated Discovery of Government Data
4
Stakeholders have seenthe need for
Federated discoveryacross catalogs,
especially from withinmajor search engines
includingBing, Google, Yahoo!
and Yandex
![Page 5: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/5.jpg)
Linked Data is Not Enough...
5
• Publishing open government data as Linked Data is not enough
• For OGD to be useful, datasets must be published using metadata, markup standards and presentation that aid discovery and use
![Page 6: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/6.jpg)
Linked Data is Not Enough...
6
• Publishing open government data as Linked Data is not enough
• For OGD to be useful, datasets must be published using metadata, markup standards and presentation that aid discovery and use
![Page 7: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/7.jpg)
Dataset Metadata for Discovery and Use
7
Recent work at TWC RPI demonstrates
the value of applying emerging standards for
uniformly describing government datasets
and catalogs
![Page 8: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/8.jpg)
International Open Government Dataset Search
8
TWC's IOGDS application is an aggregated catalog of more than 1M datasets from over 192 dataset catalogs from governments at every level around the world
See: http://logd.tw.rpi.edu
![Page 9: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/9.jpg)
9
Anticipates W3C DCAT RDF vocabulary
Demos what a comprehensive federated catalog based on DCAT and aggregation API might look like
International Open Government Dataset Search
![Page 10: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/10.jpg)
10
IOGDS is a multi-year effort based on downloading, scraping or accessing APIs, converting metadata to a proto-DCAT model, and publishing via endpoint and download
International Open Government Dataset Search
API
Download
WebWebWeb
IOGDS WorkflowIOGDS Workflow
IODGSCSVPer-site
scrapercode
ad hoccode
Csv2rdf4lodautomation
10
Catalogs
See: http://logd.tw.rpi.edu
![Page 11: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/11.jpg)
Schema.org: Semantic Markup for Discovery
11
TWC RPI has published dataset listings based on IOGDS using emerging microdata standards, esp. schema.org model endorsed by Bing, Google, Yahoo!, Yandex...
![Page 12: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/12.jpg)
Schema.org datasets extension
12
• TWC RPI's schema.org dataset extension will enable government dataset catalogs to more easily be parsed and indexed by the major search engines...
• ...which will help users find relevant datasets!
• TWC's dataset extension entered public discussion June 2012
![Page 13: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/13.jpg)
Schema.org datasets extension
13
The schema.org datasets extension enables relevant datasets to be more easily discovered by a range of stakeholders including researchers, data journalists, bloggers and developers
![Page 14: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/14.jpg)
14
Schema.org datasets extension
“...we've reviewed the current datasets schema proposal in draft, and we are comfortable with the current state of things...
“...At this point, if the group would solidify on the dataset proposal, then Data.gov would support and use it.
---Chris Musialek
![Page 15: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/15.jpg)
CKAN Data Catalog Scheme & Protocol
15
API-based catalog federation is also possible
ckan announced DCAT-based query/federation API
enables OAI-PMH-like harvesting and more
![Page 16: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/16.jpg)
Other Thoughts...
Geo-based discovery: What data is available by geo-selection?
Provenance-based discovery: How do I get the data that someone else used? “Get the Data”
Community/social-based discovery: Dude, check out this data! (Linked Data perfect for this...
![Page 17: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/17.jpg)
Other Thoughts...
Geo-based discovery: What data is available by geo-selection?
DATA.GOV Geo Viewer
![Page 18: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/18.jpg)
Other Thoughts...
Community/social-based discovery: Dude, check out this data!
OPENEI.org
![Page 19: First they have to find it: Getting Open Government Data Discovered and Used](https://reader034.vdocuments.site/reader034/viewer/2022052412/55859cb4d8b42abc7b8b50de/html5/thumbnails/19.jpg)
19
Choose your own medicine...
but do expose your metadata
and get your catalogs discovered!