open data publishing pipeline (odpp) richard cyganiak … · the problem visualisation/...
TRANSCRIPT
Open Data Publishing Pipeline (ODPP)
Richard Cyganiak DERI, NUI Galway SEMIC Conference 2013
The Problem
Data wranglers
deliver new value
from existing data.
The Problem
Visualisation/
Infographics
Mobile App
Stats package
Publish to
Website
Reporting
Open Data
Data wranglers
deliver new value
from existing data.
Spreadsheets
Legacy DB
The Problem
Log files
Biz Apps
Cloud APIs
Open Data
Visualisation/
Infographics
Mobile App
Stats package
Publish to
Website
Reporting
Open Data
Spreadsheets
Legacy DB
The Problem
Log files
Biz Apps
Cloud APIs
Open Data
Visualisation/
Infographics
Mobile App
Stats package
Publish to
Website
Reporting
Open Data
The V's of Big Data
2. Velocity
1. Volume
3. Variety
Catalogs Standards
Community Tools
• Namespace registry for
RDF/SPARQL users
• Side effect: Create
database of vocabularies
DDI Discovery Vocabulary
VoID — Vocabulary of Interlinked Datasets
DCAT — Data Catalog Vocabulary
Metadata standards
Data Cube Vocabulary
D2RQ
Database-to-
RDF mapping
RDF Export for
Google Refine
RDF Extension for Refine
Neologism
RDF Schema
Vocabulary
Editor and
Publishing
System
Great stuff, but…
Great stuff, but…
• Many separate tools that require setup and have their own learning curve
• Command-line interfaces, complex languages
• Web server configurations for publishing data and vocabularies according to best practices
• Difficult to transfer this knowledge to our partners
This is still a full-time job!
ODPP: Open Data Publishing Pipeline
Automated discovery
Catalogue
Reformat
Standardise
Publish
ODPP
Open Data Publishing Pipeline
•Web-based data publishing platform
•Map input data (relational, CSV, Excel) to semantic standards
•Visual mapping editor and vocabulary search, based on UML
ODPP
Open Data Publishing Pipeline
•Already knows all (?) semantic standards that have been published as RDF Schemas
•Special support for generating SKOS, Data Cube and Schema.org
•Under the hood: proven open-source technologies
Public beta in Q3
Trials with LGMA, Galway and Fingal CCs
Under development since early 2013
ODPP Development
How would you use ODPP?
