scotland's environment web data journey 2011-2015 dave watson, duncan taylor

Scotland's Environment Web

Data Journey 2011-2015

Dave Watson, Duncan Taylor

Session Outline• SEWeb data journey

– What has been encountered on that journey

• SEWeb as a data consumer– What do we do with the data?

• Five Star/Linked Data • SEWeb Data – what next?

Partners

Data Publication

Daughter Sites

INSPIREWMS

Eye on Earth

Gemini2,

Data Protection

Data Download Service

Scottish Government

Digital Stategy

Data Visualisation

Linked Data

National Security

SEWeb Data Journey

Partners Business as Usual

Environmental Data Portal?

Scotland’s Environment Web - Data Journey

Data Consumer

SEWeb Brand – Daughter Web Sites

Data at Source

Dataset Progress• ‘Data at Source’

– 55 WMS consumed by Map Viewer -> 239 Data Layers– 9 Rest Services consumed by Land Information Search (LIS) -> 39 Data

Layers– 10+?? Non spatial data consumed by Visualisation Tools

• Five Star /Linked Data– 68 SESO Data, 12 Water (SEPA WFD), 1 Site Conditioning (SNH)

• Data Holdings– Soils/Aquaculture Daughter Sites– Project Finder

What do we do with the data?

• Themed spatial maps• Advanced Maps• Visualisation Applications• Task Specific Applications• Linked Data Repository

Themed/Advanced Maps

Task Specific Maps – Land Information Search

Visualisation/Discover Data

# Available on the web (whatever format) but with an open licence, to be Open Data

# # Available as machine-readable structured data (e.g. excel instead of image scan of a table)

# # # as (2) plus non-proprietary format (e.g. CSV instead of excel)

# # # # All the above plus, Use open standards from W3C (RDF and SPARQL) to identify things, so that people can point at your stuff

# # # # # All the above, plus: Link your data to other people’s data to provide context

Why Linked Data? - 5 Star Model of Open Data

http://www.w3.org/DesignIssues/LinkedData.html

Linked Data Four Principles

1. Use URIs as names for things

2. Use HTTP URIs so that people can look up those names.

3. When someone looks up a URI, provide useful information, using the standards (RDF*, SPARQL)

4. Include links to other URIs so that they can discover more things.

State of Environment (SOE) – Linked Data Model

SOE(State of Environment)

soe:Chapter

consistsOf

soe:Topic

dct:Dataset

Metadata

describedBy

soe:State

State Of Environement(Linked Data)Graph Model

hasdataset

Essential|supporting

Importance

SOE – Implementation

Vocabulary/concept schemehttp://data.sepa.org.uk/def/soe Trial datahttp://data.sepa.org.uk/id/soe/chapters

SOE Data Linkages

Chapter Topic Dataset SEWEB

SOE Data Linkages

Chapter Topic=

national indicator

Dataset

European Indicator (SOE) EEA

relates to

SOE Data Linkages

Chapter Topic Dataset

Data view and download services

Data Provider

links to

Metadata

relates to

publishes

European Indicator (SOE)

SEWeb Data - What Next?• Continued Addition of Datasets• What’s in my Area? – Local Datasets/SEWeb Local• Scottish Government Digital Strategy – Data Portals• Graphical Data Models to support ‘State of

Environment’• Links to European Data Initiatives

Useful Links– SEWeb www.environment.scotland.gov.uk – Scottish Soils http://www.soils-scotland.gov.uk/ – Aquaculture http://aquaculture.scotland.gov.uk/– Linked Data Lab http://data.sepa.org.uk– SSDI http://scotgovsdi.edina.ac.uk/srv/en/main.home– INSPIRE http://inspire.ec.europa.eu/ – Water Classification Visualisation

http://www.environment.scotland.gov.uk/get_interactive/data_visualisation/water_body_classification.aspx

End of Presentation – Workshop Support Slides Follow

Linked Data Architecture

Repository

Relational Data

Consumers

Datasets.Related not Relational

Metadata

File Download

Linked Data

Bespoke Data Feed

Data Feed Future

Dataset Definition.Metadata

Cannot do any subsequent steps without this

definition. Business needs to define and prioritorise

Other Data Providers

INSPIRE

REPORTING SENSE 2/2015

Organisational,Eg EA,SG etc

SEPA Stakeholders

Public

Citizen Scientists

Data Ingestion

OntologiesVocabularies

DRIVERSSEPA

Architecture

Useful Links– SEWeb www.environment.scotland.gov.uk – Scottish Soils http://www.soils-scotland.gov.uk/ – Aquaculture http://aquaculture.scotland.gov.uk/– Linked Data Lab http://data.sepa.org.uk– SSDI http://scotgovsdi.edina.ac.uk/srv/en/main.home– INSPIRE http://inspire.ec.europa.eu/ – Water Classification Visualisation

http://www.environment.scotland.gov.uk/get_interactive/data_visualisation/water_body_classification.aspx

SENSE 3 – Schema Relationships

State of Environment Reporting

• Defined by chapters (air, water, land, etc)

• Chapters divided into topics, each with a summary quality assessment

• Datasets support and inform the assessment of the topic

• A dataset may be related to more than one topic

• Currently published as static pages

State of Environment Reporting

• Remodel as linked data

• Enable publication of metadata on datasets

• Link to data visualisation and download where available

• Provide contact details where data not yet published on line

• Provide support and examples of best practice to assist publication

SEPA as Data Provider

SEPA Reporting Requirements

Information required at many levels

• Internal – SEPA corporate systems

• National – State of Environment; SEWeb

• European – Directive Reports; INSPIRE

Where we were…

Many applicationsMany formats

Many versions

SEPA Database

ReportsGIS Applications

PublicationsWebsite

Information Requests

EU Reporting

What we decided to do

• Focus on data – not applications

• Identify key reporting datasets

• Define them once

• Use them many times…

• …in many formats

Where we’ve got to

Operational Database

Reporting Database

Publish Externally

Defined data “products”

Consistent metadata

Intranet

Reports & Analysis

SEPA Website

EU ReportingConsistent data

Where we’re getting to

Operational Database

Reporting Database

Publish as WMS; WFS; Linked data

Defined data “products”

Consistent metadata

Intranet

Reports & Analysis

EU ReportingConsistent data

Websites (SEPA, SEWeb,…)

Partners

Public

What’s helped

• Scotland’s Spatial Data Infrastructure – provided framework and standards for metadata

• SEWeb – prioritisation of datasets

• Government direction – “digital by default“

• EU reporting frameworks – SEIS, SENSE

What we need now

• Agree to use existing standards and vocabularies

• Define new ones where appropriate

• Encourage use of common reference systems

• Encourage others to use the data

What we get out of it

• Wider (and cleverer) use of data

• Less bespoke development

• Fewer information requests to deal with

• Publish data once – let everyone else get on with it

Data Architecture

Repository

Relational Data

Consumers

Single Purpose Apps

E.g. RBMP

Bespoke Data Feed

SEPA Architecture

Single Purpose Apps

Repository

Relational Data

Consumers

Metadata

WFSApplications

INSPIRE

DRIVERSSEPA

Architecture

Service Data Feed

INSPIRE Service Based Architecture

Repository

Relational Data

Consumers

Metadata

File Download

Linked Data

Bespoke Data Feed

Data Feed Future

INSPIRE

SEPA Stakeholders

Public

Citizen Scientists

Data Ingestion

DRIVERSSEPA

Architecture

Linked Data Architecture

Repository

Relational Data

Consumers

Metadata

File Download

Linked Data

RDF/XML

SPARQL

TURTLE

csv/tsv

Web Apps

Mashups

Linked Data Sites/Uers

“Big Data” Sites/Uers

“Traditional” Sites/Uers

Web Developers

Bespoke Data Feed

Data Feed Future

INSPIRE

SEPA Stakeholders

Public

Citizen Scientists

Data Ingestion

Define Equivalences

DRIVERSSEPA

Architecture

Rdf Triple StoreServer

Linked Data ‘Technology Stack’

Linked Data

# Available on the web (whatever format) but with an open licence, to be Open Data

# # Available as machine-readable structured data (e.g. excel instead of image scan of a table)

# # # as (2) plus non-proprietary format (e.g. CSV instead of excel)

# # # # All the above plus, Use open standards from W3C (RDF and SPARQL) to identify things, so that people can point at your stuff

# # # # # All the above, plus: Link your data to other people’s data to provide context

5 Star Model of Open Data

What is Linked Data?

• Data in which real-world things are given addresses on the web (URIs), and data is published about them in machine-readable formats.

• Describes a method of publishing structured data so that it can be interlinked and become more useful.

• Builds upon standard Web technologies such as HTTP, RDF and URIs, but rather than using them to serve web pages for human readers, it extends them to share information in a way that can be read automatically by computers.

• Enables data from different sources to be connected and queried.

Linked Data Four Principles

1. Use URIs as names for things

2. Use HTTP URIs so that people can look up those names.

3. When someone looks up a URI, provide useful information, using the standards (RDF*, SPARQL)

4. Include links to other URIs so that they can discover more things.

Operational System

Typical Relational Data Table

Surface Water BodiesCOLUMN NAME DATA TYPE MANDATORY

ID Number Y

NAME Varchar2(30) Y

CATEGORY Varchar2(15) N

SUB_BASIN Varchar2(30) N

CATCHMENT Number N

STATUS Varchar2(30) N

Typical Relational Data

ID NAME CATEGORY SUB_BASIN

CATCHMENT STATUS

3001 River Almond (Breich Water confluence to Maitland Bridge)

River Forth 61 Poor

3809 River North Esk (Source to Penicuik House)

River Forth 63 High

100208 Loch Shiel Lake Argyll 117 Good

200019 South Arran Coastal Clyde Good

As Linked Data

Surface Water Body 3001 is of category River

Surface Water Body 3001 is called River Almond (Breich Water confluence to Maitland Bridge)

Surface Water Body 3001 is in sub-basin Forth

Surface Water Body 3001 is in catchment 61

Surface Water Body 3001 has status Poor

Surface Water Body 200019 is of category Coastal

Surface Water Body 200019 is called South Arran

Surface Water Body 200019 is in sub-basin Clyde

Surface Water Body 200019 has status Good

As Linked Data

Surface Water Body 3001 is of category River

Surface Water Body 3001 is called River Almond (Breich Water confluence to Maitland Bridge)

Surface Water Body 3001 is in sub-basin Forth

Surface Water Body 3001 is in catchment 61

Surface Water Body 3001 has status Poor

Surface Water Body 200019 is of category Coastal

Surface Water Body 200019 is called South Arran

Surface Water Body 200019 is in sub-basin Clyde

Surface Water Body 200019 has status Good

Surface Water Body 3001 is in local authority West Lothian

Surface Water Body 3001 is in local authority City of Edinburgh

Surface Water Body 200019 is in postcode district KA27

RDF/Triplestore

Subject Predicate Object

http://data.sepa.org.uk/id/water/surfacewaterbody/3001

rdf:type http://data.sepa.org.uk/def/water/WaterBody

rdf:type http://data.sepa.org.uk/def/water/SurfaceWaterBody

rdf:type http://data.sepa.org.uk/def/water/RiverWaterBody

rdfs:label “River Almond (Breich Water confluence to Maitland Bridge)”

http://data.sepa.org.uk/def/water/currentOverallClassification

“Overall status – Poor”

http://data.sepa.org.uk/def/water/inCatchment

http://data.sepa.org.uk/id/water/catchment/61

http://data.sepa.org.uk/def/water/surfaceArea

http://data.sepa.org.uk/id/water/catchment/61

http://data.sepa.org.uk/def/water/catchmentType

“Main River”

http://data.sepa.org.uk/id/water/subbasindistrict/3

rdfs:label “Forth”

Non SEPA-SEWeb Linked Data Examples

• Data.gov.uk.http://data.gov.uk/linked-data/who-is-doing-what

• EA Bathing Watershttp://environment.data.gov.uk/bwq/explorer/index.html

Ordnance Survey

http://data.ordnancesurvey.co.uk/doc/postcodeunit/EH127AT • Winnipeghttp://now.winnipeg.ca/

• Legislationhttp://www.legislation.gov.uk/

scotland's environment web data journey 2011-2015 dave watson, duncan taylor

starlinked data seweb

data publication

data consumption

data consumers

data consumerwhat

spotfirehow muchwhat

new data servicesestablish

internal data catalogueshow

Documents

scotland's special seas - nature.scot

scotland's history of whiskey

scotland's curriculum architecture

scotland's hotel performance - hot.e 2013

scotland's offshore wind route map: developing scotland's

scotlands environment web wise/gis it workshop 07-08 may...

scotland's national dental

scotland's rangers

scotland's future

scotland's rural college conservation agriculture

scotland's innovation centres

scotland's autarkic vision

scotland's environment web nesac

scotland's jews

scotland's economic strategy march 2015

scotland's beauty at risk

scotland's 100-oldest companies project

scotland's speciality food show guide

economic value of scotland's independence

jobs in scotland's new economy