Upload File

where linked data meets big data: applying standard data models to environmental data streams

  • Home
  • Data & Analytics
  • Where Linked Data meets Big Data: Applying standard data models to environmental data streams
36
Where Big Data meets Linked Data: Applying standard data models to environmental data streams Adam Leadbetter, Damian Smyth, Robert Fuller, Eoin O’Grady Marine Institute, Ireland Adam Shepherd Woods Hole Oceanographic Institution, US [email protected]

Upload: adam-leadbetter

Post on 19-Jan-2017

147 views

Category:

Data & Analytics


0 download

Report

TRANSCRIPT

Page 1: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

Where Big Data meets Linked Data:

Applying standard data models to environmental data

streamsAdam Leadbetter, Damian Smyth, Robert Fuller, Eoin

O’GradyMarine Institute, Ireland

Adam ShepherdWoods Hole Oceanographic Institution, US

[email protected]

Page 2: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

http://whatis.techtarget.com/definition/3Vs

Page 3: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

HTTP://XKCD.COM/927/

Page 4: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

https://s-media-cache-ak0.pinimg.com/564x/e7/c2/6b/e7c26b7963b2628d10fe649008b02d23.jpg

Page 5: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

https://upload.wikimedia.org/wikipedia/commons/thumb/9/9d/Sir_Tim_Berners-Lee.jpg/640px-Sir_Tim_Berners-Lee.jpg/

Page 6: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

@prefix mi-feature: < http://linked.marine.ie/feature/>.@prefix mi-gbco: <http://spiddal.marine.ie/2016-08-18-11-42-59-782#>.@prefix nvs-l22: <http://vocab.nerc.ac.uk/collection/L22/current/>.@prefix nvs-p01: <http://vocab.nerc.ac.uk/collection/P01/current/>.@prefix om: <http://def.seegrid.csiro.au/isotc211/iso19156/2011/observation#>.

mi-gbco:temperature om:process nvs-l22:TOOL0861;om:observedProperty nvs-p01:TEMPPR01;om:result <_:result>.om:resultTime "2016-08-18T11:42:59.782Z" .

<http://spiddal.marine.ie/2016-08-18-11-42-59-782> om:featureOfInterest mi-feature:GalwayBayCableObservatoryLocation;om:phenomenonTime "2016-08-18T11:42:59.782Z" .om:relatedObservation mi-gbco:temperature.

Page 7: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

@prefix mi-feature: < http://linked.marine.ie/feature/>.@prefix mi-gbco: <http://spiddal.marine.ie/2016-08-18-11-42-59-782#>.@prefix nvs-l22: <http://vocab.nerc.ac.uk/collection/L22/current/>.@prefix nvs-p01: <http://vocab.nerc.ac.uk/collection/P01/current/>.@prefix om: <http://def.seegrid.csiro.au/isotc211/iso19156/2011/observation#>.

mi-gbco:temperature om:process nvs-l22:TOOL0861;om:observedProperty nvs-p01:TEMPPR01;om:result <_:result>.om:resultTime "2016-08-18T11:42:59.782Z" .

<http://spiddal.marine.ie/2016-08-18-11-42-59-782> om:featureOfInterest mi-feature:GalwayBayCableObservatoryLocation;om:phenomenonTime "2016-08-18T11:42:59.782Z" .om:relatedObservation mi-gbco:temperature.

Page 8: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

@prefix mi-feature: < http://linked.marine.ie/feature/>.@prefix mi-gbco: <http://spiddal.marine.ie/2016-08-18-11-42-59-782#>.@prefix nvs-l22: <http://vocab.nerc.ac.uk/collection/L22/current/>.@prefix nvs-p01: <http://vocab.nerc.ac.uk/collection/P01/current/>.@prefix om: <http://def.seegrid.csiro.au/isotc211/iso19156/2011/observation#>.

mi-gbco:temperature om:process nvs-l22:TOOL0861;om:observedProperty nvs-p01:TEMPPR01;om:result <_:result>.om:resultTime "2016-08-18T11:42:59.782Z" .

<http://spiddal.marine.ie/2016-08-18-11-42-59-782> om:featureOfInterest mi-feature:GalwayBayCableObservatoryLocation;om:phenomenonTime "2016-08-18T11:42:59.782Z" .om:relatedObservation mi-gbco:temperature.

Page 9: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

@prefix mi-feature: < http://linked.marine.ie/feature/>.@prefix mi-gbco: <http://spiddal.marine.ie/2016-08-18-11-42-59-782#>.@prefix nvs-l22: <http://vocab.nerc.ac.uk/collection/L22/current/>.@prefix nvs-p01: <http://vocab.nerc.ac.uk/collection/P01/current/>.@prefix om: <http://def.seegrid.csiro.au/isotc211/iso19156/2011/observation#>.

mi-gbco:temperature om:process nvs-l22:TOOL0861;om:observedProperty nvs-p01:TEMPPR01;om:result <_:result>.om:resultTime "2016-08-18T11:42:59.782Z" .

<http://spiddal.marine.ie/2016-08-18-11-42-59-782> om:featureOfInterest mi-feature:GalwayBayCableObservatoryLocation;om:phenomenonTime "2016-08-18T11:42:59.782Z" .om:relatedObservation mi-gbco:temperature.

Page 10: Where Linked Data meets Big Data: Applying standard data models to environmental data streams
Page 11: Where Linked Data meets Big Data: Applying standard data models to environmental data streams
Page 12: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

http://iode.org/index.php?option=com_oe&task=viewDocumentRecord&docID=875

Page 13: Where Linked Data meets Big Data: Applying standard data models to environmental data streams
Page 14: Where Linked Data meets Big Data: Applying standard data models to environmental data streams
Page 15: Where Linked Data meets Big Data: Applying standard data models to environmental data streams
Page 16: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

http://smartbay.marine.ie

Page 17: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

http://github.com/IrishMarineInstitute/uwobs

Page 18: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

2016-10-03T00:10:04.863Z|I-OCEAN7-304-XXXX| 24.51 14.837 37.002 29.784 1500.4781 00:15:13.86M

Page 19: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

2016-10-03T00:10:04.863Z|I-OCEAN7-304-XXXX| 24.51 14.837 37.002 29.784 1500.4781 00:15:13.86M

%{NUMBER:pressure:float}%{SPACE}%{NUMBER:temperature:float}%{SPACE}%{NUMBER:conductivity:float}%{SPACE}%{NUMBER:salinity:float}%{SPACE}%{NUMBER:sound_velocity:float}%{SPACE}%{TIME:raw_time}

Page 20: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

2016-10-03T00:10:04.863Z|I-OCEAN7-304-XXXX| 24.51 14.837 37.002 29.784 1500.4781 00:15:13.86M

%{NUMBER:pressure:float}%{SPACE}%{NUMBER:temperature:float}%{SPACE}%{NUMBER:conductivity:float}%{SPACE}%{NUMBER:salinity:float}%{SPACE}%{NUMBER:sound_velocity:float}%{SPACE}%{TIME:raw_time}

Page 21: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

An observation is an action whose result is an estimate of the value of some property of the feature-of-interest, obtained using a specified procedure

Page 22: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

An observation is an action whose result is an estimate of the value of some property of the feature-of-interest, obtained using a specified procedure

Observation

Page 23: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

An observation is an action whose result is an estimate of the value of some property of the feature-of-interest, obtained using a specified procedure

Observation

result

Page 24: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

An observation is an action whose result is an estimate of the value of some property of the feature-of-interest, obtained using a specified procedure

Observation

result

property

Page 25: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

An observation is an action whose result is an estimate of the value of some property of the feature-of-interest, obtained using a specified procedure

Observation

result

Feature property

Page 26: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

An observation is an action whose result is an estimate of the value of some property of the feature-of-interest, obtained using a specified procedure

Observation

result

Feature property

procedure

Page 27: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

{ "id":"foo", "phenomenonTime": {"instant":"2016-08-18T11:42:59.782Z"}, "featureOfInterest":{"href":"http://linked.marine.ie/feature/exampleURI"}, "member":[{ "id":"#temperature", "type":"Measurement", "result": { "value":16.118, "uom":"http://vocab.nerc.ac.uk/collection/P06/current/UPAA/" }, "observedProperty": { "href":"http://vocab.nerc.ac.uk/collection/P01/current/TEMPPR01/" }, "procedure":{ "href":"http://vocab.nerc.ac.uk/collection/L22/current/TOOL0861/" }, "resultTime":"2016-08-18T11:42:59.782Z“}]}

Page 28: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

{ "@context": "http://example.org/om-context", "id":"foo", "phenomenonTime": {"instant":"2016-08-18T11:42:59.782Z"}, "featureOfInterest":{"href":"http://linked.marine.ie/feature/exampleURI"}, "member":[{ "id":"#temperature", "type":"Measurement", "result": { "value":16.118, "uom":"http://vocab.nerc.ac.uk/collection/P06/current/UPAA/" }, "observedProperty": { "href":"http://vocab.nerc.ac.uk/collection/P01/current/TEMPPR01/" }, "procedure":{ "href":"http://vocab.nerc.ac.uk/collection/L22/current/TOOL0861/" }, "resultTime":"2016-08-18T11:42:59.782Z“}]}

Page 29: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

@prefix mi-feature: < http://linked.marine.ie/feature/>.@prefix mi-gbco: <http://spiddal.marine.ie/2016-08-18-11-42-59-782#>.@prefix nvs-l22: <http://vocab.nerc.ac.uk/collection/L22/current/>.@prefix nvs-p01: <http://vocab.nerc.ac.uk/collection/P01/current/>.@prefix om: <http://def.seegrid.csiro.au/isotc211/iso19156/2011/observation#>.

mi-gbco:temperature om:process nvs-l22:TOOL0861;om:observedProperty nvs-p01:TEMPPR01;om:result <_:result>.om:resultTime "2016-08-18T11:42:59.782Z" .

<http://spiddal.marine.ie/2016-08-18-11-42-59-782> om:featureOfInterest mi-feature:GalwayBayCableObservatoryLocation;om:phenomenonTime "2016-08-18T11:42:59.782Z" .om:relatedObservation mi-gbco:temperature.

Page 30: Where Linked Data meets Big Data: Applying standard data models to environmental data streams
Page 31: Where Linked Data meets Big Data: Applying standard data models to environmental data streams
Page 32: Where Linked Data meets Big Data: Applying standard data models to environmental data streams
Page 33: Where Linked Data meets Big Data: Applying standard data models to environmental data streams
Page 34: Where Linked Data meets Big Data: Applying standard data models to environmental data streams
Page 35: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

“There are emergent technologies throughout the

fields around oceanography which we will

incorporate into oceanography, and through

that convergence we will make oceanography

into something even more magical”

John Delaney

Page 36: Where Linked Data meets Big Data: Applying standard data models to environmental data streams

Adam Leadbetteradam.leadbetter@ma

rine.ie @AdamLeadbetter


1 Stream-based Data Management IS698 Min Song 2 Characteristics of Data Streams Data Streams Data streams — continuous, ordered, changing, fast, huge

Mining High-Speed Data Streams

Analysing Real World Data Streams with Spatio-Temporal ...epubs.surrey.ac.uk/845622/1/Analysing Real World Data Streams wit… · Analysing Real World Data Streams with Spatio-Temporal

Data Science meets Linked Data

Section 8.1. Mining Data Streams

Algorithms for geometric data streams 

Finding Frequent Items in Data Streams - DIMACSdmac.rutgers.edu/Workshops/WGUnifyingTheory/Slides/cormode.pdf · Finding Frequent Items in Data Streams 2 Data Streams ♦Many large

Aggregating Data Streams for More Effective Revenue Management€¦ · Aggregating Data Streams for More Effective Revenue Management. AggregAting DAtA StreAmS for more effective

Monitoring Distributed Data Streams

Mining Data Streams Presentation

Modul 2 - Data Streams

Mining Data Streams

Big Data Meets Data Nationalism

Managing Distributed Data Streams – II

Data Meets Numberwang

Data Organization Data files Databases Data streams Data files Databases Data streams

5.1 Mining Data Streams

Continuous Queries over Data Streams

Amazon Kinesis Data Streams - AWS Documentation · Amazon Kinesis Data Streams Developer Guide What Can I Do with Kinesis Data Streams? What Is Amazon Kinesis Data Streams? Use Amazon

Managing Data Streams

Query Assurance on Data Streams

Alibali mult data streams a

Capture Data Streams

Data Streams & Continuous Queries

SUMMIT - Amazon Web Services... · Kinesis Data. Streams. Kinesis. Data. Analytics. Kinesis Data Firehose • Capture • Transform • Load data streams • Process data streams

Course : Data mining - Lecture : Mining data streams

Integrating Distributed Data Streams

Mining Association Rules Events over Data Streams · Abstract Mining Association Rules Events over Data Streams Aref Faisal Mourtada Data streams have gained considerable attention

Multiple Aggregations Over Data Streams

Data Stream Management Systems 1. 2 Data Streams data sets Traditional DBMS – data stored in finite, persistent data sets data streams New Applications

Data Streams

Big Data meets Big Data

Privacy Preservation for Data Streams

18 Data Streams

Distributed clustering from data streams