automating semantic metadata collection in the field with mobile application

1

Automating Semantic Metadata Collection in the Field with

Mobile ApplicationLaura Kinkead*, Paulo Pinheiro,

Deborah L. McGuinnessTetherless World ConstellationRensselaer Polytechnic Institute

* Now at Athena Health

Motivation: Next Generation Monitored Ecosystems

The Jefferson Project (JP) is a joint effort between Rensselaer Polytechnic Institute (RPI), IBM and the Fund for Lake George aimed at creating an instrumented water ecosystem along with an appropriate cyberinfrastructure that can serve as a global model for ecosystem monitoring, exploration, understanding, and prediction.

3

Historical Sampling to Sensors, Models, Experiments

• Sampling at 12 locations• Only water chemistry was previously measured• Key previous results:

Salt levels increasing – now dominant in the lake Chlorophyll slowly increasing Hypoxia in Caldwell Basin changed little

• Key resulting hypotheses: Increasing salt levels and organic nutrients may favor dominance of

cyanobacteria in the phytoplankton Ca levels may limit spread of invasive zebra mussels Chlorophyll increase may be caused by nutrient loading Food web mostly driven by “bottom-up” factors (i.e. nutrients, growing

season length)

Moving to sensors, streaming data, and a smarter, instrumented lake with the goal of providing a foundation to form and evaluate hypotheses much more effectively enabling a new generation of strategic science dedicated to fuller understanding of the Lake's ecological health.

4

Science to Inform Solutions

To Realize a truly Smart Lake:We need an integrative approach to

understanding lake stressors, identifying correlations, hypothesizing

causation, experimentally testing hypotheses, and proposing actions

Science-based Solutions:

Leveraging deep understanding of

multiple communities and their research content to propose solutions along with

evidence

informs

Cyberinfrastructure/Data Platform/Viz Lab

Semantic DataModel Current focus has been on

observations &sensor networks

Paulo Pinheiro, 04/09/2014

In major research projects, we rarely we see situations where observational data is combined with simulation data or experimental data. In the Jefferson Project, such combined used of observational, simulated and experimental data is supposed to be the norm

5

Traditional Data Collection

Notes

Notes taken in the field with the use of pen and paper

Notes are rarely attached to data

There is no community-wide consensus on how to take and reuse field notes

6

Mobile Context Capture for Sensor Networks (MOCCASN)

COLLECT

METADATA

One single mobile application capable of taking field notes and connect the notes to data as semantic annotations

SOLR-CCSV SOLR-CCSV

CCSV-LoaderCCSV-Loader

dataStatic

metadata

CSV2CCSV(ICS)

CSV2CCSV(ICS)

CCSV-Annotator*CCSV-Annotator*MOCASSNMOCASSN

HASNetO-LoaderHASNetO-Loader

Dynamicmetadata

Sensornetwork

technician scientist

data user(incl. scientists)

maintains

reportshuman Interventions(deployments,sensor config,calibrations)

Single instrument

data (csv)

ccsv

data (csv)

Spreadsheet of static metadata

ccsv

static metadata turtle

SPARQL and Lucene queries

CCSV BrowserCCSV Browser

SPARQL and Lucene queries

Faceted search

annotatedcsv

Dynamic metadata

IN-SITU

DATA-SITE

DATA-SITE

WWW

uses

reportsneeds

Dynamic metadata

Ontologies(HASnetO, OBOE,

PROV, VSTO)

* Tool to be developed

metadata metadata

mainly data flowmainly metadata flow

8

a simple example

Human-Aware Sensor Network Ontology (HASNetO)

9

vstoi:Detector

vstoi:Instrument

vstoi:Platform

hasneto:Sensing

Perspective

oboe:Characteristic

oboe:Entity

vstoi:DetachableDetector

vstoi:AttachedDetector

*

*

*

1

0..1

*hasPerspectiveCharacteristic

perspectiveOf

prov:Activity

hasneto:DataCollection

vstoi:Deployment

xsd:dateTime

xsd:dateTime

hasDataCollection

1*

prov:Agent

wasAssociatedWith startedAtTime

endedAtTime

1

1

*

*

*

*

oboe:Measurement

of-characteristic

hasneto:hasMeasurement

1

1

*

*

Platform 3952

Instrument 3

D 38 D 94

10

Platform 3952

Instrument 3

D 38 D 94

RFID Type Parent Deployed Location Start Time End Time3952 Platform NA TRUE 43.1, -73.2 2014-10-

27T12:00NA

3 Instrument 3952 TRUE 43.1, -73.2 2014-10-27T12:00

NA

38 Detector 3 TRUE 43.1, -73.2 2014-10-27T12:00

NA

94 Detector 3 TRUE 43.1, -73.2 2014-10-27T12:00

NA

Example Knowledge Base

11

Platform 3952

Instrument 5

D 74

Instrument 3

D 38 D 94


27T12:00NA


NA

38 Detector 3 TRUE 43.1, -73.2 2014-10-27T12:00

NA

94 Detector 3 TRUE 43.1, -73.2 2014-10-27T12:00

NA


New instrument deployment

12

Platform 3952

Instrument 5

D 74

Instrument 3

D 38 D 94

38 94

3952

74

5

3

RFID Type Parent Deployed Location Start Time End Time

3952 Platform NA TRUE 43.1, -73.2 2014-10-27T12:00

NA


NA

38 Detector 3 TRUE 43.1, -73.2 2014-10-27T12:00

NA

94 Detector 3 TRUE 43.1, -73.2 2014-10-27T12:00

NA


13

Platform 3952

Instrument 5

D 74

Instrument 3

D 38 D 94


27T12:00NA


NA

38 Detector 3 TRUE 43.1, -73.2 2014-10-27T12:00

NA

94 Detector 3 TRUE 43.1, -73.2 2014-10-27T12:00

NA


NA

74 Detector 5 TRUE 43.1, -73.2 2014-10-27T16:30

NA


14

Platform 3952

Instrument 5

D 74

Instrument 3

D 38 D 94


27T12:00NA


NA

38 Detector 3 TRUE 43.1, -73.2 2014-10-27T12:00

NA

94 Detector 3 TRUE 43.1, -73.2 2014-10-27T12:00

NA


NA

74 Detector 5 TRUE 43.1, -73.2 2014-10-27T16:30

NA


15

a more complicated example – taking an instrument out of service and adding

instruments

16

RFID Type Parent Deployed Location Start Time End Time9754 Platform NA TRUE 43.2,

-73.12014-10-01T11:00

NA

8 Instrument 9754 TRUE 43.2, -73.1

2014-10-01T11:00

NA

43 Detector 8 TRUE 43.2, -73.1

2014-10-01T11:00

NA

Platform 9754

Instrument 8

D 43


17

Platform 9754

Instrument 2

D 61

Instrument 6

D 09


01T11:00NA


NA

43 Detector 8 TRUE 43.2, -73.1 2014-10-01T11:00

NA


Undeploy one instrument (8) (with one detector(43)) and deploy 2 new instruments (each with a detector)

18

Platform 9754

Instrument 2

D 61

Instrument 6

D 09

09 61

9754

6

2


01T11:00NA


NA

43 Detector 8 TRUE 43.2, -73.1 2014-10-01T11:00

NA


19

Platform 9754

Instrument 2

D 61

Instrument 6

D 09

09

61

6

2

Does Detector 09 belong to Instrument 2?

Yes No


01T11:00NA


NA

43 Detector 8 TRUE 43.2, -73.1 2014-10-01T11:00

NA


20

RFID Type Parent Deployed Location Start Time End Time9754 Platform NA FALSE 43.2, -73.1 2014-10-

01T11:002014-10-27T17:00

8 Instrument 9754 FALSE 43.2, -73.1 2014-10-01T11:00

2014-10-27T17:00

43 Detector 8 FALSE 43.2, -73.1 2014-10-01T11:00

2014-10-27T17:00

9754 Platform NA TRUE 43.2, -73.1 2014-10-27T17:00

NA


NA


NA

61 Detector 2 TRUE 43.2, -73.1 2014-10-27T17:00

NA

9 Detector 6 TRUE 43.2, -73.1 2014-10-27T17:00

NA

Platform 9754

Instrument 2

D 61

Instrument 6

D 09

Does Detector 09 belong to Instrument 2?

Yes No


21

RFID Type Parent Deployed Location Start Time

End Time9754 Platform NA FALSE 43.2,

-73.12014-10-01T11:00

2014-10-27T17:00

8 Instrument 9754 FALSE 43.2, -73.1

2014-10-01T11:00

2014-10-27T17:00

43 Detector 8 FALSE 43.2, -73.1

2014-10-01T11:00

2014-10-27T17:00

9754 Platform NA TRUE 43.2, -73.1

2014-10-27T17:00

NA


2014-10-27T17:00

NA


2014-10-27T17:00

NA


2014-10-27T17:00

NA


2014-10-27T17:00

NA

Platform 9754

Instrument 2

D 61

Instrument 6

D 09


Automatic update from answering one simple question: lightweight use of semantics

22

Conclusion

• Automated Metadata capture can enable current and next generation sensor-based science by enabling ubiquitous capture of contextual information – helps eliminate forgetting to annotate

• Mobile technology should and can enable contextual capture even without connectivity

• Relatively light weight semantics can significantly

• Improve deployment quality by using semantic constraints to check for inconsistencies and help identify / resolve ambiguities

• Enable integration

• Enable discovery

Questions? Interested in collaborating? [email protected] [email protected]

mailto:[email protected]

mailto:[email protected]

Extras

23

25

Recognized Challenges

• What do you do when there’s no cell service?

• How do you make sure the instruments on the boat are excluded?

26

Intelligent Deployment of Sensor Networks

• Automates the collection of metadata

‣ faster

‣ harder to forget to do

‣ less error-prone

The Human-Aware Sensor Network Ontology

27

Science to Inform Solutions

Paulo Pinheiro, 04/09/2014

In major research projects, we rarely we see situations where observational data is combined with simulation data or experimental data. In the Jefferson Project, such combined used of observational, simulated and experimental data is supposed to be the norm