detecting sewer rising main events using an ontology-driven event processing system csiro land and...

Detecting sewer rising main events using an ontology-driven event processing system

CSIRO LAND AND WATER

Jonathan Yu | Research software engineerPaul Davis, Irina Emelyanova, Scott Gould, Kerry Taylor, Donavan Marney

22 Feb 2013

Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu

The problem with sewer rising mains

Sewer rising main / pressure sewers• pipeline that carries sewerage at pressure from a pumping station• transport sewage where gravity flow is not possible or practical

Failures can be severe• Direct costs: water service providers ($ mil. per event)

– Pipeline repair – rising mains can be long... ~10km+– Sewerage removal by contractor:

say 3 weeks = 12 runs/day [24/7] @ 10 kL per run

• Indirect costs: social, environmental ($10k - $1 mil. per event)– In a recent case, a burst in a relatively new pressure sewer led to

undetected sewage discharge to a nearby creek for approximately 3 months

2 |


Addressing sewer rising main events An option: Retro-fit commercial pressure sewer monitors

Costly and time-consuming...

3 |


Q: Can we extract value from ‘business-as-usual’ data for early detection and pre-empting of these failures?

4 |

• In most cases, data is already collected• Wet-well levels• Pressure at pump station• Flow rate

• Look for breakpoints in sewer inflow rate timeseries by analysing trend analysis

• Capture these rules/event conditions


Our approach

1) Investigate how to detect of sewer rising main events• Trend analysis methods and algorithms• Stream processing engines to enable real-time detection

2) System for user-definition and deployment of event constraints • Capture semantics of event constraints • System for deployment of event constraints on stream processing engine• Propose ontology-driven approach

5 |


Sewer rising mains case study

Prototyping event detection algorithmsSewer rising mains pipe burst detection – flow, pump pressure

• Simple Moving Average, Exceeded thresholds, • Breakpoint analysis and Near Real-Time Disturbance detection (Irina)

6 |

0 2000 4000 6000 8000 10000 12000 14000 160000

20

40

60

80

100

120

140

160

Time (mins)

Flow

rate

(l/s

)

Failure event at 4260 minutes


Stepped notifications: < 5 consequtive = low, 5 - 20 consequtive = moderate, > 20 = high risk

Applying Simple Moving Average & Stepped notifications

7 |

(Low risk)

(High risk)

Stepped Notifications

Flow observations

Simple Moving Average

Looking for when flow exceeds a preset threshold over the Simple Moving average


Applying “Near Real-Time Disturbance detection” [1]

8 |

[1]. Verbesselt J, Zeileis A, Herold M (2011). Near Real-Time Disturbance Detection in Terrestrial Ecosystems Using Satellite Image Time Series: Drought Detection in Somalia. Working Paper 2011-18. Working Papers in Economics and Statistics, Research Platform Empirical and Experimental Economics, Universitaet Innsbruck. http://EconPapers.RePEc.org/RePEc:inn:wpaper: 2011-18. Submitted to Remote Sensing and Environment.

Start of monitoring periodHistorical data (input)

http://econpapers.repec.org/RePEc:inn:wpaper


Applying Near Real-Time Disturbance detection pt. 2

9 |

Historical data (input) Monitoring period

Break-point identified


Use of stream processing engine (GSN)

10 |

SensorMiddleware

(GSN)

Sensor Network

End users

Open source sensor middleware.Provides abstraction APIs on raw streaming

sensor data(windowing, aggregate sensor sources, low-level

processing libraries, flexible output options)

Real-time streaming sensor data

Implements event detection algorithms

(Scripting via R, Groovy)

Email, SMS, Text2Speech, Integration with existing monitoring systems


Real-time sensor stream data processing

• High level entry for an end user e.g. Scientists and managers

• Knowledge hidden behind code or implicit in people’s heads• Possible barrier for reusability

11 |

Curation CodingAnalysis,

Monitoring, Management

SensorMiddleware

(GSN)

Sensor Network

End users

Programmers

Semantics-based approach for defining complex event rules for algal bloom detection | Jonathan Yu

Problem of data heterogeneity, integration

• Multiple datasets• Often multiple data schemas and formats

• Example: The use of the observation property “Flow rate”• Flow• FLOW_RATE• RATE_OF_FLOW_L_per_s

• Want to be able to have mechanism of translating and mapping differing fields, labels to something commonly understood• Enhance interoperability

12 |


“Semantics-based approach”

Ontologies• Capture semantics• Lingua franca• Machine readable/processable

Vocabulary of things you care about in your data/system• E.g. Ability to refer to ‘Flow rate’ concept, rather than FLOW_RATE

We use ontologies for:• Providing translation between fields within sensors, datasets• Defining event rules • Generating code for actioning event rules on the sensors

13 |

OntologiesSemantic

Sensor Net. Ontology

DomainOntology


Ontology-driven event detection system

14 |

SensorMiddleware

(GSN)

Sensor Network

End users

Ontology-enabledUser Interface

OntologiesSemantic

Sensor Net. Ontology

DomainOntology

Annotates available sensors and their capabilities

e.g. Pump pressure sensor data at Location X

Generate appropriate code to perform event detection on available sensors using event constraint semanticse.g. Identify break-point in sewer inflow rate according to trend

Populates user interface elements based domain semantics and sensor network annotations.

Allow users to define event constraints using ontology semantics

Return notifications from triggered events with metadata based on ontology semantics

e.g. The sewer rising main has a problem due to <...>

Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu15 |


Future work

• Implementation of breakpoint analysis and “near real-time disturbance” algorithms in our system

• Continuing ontology engineering for sewer rising main event detection• Harmonising with standard units of measure ontologies

• User interface refinement, and user testing• Very much a prototype / proof-of-concept

• Code generation module for deploying event constraints into GSN

• Performance and load testing to handle volumes of data

16 |


Extension idea - fusing real-time data with domain knowledge base

17 |

KnowledgeBase

Sensor NetworkReal-time data

Event of Interest

Query knowledge base(domain knowledge)

Notifications

Populate knowledge base with parameterised historical pipe failure data.

Infer likelihood of pipe failure based on physical attributes and known operating environment


Summary

Value of analysing ‘business-as-usual’ data for early detection of sewer mains pipe failure

Investigated a variety of timeseries analysis methods that is suitable for breakpoint detection of sewer rising mains failure events – SMA, Breakpoint analysis, Near real-time trend detection• Does not require extensive training datasets

Ontology-driven event detection system• User interface for defining machine readable event constraints using domain-

specific ontologies• System for deploying these event constraints for detection over sensor

network

18 |

Land and WaterScott GouldResearch Projects Officert +61 3 9252 6103e [email protected] www.csiro.au/clw

ICT CentreKerry TaylorPrincipal Research Scientistt +61 2 6216 7038e [email protected] www.csiro.au/ict

Land and WaterDonavan MarneyResearch team leadert +61 3 9252 6585e [email protected] www.csiro.au/clw

LAND AND WATER

Thank youLand and WaterJonathan YuSoftware Engineert +61 3 9252 6440e [email protected] www.csiro.au/clw

Land and WaterPaul DavisResearch Scientistt +61 3 9252 6310e [email protected] www.csiro.au/clw

Land and WaterIrina EmelyanovaResearch Scientistt +61 8 9333 6243e Irina. Emelyanova @csiro.auw www.csiro.au/clw


Advantages of semantics-based approach• Transparent and transferrable

• Rules, vocabularies, mappings are captured in the ontologies• Can deploy to other systems as long as they are mapped

• Traceable• Alerts can attach metadata to describe triggers: what, why, when

• End users can focus on exploring real-time datasets

20 |

Curation Coding Analysis, Monitoring, Management

Curation CodingAnalysis,

Monitoring, Management

Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu21 |

Ontology-driven event detection system1. Composes CE

Sensor Network

Ontology-enabledUser Interface

Semantic Mediator

GSN

Ontologies

SSN Ontology

DomainOntology

7.Updates UI withalert

3. Deploys CE to GSN as VSensor via translation

capture rule to sensor API mappingcapture sensor / data sources mappings

6. Matching event alert generated

2. Submits CE definition

captures alerts

captures CE definition

8. Views alert

5. Sensor streams

data

Users

detecting sewer rising main events using an ontology-driven event processing system csiro land and...

Documents