aspect-oriented fault modeling and anomaly detection in … · 2018-04-03 · 40 60 80 100 120 40...

40 60 80 100 120

40

60

80

mmAspect-Oriented FaultModeling and

AnomalyDetection inPtolemy II

Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Aspect-Oriented Fault Modeling and AnomalyDetection in Ptolemy II

Ilge Akkaya

10th Biennial Ptolemy MiniconferenceUniversity of California, Berkeley

November 8, 2013

UC Berkeley Ilge Akkaya 1 / 26

40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

1 Introduction

2 Fault Models

3 Applications

4 Anomaly Detection

5 Summary

UC Berkeley Introduction Ilge Akkaya 2 / 26

40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Monolithic Smart-Grid Communications Model

I PMU: Phasor Measurement UnitI PDC: Phasor Data ConcentratorI Area: Balancing Authority running on a High Performance Computing(HPC) Cluster

A communication model for a three-area distributed smart-grid application


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Aspect-Oriented Modeling: A Comparison

vs

QM={LocalNet, MW, MWNetwork}


QM= {LocalNet, MW, MWNetwork}

QM = {MW}

QM = {MW}

QM = {MW}

QM = {PMULink}

QM = {PMULink}

QM = {PMULink} QM = {LocalNet}

QM = {LocalNet}

QM = {LocalNet}


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

The Aspect-Oriented Way



QM= {LocalNet, MW, MWNetwork}

QM = {MW}

QM = {MW}

QM = {MW}

QM = {PMULink}

QM = {PMULink}

QM = {PMULink} QM = {LocalNet}

QM = {LocalNet}

QM = {LocalNet}

For the communication archi-tecture, the specific imple-mentation uses four differentnetwork fabrics, each modeledas a separate aspect.


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Fault Models

I Faults can be viewed as an aspect of a system, as they areoften orthogonal to the application model

I A fault model is a structured and well-definedrepresentation of a faulty behavior of a system

I Help identify and isolate an anomaly in a systemcomponent

UC Berkeley Fault Models Ilge Akkaya 6 / 26

40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

History of Fault Models

I Fault hypotheses are essential for complex systemarchitectures

I AerospaceI AutomotiveI Energy systemsI Manufacturing, ...

I Architecture Analysis & Design Language(AADL)I Standardized by SAE (Society of Automotive Engineers)I Error-model annex: fault and fault propagation models

http://www.chipestimate.com/blogs/IPInsider/?p=92


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Fault Models in Ptolemy: Goals

I Cyber-Physical System Design goalsI FlexibilityI Fault-toleranceI Robustness

I Modal faults : Learning and detection of faulty and normalsystem behavior

I Parameter estimationI Maximum-Likelihood classification

using Machine Learning techniquesI Refinement of the functional model


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Fault Models in Ptolemy

I Goal: introduce reusable design patterns for faultmodeling and detection

I Aspect-oriented models used in modelingI Communication/Network behavior

I Network delaysI Queuing behaviorI Packet drops

I Atomic Probabilistic FaultsI Stuck-At FaultsI Bit-Flip Faults

I Modal Models extended with Probabilistic TransitionsI Adds modeling capability of numerous stochastic models:I Markov Models, Markov Chains, Mixture Models, etc.


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Fault Models as Aspects


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Stochastic Fault Models for System Identification

Figure : [en.wikipedia.org]

I Transition Systems are an effective tool for modelingdeterministic modal behavior

I However, the uncertainty in the nature of faults motivatesuse of probabilistic models


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Extending Modal Models in Ptolemy: MarkovChains

normal stuck0.1

0.91.0

StuckAt Fault as a Probabilistic Automaton


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Extending Modal Models in Ptolemy: HiddenMarkov Models

I An HMM is a Markov model with unobservable states(latent variables)

I qi : Hidden State : evolves according to a Markov processwith transition probability matrix A

I yj : Observed state: P(yj|qj = q) is a random variableI π: Prior distribution of hidden states Above: arrows

indicate probabilistic dependence.


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary



40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary


WiFiLTE

anomalous

0

1

2

3

4

5

6

x102

0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70

LatencyDistribution


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Aspect-Oriented Composite Fault Models

I Goal: Building a generic anomaly detection library toI Perform unsupervised learning on data streamsI Notify services ( and/or TerraPlane) when anomaly has

been inferred

UC Berkeley Applications Ilge Akkaya 17 / 26

40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Service Channel

WiFiLTE

anomalous

0

1

2

3

4

5

6

x102

0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70

LatencyDistribution

UC Berkeley Applications Ilge Akkaya 18 / 26

40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Anomaly Detection

I Goal: Building a generic anomaly detection library toI Perform unsupervised learning on data streamsI Notify services ( and/or TerraPlane) when anomaly has

been inferredI In the HMM setting, the best effort solution is the

Maximum Likelihood estimate ofI Underlying parameters of the stochastic modelsI Maximum-likelihood classification of hidden state

sequences

UC Berkeley Anomaly Detection Ilge Akkaya 19 / 26

40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Anomaly Detection:Expectation-Maximization(EM)

I EM is an iterative algorithm used for finding theMaximum-Likelihood parameter estimates of aprobabilistic graphical model.

I Given the observations, finds the most likely set ofparameters that explain the observations.

I Statistically-sound version of the well-known heuristic:K-Means Clustering algorithm


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

A Ptolemy Toolkit for HMM-Based AnomalyDetection


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

AnomalyDetection

2x10

classified labelstrue labels

0.00.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0

-0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0

HMM Classification Series on Test Data

0.6

0.7

0.8

0.9

1.0

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Success Ratio per Classification

Sequence Index

Suc

cess

Rat

io


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Smart Grid Communication Fault Injection

UC Berkeley Summary Ilge Akkaya 23 / 26

40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Smart Grid Communication Fault Injection


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Summary

I Aspect-oriented modelingI Probabilistic fault models implemented in PtolemyI Goal: Provide a generic parameter estimation and

classification library to perform generic anomaly detectionon real-time data streams

I Investigating future methods to better model separationbetween environmental and system-imposed uncertainties

I Additional fault-detection mechanismsI Dynamic, multi-sensor fault modelsI A framework for generic anomaly detection on aggregate

sensor streams


40 60 80 100 120

40

60

80



Ilge Akkaya

Introduction

Fault Models

Applications

AnomalyDetection

Summary

Thanks!

Questions ?Comments ?


aspect-oriented fault modeling and anomaly detection in … · 2018-04-03 · 40 60 80 100 120 40...

Documents