doc prod 00015

8/3/2019 Doc Prod 00015

http://slidepdf.com/reader/full/doc-prod-00015 1/20

Complex Event Processing for Operational Intelligence

A Vitria Technical White Paper

>

8/3/2019 Doc Prod 00015


8/3/2019 Doc Prod 00015


VITRIA TECHNOLOGY, INC. > Complex Event Processing or Operational Intelligence

Introduction

Economic conditions, competition, social media-driven customer expectations, and

the ood o digitized data are placing unprecedented pressure on organizations to

assimilate inormation and act appropriately and quickly. However, business events

have a way o staying a step or two ahead o our ability to capture, analyze,

understand, and act upon them. Simply producing reports on a pre-dened

schedule is not enough. Decision-makers need real-time alerts and insight so that

they may take meaningul action while it still matters.

Operational Intelligence is a new approach or decision-making that allows the

optimal response to be taken at the right time. Operational Intelligence provides

three key capabilities:

• Visibility—theabilitytoseeandaccessinformationfromawidevarietyof

sources using a rich and interactive user interace

• Insight—theabilitytoanalyzeanddrawconclusionsfrommultiple

real-time and historical data sources as the inormation changes

• Action—theabilitytorespondinameaningfulwaytopositivelyimpact

business, processes, and customers

Vitria’s M3O platorm or Operational Intelligence brings together visibility via rich

Web 2.0 dashboards, insight rom powerul analytics via Complex Event Processing

(CEP), and policy-based action with integrated business process management (BPM)

M3O (which stands or Model, Manage, Monitor, and Optimize) provides these

capabilities in a complete, integrated platorm or building Operational Intelligence

applications in an Event-Driven Architecture, where inormation systems are

modeled rom a business event perspective. M3O allows analysts and developers to

work collaboratively across the whole o the business event liecycle.

This paper ocuses on the CEP components o M3O that deliver the “Insight” in

Operational Intelligence. We rst dene what is meant by Complex Event

Processing, examine the dierent types o CEP engines, and then explain why Vitria’s

unique approach to analyzing real-time XML event data has a number o important

advantages over other types o CEP engines. We then explore M3O’s CEP

components and how they integrate to provide insight to decision makers:

• M3OQueryModeler—thegraphicalWeb2.0interfaceusedbyanalystsand

developers to build CEP queries

• M3OFeedServer—managesmultipleinstancesandtypesofreal-timefeed

sources and provides scalability and enterprise quality o service to

real-time eeds

• M3OAnalyticServer—thehigh-performanceCEPenginewhichexecutes

queries on real-time eeds, creating results ready or visualization in

real-time dashboards or or automated, policy-based action and exception

management

Table o Contents

1 Introduction

2 What is Complex Event Processing?

5 The Importance o XML in

Complex Event Processing

5 - Dierent Types o CEP Engines

6 - StreamXQueryTM: Efcient Queries

on Real-time XML Data

7 - StreamXQueryTM Windows

8 - A StreamXQueryTM Example

9 - Advantages o StreamXQueryTM

over SQL-based Query Engines

10 M3O®: A Modern Architecture or CEP

10 M3O® Query Modeler

12 M3O® Feed Server

13 M3O® Analytic Server

15 Presentation Manager and Open Query API

15 Scaling Complex Event Processing in M3O®

16 Conclusion

8/3/2019 Doc Prod 00015


2VITRIA TECHNOLOGY, INC. > Complex Event Processing or Operational Intelligence

What is Complex Event Processing?

Complex Event Processing is a key Operational Intelligence technology that allows

an individual to request an inquiry or analysis once, and then have it continuously

evaluated over time against one or many streams o events in a highly efcient

manner.

Forexample,perhapsyouwanttoknowtheaveragewaittimeofcustomersbroken

out by region, product requested, and customer type. The underlying data arrives as

a series o events in a continuous event stream and computation must be perormed

onthereal-timeeventsinordertocalculatetheaveragewaittimes.InFigure1we

provide some basic denitions o these concepts.

Figure 1: Events, Streams, Feeds, and CEP

CEP is a powerul technology or analyzing multiple events over a specic period o

time,detectingcomplexpatternsandmakingcorrelations.FromanEvent-Driven

Architecture perspective, we can regard CEP as the key technology or processing in

real time the potentially high volumes o low-level events and transorming these

low-level events into higher-level, aggregated and composite business events or

visualization and automated response.

Forexample,CEPcanbeusedtodetectsuspiciouscreditcardusagebymonitoring

credit card activity, as it occurs. It can perorm time-series analysis and trending over

streams o events, and it can correlate a stream o real-time inormation with stored

and historical data, such as new credit card activity with customer inormation rom

a CRM system.

The need to intelligently process large volumes o real-time events exists across

a number o domains, including nancial trading, risk management, compliance

monitoring, Service Level Agreement (SLA) and Key Perormance Indicator (KPI)

Monitoring, supply chain management, clickstream analysis, cyber security,

business process monitoring, logistics, power grid monitoring, inrastructure

monitoring, military intelligence, and many others.

From an Event-Driven Architecture

perspective, we can regard CEP as

the key technology or processing

in real time the potentially highvolumes o low-level events and

transorming these low-level events

into higher-level, aggregated and

composite business events or

visualization and automated

response.Event – a message indicating that something has

happened

Event Stream or “Feed” – an ordered sequence

of events of the same type

Complex Event Processing – matching and

transforming source events into result eventsCEP

SOURCES RESULTS

8/3/2019 Doc Prod 00015



SLA Monitoring is an important application area or CEP. It is an important part o

most business operations as the speed o business moves to real-time rather than

batch-oriented operations. Here, CEP can lter large volumes o events to identiy

events outside o a specied range, correlate multiple eeds to determine whether

diverse events are related to a single issue, aggregate events to determine theextent or seriousness o the issue, and enrich detected issues with customer

inormation to determine which customers will be aected by potential or actual

SLA violations.

In the Utilities industry, ltering can be used to identiy meter readings rom

customers participating in Demand Response programs, which can then be

correlated and aggregated to determine i customers’ energy curtailment is on

track, while enrichment rom reerence data sources can be used to identiy the

economic impact o energy curtailment programs in real time.

Table 1 lists the most important undamental use cases or Complex EventProcessing and how they might be applied in some o the domains listed above.

These are the CEP building blocks which, when coupled with the ability to perorm

these unctions across vast amounts o data in real time, provide the basis or

building complex Operational Intelligence applications.

Use Case Description Example Application

Filtering Select events thatmatch user-speciedcriteria usingcomparisonexpressions

SLA Monitoring: detect whenresponse time exceeds a targetvalue (KPI threshold or limit) oreach region

Trading: lter high volume trades

Power Networks: captureabnormal sensor readings

Clickstream Analysis: captureorders rom a specic set o IPaddresses by geography

Correlation Join events romdierent eeds basedon common attributesand/or expressions

SLA Monitoring: join multipleproduct order eeds over slidingtime- or count-based windows

Exception Management: correlateinormation rom multiple systemsinvolved in a transaction or

business process

Cyber Security: correlate intrusionattempts rom multiple intrusiondetection systems, log les, etc.

8/3/2019 Doc Prod 00015



Use Case Description Example Application

Aggregation Compute variousstatistics rom eventdata over time- andcount-based sliding and

jumping windows,including Count,Average, Sum, Min,Max

SLA Monitoring: compute averageresponse over all regions

Network Management: compute

one-minute histogram o average,minimum, and maximum through-put

Clickstream Analysis: compute thenumber o visitors who click on aparticular link within a speciedtime interval

EventPatternMatching

Detect sequential fowo state-changes o oneor more event streamsover time (i.e., generatean alert if Event Aoccurred, and thenEvent B occurred within60 seconds)

SLA Monitoring: determinewhether linked SLA metrics areaected by one going out obounds

Fraud Management: detectsuspicious linked transactions

Trading: detect “wash trading”patterns

Clickstream Analysis: how long didthe custome take to click “Buy”ater opening an advertisementpage?

Enrichment Merge reerence datarom external DBMSsystems into analyticsmodels to provide ullbusiness context orunderlying events

SLA Monitoring: historicalcomparison (i.e., networkperormance last month); retrievedetail inormation rom masterrecords (look up customer detailsusing customer ID)

Retail: look up product data romRFID tag

Cyber Security: retrieve existingintrusions rom a particular IPaddress

Multi-dimensionalAnalysis

Compute variousstatistics rom the eventdata broken down byone or more attributes(dimensions)

SLA Monitoring: compute averageresponse per customer level, perpartner type, per region

Retail: compute sales volume perproduct, per region

SituationalAnalysis

Overlay contextualinormation (such asgeo-spatial inorma-tion) on event data

SLA Monitoring: visualize SLAviolations by geography

Logistics: mash up delivery feetlocations with Google Maps

Military Intelligence: monitormovements o people o interest

Network Management: provide“heat maps” o network through-put to highlight bottlenecks

Table 1: Sample CEP Use Cases and Applications

8/3/2019 Doc Prod 00015



The Importance of XML in Complex Event Processing

Dierent Types o CEP Engines There are two major types o CEP engines on the market:

• Query-based—appliesdatabase-likequeriesagainsteventstreams

• State-based—representsexpectedsequencesofeventsasnitestate

machine models

Query-basedCEPenginesarethemostgeneral-purposeandwidelyusedCEP

engines and they cover all the CEP use cases listed in Table 1. In contrast, state

machine-based CEP engines are designed to excel at sequence and pattern

matching with high transaction volumes over long time durations, but are less

exible than query-based engines in addressing the ull range o CEP use cases.

Implementing aggregations, correlations, time-series analysis, and so on in

state-based engines generally requires relatively complex programming in aproprietary event programming language, rather than using relatively simple

declarative queries.

Mostquery-basedstreamprocessingsystemsuseaderivativeofSQLastheirquery

modeling language. This is understandable given the amiliarity most developers

havewithSQL.However,ratherthanusingSQLasitsmodelinglanguage,theVitria

M3O Analytic Server is a query-based CEP engine specically designed to process

real-timeXMLeventstreamsandusesXQueryasitsquerymodelinglanguage.

WhataretheadvantagesofusinganXMLquerylanguageovertherelationalSQL

query language?

• Relationaltablesareat,whereasmostenterprisedata(including

“business objects”) are naturally represented and passed as hierarchically

structured objects, oten several levels deep. XML is a natural t or

representing this hierarchical structure.

• Relationaltablesarerigidlyuniform,whileXMLdatatendstobemore

highly variable. XML is much more adept at handling structural variations,

typing variations and missing data, than relational structures, and these

situations are very oten the norm in rapidly evolving systems.

• Relationaldataisnaturallyunordered,whileorderoftenhasavery

important meaning in XML data (particularly or document data).

• Relationaltableshaverelativelystaticschemasthatcanbedicultto evolve, while XML schemas tend to be more extensible, and the

“sel-describing” nature o XML blurs the data/metadata distinction.

These are many o the same “impedance mismatch” arguments made or the

advantages o object and native XML persistent storage engines and databases

over the more amiliar relational database management systems. However, while

the relational model remains ubiquitous or data at rest and has relegated native

XML persistence to relatively specialized applications, the case or data in motion is

It is essential that any Complex

Event Processing system can

eciently handle XML since most

real-time eeds will contain XMLmessages.

8/3/2019 Doc Prod 00015



very dierent. Today, most business data exchanged between systems is expressed

in XML ormat, both inside and across the rewall, taking advantage o XML’s

platorm independence, its sel-describing nature, and its extensibility. The

mainstream adoption o the Web Services model to implement Service Oriented

Architectures has rapidly accelerated this trend.

With so much data in motion expressed in XML it is essential that any Complex

Event Processing system can efciently handle XML since most real-time eeds

will contain XML messages. M3O Analytic Server is a query-based CEP engine that

uniquelycombinestheabilitytoqueryXMLeventstreamswithSQLqueriesfor

enrichment rom stored data.

StreamXQuery™: Efcient Queries on Real-time XML DataXQueryisthestandardquerylanguageforXMLdataanddocumentsandis

analogoustotheSQLlanguageforrelationalqueries.TheXQuerylanguageisrich

enough to support navigation within an XML input document, the combining o data rom multiple XML inputs, and the generation o new XML structures rom one

or more XML inputs.

ThemostimportantexpressioninXQueryistheFLWOR(pronounced“ower”)

expression,whichisanalogoustoSELECT-FROM-WHERE-ORDERBYqueriesinSQL.

ThegeneralformoftheFLWORstatementis:

for $variable1 in collection1, $variable2 in collection2, …

let $variable10 := value10, $variable11 := value11, …

where predicate

[group by partition-expression]

order by element-name

return result-expression

Example 1: XQuery FLWOR Expression Structure

Fordatahandling,XQueryhasmuchoftherichnessofSQLandmore—itincludes

support or sub-queries, union, intersection, dierence, aggregate unctions,

sorting, existential and universal quantication, conditional expressions, user-

dened unctions, and static and dynamic typing, in addition to various constructs

to support document manipulation, such as query primitives or element

order-related operations.

StreamXQueryisVitria’simplementationoftheXQuerylanguageintheM3O

Analytic Server. The implementation is designed and tuned specically to handle

continuousreal-timestreamsofXMLevents.TheStreamXQueryimplementation

providesthefollowingenhancementstothebaseXQuerylanguage:

• Anextensionfunction(i.e.,partoftheXQuerylanguagesyntax)for

dening event windows

• ArichsetofextensionfunctionsforCEPusescasesincludingsumming;

computingminima,maxima,averages;performingSQLqueries;

comparingaggregations;returningsysteminformation

M3O Analytic Server is a query-

based CEP engine that uniquely

combines the ability to query XML

event streams with SQL queries or enrichment rom stored data.

8/3/2019 Doc Prod 00015



• Geospatialextensionfunctionsthatincludecomputingthedistance

betweentwopoints;checkingwhetherapointisinsideacircular,

rectangular,orpolygonalgeo-fence;andreturningthelatitude,longitude,

or altitude o a given point

• Statisticalextensionfunctionstocomputelinearregression,standarddeviation, and variance

• Theadditionofthegroup byclauseintheFLWORsyntaxforgrouping

results

StreamXQueryTM WindowsIn order to perorm meaningul analytics on a continuous stream o events, the

stream must rst be bounded in order to create a collection o scoped events

uponwhichqueryprocessingmaybeperformed.InStreamXQuery,thestreamis

boundedbywindows.Awindowisanalogoustoatableinarelationalquery—it

represents the bounds o the data set.

Each event in a stream carries a timestamp. In Table 2 we show how the M3O

Analytic Server guarantees that events are processed strictly in time order

according to their timestamps and also how M3O Analytic Server supports

time-based windows based on this timestamp data as well as count- and

attribute-based windows.

Use Case Description

Time-based Indicates that only events entering M3O Analytic Server ina past time period as specied will be processed. The timerange can be expressed in Day, Hour, Minute, or Secondunits.

A trailing time-based event window takes a time intervalω as a parameter. At time λ, the event window containsall events with timestamps between λ – ω and λ.

Count-based Process only the specied number o latest events eachtime. An event range window takes an integer N > 0 as aparameter. At time λ, the event window contains amaximum o N events having the latest timestamps ≤ λ.

Attribute-based

For each distinct value o the attribute (i.e., customer IDor region name), store in the window the most recentevent having that value. Compound attributes aresupported.

Table 2: Window Types

Both time- and count-based windows can be either sliding or jumping. With sliding

windows, events move into and out o the window as time or the number o events

progresses. With jumping windows, events ll the window until the upper time- or

count-based boundary is reached. They are then discarded, at which point a new

window starts lling up with new events as they arrive.

WindowsaredescribedinStreamXQueryqueriesbyusingtheXQueryextension

unction qs:window(). The signature o this unction is:

qs:window($sn as xs:string, $wn xs:string?)

as document-node()*

8/3/2019 Doc Prod 00015



The argument$sn is the registered stream name, $wn is the window specication.

Based on the window specied in the qs:window()unction, an event stream is

pipedtotheStreamXQueryprocessorinM3OAnalyticServerwhichcreatesanode

collection o events that all into the window. This collection is then iterated over by

theforstatementofthequery’sFLWORexpression.

Some example count- and time-based window specications are described in

Table 3.

Window Specifcation Meaning

sliding size 10 Count-based sliding window size 10without “step” implies “step 1”

sliding size P1D Time-based sliding 1 day window with-out “step” means window moves oreach new timestamp (that is, the step isvariable)

sliding size P1D step P1M Time-based sliding window size 1 daywith step 1 minute

sliding size P1D step P1Dstart 2009-10-01T00:00:00Z

Time-based jumping window (size ==step), starting at midnight 1st October2009

Table 3: Window Specifcation Examples

A StreamXQueryTM ExamplePuttingallofthistogether,thefollowingexampleillustratesaStreamXQueryquery

that perorms analysis on a real-time eed o stock trades(StockFeed)using a

sliding window one minute in length with step one second. The events are ltered

or trade volumes greater than 5000 and grouped by stock ticker symbol. The result

is an XML document that contains each symbol name and a computation o the

aggregate total amount (price times volume) as the window slides.

declare namespace ns1 =

“http://schema.vitria.com/Event/StockTrade”;

declare option outputstream “StockFeed”;

for $v in qs:window(‘StockFeed’,

‘sliding size P1M step P1S start 1970-01-01T00:00:00Z’)

where $v/ns1:StockTrade/Volume >= 5000

group by $v/ns1:StockTrade/Symbol

return

<Report>

<Symbol>{fn:string($v/ns1:StockTrade/Symbol)}

</Symbol>

<TotalPrice>

{qs:sum((fn:number($v/ns1:StockTrade/Price) *

$v/ns1:StockTrade/Volume ))}

</TotalPrice>

</Report>

Example 2: Stock Feed StreamXQueryTM Example Query

8/3/2019 Doc Prod 00015



Advantages of StreamXQueryTM over SQL-based Query EnginesVitria’sapproachtoquery-basedCEPisuniqueduetoitsuseofStreamXQuery

ratherthantakingthemorecommonSQL-basedapproach.ThisgivesM3OAnalytic

Server the ability to query complex XML types on the y directly without the need

to rst marshal the XML data into tabular data structures. By perormingincremental logic on each event, the number and size o computationally expensive

joins is reduced.

ThisisillustratedinFigure2,whereinordertodeterminetheIncompleteOrdersby

Customer&Productqueryoverthelasttwominutes,aSQL-basedenginewould

have to:

• Normalizethethreeeventfeedsintofourtabulardatastructures.The

complex XML event o Orders & Line Items would need to be attened into

two tabular structures to handle the nesting in the XML document.

• Performninejoins—includingajoinagainsttheOrderandOrderLineItems tables that were created to atten the Order & Line Items document,

as well as eight other joins, continually looping back to each table.

This is a computationally expensive approach, especially as the number and

complexityoftheeventsgrows.Incontrast,theStreamXQuery-basedM3OAnalytic

Server executes as ollows:

• QueriessimpleandcomplexXMLtypesdirectlywithoutneedingto

normalize

• Performsqueriesandcalculationsincrementallyonresultsets,limitingthe

total number o joins to just three in this example

Vitria’s approach to query-based

CEP is unique due to its use o

StreamXQuery rather than taking

the more common SQL-based approach.

Order & Line ItemsOrder & Line Items

ProvisioningProvisioning

Activation LogActivation Log

SQLSQL

SQLSQL

SQLSQL

XQueryXQuery

XQueryXQuery

XQueryXQuery

SQL-based

Simple & Full Complex Event Types

Event Type NormalizationReal-time

Event Sources

(XML)

Real-time

Event Sources

(XML)

Total Orders by product

line last 30 seconds

Order Lines

Service

Activation

Order Customer Orders

by product line = AB1234

for last 1 minute

Customer Orders

by product line = AB1234

for last 1 minute Incomplete Orders

by Customer &

Product for

last 2 minutes

Incomplete Orders by

Customer & Product for

last 2 minutes

Join

Join

Join

Order & Line ItemsOrder & Line Items

ProvisioningProvisioning

Activation LogActivation LogJoin

StreamXQueryTotal Orders by product

line for last 30 secondsOrder

Order ID

Date

Status

Customer

Customer ID Order

Order ID

Date

Status

Customer

Customer ID

Order

Order ID

Date

Status

String

Name

Network

CustomerID

Customer

Customer ID

Order

Order ID

Date

Status

String

Name

Network

CustomerID

Customer

Customer ID

<a><b>

</b></a>

<a><b>

</b></a>

Join

Figure 2: Advantages oStreamXQueryTM Over SQL-

based Query Languages

8/3/2019 Doc Prod 00015



M3O®: A Modern Architecture or CEP

Complex Event Processing sits at the heart o the M3O Operational Intelligence

architecture.InFigure3weseethateventscancomefromavarietyofsources,

including transports such as JMS, Web Services, relational databases, and web-

based eeds such as RSS. The diagram also illustrates that business process

execution engines such as M3O Business Process Server (or even third-party process

engines) can also provide process execution events as a eed so that those events

can be correlated and visualized with events rom other eeds.

TheseeventsourcesarenormalizedasfeedsbytheM3OFeedServerandthefeeds

are streamed into the M3O Analytic Server (the CEP engine) or computation. The

M3O Analytic Server then perorms in-memory operations against the eeds,

writing results to the repository and archives or later auditing and analysis,

painting real-time graphical dashboards or human decision making, and initiating

actions in the orm o outbound alerts or the execution o event resolution business

processes in the M3O Business Process Server.

The M3O Analytic Server can also query data persisted in relational databases and

combine that with real-time streamed data to enrich the streamed data, and to

provide trending and comparisons against historical inormation.

Figure 3: CEP High Level Architecture

M3O® Query Modeler

TheM3OQueryModeleristhegraphicalenvironmentinwhichanalystsand

developersmodelStreamXQueryandSQLqueriesusingdrag-and-dropandpick-

list selection techniques. The graphical UI allows users to quickly build queries using

either the graphical techniques or by manually editing queries, depending on the

user’sfamiliaritywithXQueryandSQL.Queriesarestoredinashared,version-

The M3O Analytic Server can also

query data persisted in relational

databases and combine that with

real-time streamed data to enrichthe streamed data and to provide

trending and comparisons against

historical inormation.

ESBESB

Event Sources Result Feed Consumers

M3O

BPM-Workflow

M3O Dashboards

M3O BPMS

Custom Apps

App Databases / Warehouses(Enrichment/Historical Comparison)

M3O Policy Manager

Event Archive Results Archive

BPM-Workflow

Web Services, JMS

Relational Databases

RSS, Web Feeds

Custom Apps

M3O Query Modeler

M3O Feed

Server

FeedSource

Process Audit

Workflow Audit

Custom

Feeds

DataCollector (Display)

Result FeedArchive

(Playback)

Result Feed

FeedArchive

FunctionsCorrelation Analysis

Presentation Manager

M3O Analytic Server

Aggregation

Transformation

Ordering

Numeric

PatternFilters

Join

SequenceMatching

Real-Time(Continuous)

ContextualDrill-down

Multi-Dimensional

MovingAverages

Trends

Time Series

Data TypesSimple/Complex

Date Time

8/3/2019 Doc Prod 00015



controlled repository which promotes collaboration while providing control over

the versions o the query models deployed in the M3O Analytic Server runtime.

Figure4illustratesagraphicalXQueryquerybeingconstructedinanSLA

management application. In the interace, users wire the let-hand side, whichcontains eeds and the data on those eeds represented by their XML Schema, to

the right-hand side which contains the query result along with the window

specication and any grouping constructs. The wiring includes examples o

intermediate unctions being applied to the data as it is mapped rom source eed

toresultschema.Functionsareaddedtothewiringusingdraganddropfroma

palette that includes unctions or perorming aggregations, numeric calculations,

date-time, geospatial, and custom processing.

Figure 4: M3O® Query Modeler—Graphical XQuery Modeling

QueriesarestoredasmodularcomponentswhichcanbewiredtogetherintoEvent

Processing Networks in the graphical interace. In this way, sub-queries can be

reusedinthesameormultipleprojects.Figure5showssourceandintermediate

eeds being wired together with queries in an Event Processing Network used to

create a queue backlog query in the SLA management application.

Figure 5: Wiring an Event Processing Network

8/3/2019 Doc Prod 00015



M3O® Feed Server

M3OFeedServeristhearchitecturalcomponentdedicatedtomanagingevent

sources, or eeds. To ensure optimal scalability and extensibility, the management

o eeds is separated rom the event analysis perormed in the M3O Analytic Server.

M3OFeedServerprovidesarobust,enterprise-gradequalityofserviceforfeedsto

support subscriber recovery, event archive, snapshots, scalable deployment, and

collaborative sharing o eeds across multiple projects. It provides connectivity to

traditional and non-traditional enterprise sources, including JMS, RSS, Web Services,

and relational databases.

M3OFeedServerhasaplug-and-play,distributedarchitecture.Thearchitecture

can scale to manage varying and unpredictable load volumes by increasing and

decreasingthenumberofFeedServerinstancesasrequired.Usingthisapproach,

M3OFeedServercanstreammillionsofreal-timeeventsamongmultiplepublishers

and consumers.

Inadditiontostandardizedfeedconnectivity,M3OFeedServerprovidesenterprise

qualityofservice(QoS)tounreliablefeedsources,includingdurability,scalability,

and availability o high volume eeds in order to acilitate real-time analysis.

Guaranteed message delivery ensures that subscribers receive events in proper

event order and that no duplicate events are transmitted.

InadditiontosupportingQualityofService,the Event Archive provides the ability

or analysts to conduct orensic what-i analysis o eeds with interactive, point-

in-time playback o events rom the archive. This analysis can be used in an SLA

management use case, or example, to allow the analyst to understand past

perormance so that SLA thresholds can be intelligently recalibrated.

M3OFeedServersupportsthecreationofSnapshots that eectively give the

analyst the ability to create on-the-y data marts rom events persisted in the Event

Archive. Snapshots can be used to compare current perormance with historical

perormance to detect trends and to understand deviations rom past perormance

Data interoperability with traditional BI systems can be used to enhance existing BI

systems with snapshots rom real-time eeds.

Summary windows maintain the state and history o a eed and allow that history

to be instantly replayed. Analysts are able to access and play back a eed state

immediately, even i the connection to the eed is broken or i they are a new

subscriber and need to catch up with the eed history in their dashboards.

M3OFeedServerprovidesfederatedcollaborationoffeedsacrossmultiple

organizational groups and business units. By providing access to shared eed

instances,M3OFeedServerpromotescollaborationandconsistencytogetherwith

the highest quality o service.

M3O Feed Server provides a robust,

enterprise-grade quality o service

or eeds to support subscriber

recovery, event archive, snapshots, scalable deployment, and

collaborative sharing o eeds across

multiple projects.

8/3/2019 Doc Prod 00015



M3O® Analytic Server

M3O Analytic Server is the CEP core o the M3O Operational Intelligence platorm.

ItmanagesmultipleconcurrentinboundfeedsfromtheM3OFeedServer,performs

StreamXQueryqueriesonthosefeeds,andgeneratesasetofoutboundresultfeeds

asillustratedinFigure6.

Figure 6: M3O® Feed Server and M3O® Analytic Server

Aresultfeedcontainsoutput(XML)eventsresultingfromStreamXQueryquery

execution. A result eed can be used in several ways:

• Asaninputtofurtherdownstreamqueries(modularsub-query

componentscanbegraphicallyassembledusingtheQueryModelerinto

query networks)

• Asachannelforreal-timeeventvisualizationinM3Odashboards

• ByM3OPolicyManager,sothateventsinthefeedcanbedispatchedto

business processes in the M3O Business Process Server or handling and

workow in a BPMN business process

• Byanysubscriberapplication(viaanoutboundM3Ofeed)

M3O Analytic Server continuously executes each query dened in the runtime upon

arrival o a new event into any o the windows a query reerences. Event

processing is implemented as a sliding window algorithm where events are

processed in chronologically ascending order based on their timestamps across all

windows in a query. Such query-wide event process ordering is crucial to producing

results that accurately reect the context at the specied point in time. M3O

Analytic Server buers inbound eeds persistently until the events on those eeds

are processed, thereby guaranteeing that no events are lost beore processing.

FeedFeed FeedFeed

M3O Analytic Server M3O Analytic Server

QueryQuery

EventEvent EventEvent

StreamStream StreamStream

DBDB

M3O Feed Server

Event

XQuery (window,aggregation,

correlation,

enrichment, etc.)

Even

M3O Analytic Server provides a

wide array o powerul analytical

and calculative eatures that enable

users to analyze, understand, and

act upon inormation in real time.

8/3/2019 Doc Prod 00015



Incremental query processing is an internal optimization technique o the M3O

Analytic Server query processor that minimizes the computational overhead o

continuous query execution. M3O Analytic Server uses the incremental calculation

algorithm as much as possible, particularly when computing aggregations.

M3O Analytic Server provides a wide array o powerul analytical and calculative

eatures that enable users to analyze, understand, and act upon inormation in real

time. These eatures are summarized in Table 4.

Feature Capability

Real-time event capture, ltering,pattern matching and detection,and aggregation

Detect patterns and trends in real time

Correlate real-time and historical data

Detect and capture specic businessevents

Filter signicant events

Aggregate and enrich inormation

Real-time predictive analytics,trending, orecasting, and what-ianalysis

Compute trend lines in real time andexplore deviations

Conduct orensics o real-time events

Recalibrate Service Level Agreements

Moving averages over temporalconditions

Analyze data over various timewindows

Easily adjust time windows and seeanalysis immediately

Recalculation on the fy

Multi-dimensional analysis ocontinuous real-time eeds

View and analyze data along a numbero dierent dimensions

Control data altitude through drill-down

Snapshots View incremental inormation andprovide additional event processing tourther analyze the data and take real-time actions

Analyze relevant time dimensions andhistorical context

Geospatial analytics Situational Awareness: deduce andanalyze geographical relationshipsbetween people and entities

Dene by specied geoence (circle,rectangle, polygon) or by entity(including entry and exit o geoence)

Track proximity o key objects

Mash up and display on Google Maps

8/3/2019 Doc Prod 00015



Feature Capability

Linear regression Statistical unctions using linear squaresalgorithm or best-t line analysis tocompute:- Slope, Intercept, Standard Deviation,Variance

- Regression (r2)- DateTime-to-Milliseconds, DateTime-to-Seconds

- Milliseconds-to-DateTime, Seconds-to-DateTime

Provides oundation or dashboardwidgets to display linear regression oreal-time data series, layering with SLAthresholds, and predictive orecastingo SLA violation times

Table 4: M3O® Analytic Server Features and Capabilities

Presentation Manager and Open Query API The presentation manager provides the interace between queries created in the

M3OQueryModelerandtheirexecutionintheM3OAnalyticServer.The

presentationmanageralsoprovidestheOpenQueryAPIthatallowspartnersand

customerstousecustombuiltorcommercialo-the-shelf(COTS)FlexorJava-based

applications to interace with and push queries into M3O Analytic Server. The API

enablesnon-VitriaFlexandJavaclientstoaccessM3OAnalyticServerto

programmatically create new queries or modiy existing queries, dene result

destinations, and so on.

Scaling Complex Event Processing in M3O®Perormance is crucial in real-time computing and analytics. Equally important is the

ability to scale perormance in line with event patterns in order to deliver continued

perormance over time.

Event sources, event volumes, and analytic complexity may scale in line, but more

likelywillscaleindependentlydependingontheevolutionofthebusiness.For

example, we can imagine a scenario with hundreds o event sources and millions

o events, and yet very simple analytical computations. Conversely, we can also

envision situations with very ew sources and relatively low event volumes that

nonetheless require intensive calculations, or perhaps relatively long computationatime windows, thereby requiring the server to hold events in memory or extended

periods.

In the high-volume-low-complexity scenario, we would want to scale up and cluster

theM3OFeedServertohandlethenumberofeventsourcesandeventvolume,but

we may be able to unction with a relatively smaller M3O Analytic Server ootprint

due to the lighter processing load. In the low-volume-high-complexity scenario,

thescalingmightbereversed:arelativelysmallFeedCluster,butarelativelylarge

Analytic Server Cluster ootprint to work through the more complex processing and

long time window horizons.

Vitria’s architectural approach

allows or fexible scalability by

separating the M3O Feed Server

rom the M3O Analytic Server and

enabling clustering on each

independently.

The Open Query API enables

non-Vitria clients to access M3O

Analytic Server to programmatically

create new queries, modiy existing

queries, start and stop queries, and

dene query result destinations.

8/3/2019 Doc Prod 00015



Vitria’s architectural approach allows or this kind o exibility by separating the

M3OFeedServerfromtheM3OAnalyticServerandenablingclusteringoneach

independently.ThisexiblescalingapproachisillustratedinFigure7.

Figure 7: Flexible Scaling o M3O® Feed Server and M3O® Analytic Server

Conclusion

As we have shown, Complex Event Processing orms the cornerstone o Operational

Intelligence. Vitria’s M3O Operational Intelligence platorm provides a ully-

integrated suite that combines a scalable CEP engine specically designed orhandling XML events efciently together with a modern, Web 2.0 interace or both

design-time modeling and run-time visualization, and a business process

engine to enable automated responses. By providing the real-time insight into

currentbusinessoperations,M3OQueryModeler,M3OFeedServer,andM3O

Analytic Server enable important inormation to be analyzed, visualized, and acted

upon when it matters most.

Feed AFeed A

Feed BFeed B

Feed Server (1b)Feed Server (1b)

Feed Server 1

DatabaseClustering &

Failover

Feed Cluster Analytic Server Cluster

Feed CFeed C

Feed DFeed D

Feed Server (2b)Feed Server (2b)

Feed Server 2

CEP Engine (1b)CEP Engine (1b)

CEP Engine 1

CEP Engine (2b)CEP Engine (2b)

CEP Engine 2

Feed

Messages

Feed

Messages

QueryStatesQueryStates

FeedMessages

FeedMessages

QueryStatesQueryStates

Event SourcesEvent Sources

StreamXQuery 1StreamXQuery 1

StreamXQuery 2StreamXQuery 2

StreamXQuery 4StreamXQuery 4StreamXQuery 3StreamXQuery 3

About Vitria

Vitria Technology, Inc. is the industry’s

leading Operational Intelligence company.

Our innovative Operational Intelligence

solutions empower customers to analyzebusiness activities in process and take

real-time action. The result is better

decisions when they matter most—before

opportunities have faded or problems have

escalated. With a rich heritage as a pioneer

of BPMS, Vitria’s award-winning solutions

provide the backbone for many Global

2000 companies’ mission-critical business

processes. Vitria has customers in North

America, South America, Europe, Asia, and

Australia.

8/3/2019 Doc Prod 00015


8/3/2019 Doc Prod 00015


945 Stewart Drive, Sunnyvale, CA 94085

Tel: +1 (877) 365-5935

Email: [email protected]

www.vitria.com

© 2010 Vitria Technology, Inc. All rights reserved. Vitria, BusinessWare,M3O, Operations Book, and StreamXQuery are trademarks or registeredtrademarks of Vitria Technology, Inc. in the United States and othercountries. All other brand or product names are trademarks or registeredtrademarks of their respective owners companies or organizations

doc prod 00015

Documents