dr gareth smith

INNOV-17: How to Build Event Stream Processing (ESP) and Business Activity Monitoring (BAM) into Your Application

Dr Gareth SmithVP Products

Progress ApamaPrincipal ArchitectProgress Apama

Dr John Bates

© 2006 Progress Software Corporation2INNOV-17: How to Build ESP and BAM into Your Application

Agenda

Interactive Introduction to Apama Scenario Creation

Definitions

• What is Event Stream Processing (ESP)?

• What is Business Activity Monitoring (BAM)?

Apama platform

• Architecture & components

Building an example application

• Revenue Assurance

Conclusions


Event Stream Processing: “When 3 credit card authorizations for the same card occur in any 5 second

window, deny the request and check for fraud.”



Event Stream ProcessingA New Computing Physics

time1 2 3 4 5 6 7 8 9

Static Data Processing: “How many shoes did we sell in our New York

stores last week?”

Static Data Processing: “How many shoes did we sell in our New York

stores last week?”






ESP & BAMA New Computing Physics

time1 2 3 4 5 6 7 8 9

Business Activity Monitoring: “Show on a dashboard (a) how many credit card transactions we

have processed today (b) average & peak transactions per minute (c) how many fraudulent transactions per

hour.”

Business Activity Monitoring: “Show on a dashboard (a) how many credit card transactions we

have processed today (b) average & peak transactions per minute (c) how many fraudulent transactions per

hour.”


Example Event Driven ApplicationsMonitor, Analyze, and Act on Business Conditions in Real Time

Application Event Examples Event-Driven Analysis & Action

Customer Service

Customer requests, CRM records

Alert me if an open customer request with priority > 3 did not get any reply event for at least 2 hours

Alert me if an open customer request was reassigned more than 3 times

Supply ChainInventory system, warehouse deliveries, warehouse restock

When (Warehouse 1 capacity < 50%) and (Warehouse 2 capacity < 30%) and

(Warehouse 3 capacity < 25%)Alert the head of operations with the average capacity in each of the warehouses

RFID/Supply Chain

RFID reader detecting EPC codes, goods delivery schedule

When pallet is loaded onto the wrong truck alert warehouse manager

Automated Trading

Stock tick & quote data

Within any 20 second window, when HP rises by more than 2%, but IBM doesn’t, buy IBM

Transportation Smartcard ID readWhen the rider travels more than 5 times within zone 1, charge at single-ride rate for subsequent trips


What do events look like?

Events are typed messages describing a change within a system• Example:

– NewInvoice(customer, product, amount)– StockTick(symbol, price, volume)– TruckLocation(truck, driver, cargo, x,y,z)

Events are analogous to a new entry in a database table or a Sonic message

Detecting patterns in individual events or a correlation of several events over time can indicate an opportunity or threat to the business


Business Benefits of ESP

Monitor, Analyze, ACT• The ability to take an action immediately when a business

scenario is detected– Enables your application to respond instantly to opportunity or

threat

It’s easy to add powerful capabilities to your existing application• You already have the events flowing in your system – you

just need to send them into Apama

Add real-time rules to your application without continually changing your code• You don’t have to go on extending your application• Once the events are flowing into Apama, simply add or

modify a rule in Apama• Rules can be defined by business or technical users

– Codeless and code based development


Agenda


Definitions



Apama Platform




Conclusions


Apama Product Suite

Event Store

Dashboards

Integration Adapter FrameworkIntegration Adapter Framework

Developer StudioApama

IDEDashboard

StudioScenario Modeler

Research Studio

Event Manager


MonitorScriptVirtual MachineMonitorScript

Virtual Machine

JavaVirtual Machine

JavaVirtual Machine

EventsEvents EventsEvents

C/C++C/C++

Composite event

sequencer

Composite event

sequencer

Match Notification

Event Registration

Scenarios

Multi-dimensional

event matcher

Multi-dimensional

event matcher

Event Input

Queue

Event Input

Queue

Inside the Event Manager

Event Output Queue

Event Output Queue

Feedback


Monitorscript

monitor RingtoneServiceMonitor {

SMS-Request request;

action onload {

on all SMS-Request (service=“Ringtone"):request {

on SMS-Request (user = request.user) within(60.0) {

emit UpdateBilling(request.user, “Double Discount”);}

} }}

event SMS-Request { string user; string service; location loc;}


JMON: Monitors in Java

import com.apama.jmon.*;

public class BillRequestService implements Monitor, MatchListener {

public void onLoad() {

EventExpression eventexpr = new EventExpression(“SMS-Request(service=\“Billing\”)”);

eventexpr.addMatchListener(this); }

public void match(MatchEvent event) {

System.out.println(“Billing Information Requested”); }}


Apama IDE


The Apama Scenario Modeler Enables Graphical Construction of ESP Scenarios

Intuitive visual user interface designed for business analysts

Express time-based real-time rules with a high level development tool

“SmartBlocks” encapsulate pre-packaged modules made available to non-programmers.

Each scenario, or group of rules, represents a “pattern” which can be adjusted by business users to specify conditions to monitor, analyze and act on.


Apama Dashboard StudioCreate Customized Dashboards for Your Real-Time Business Processes

Select from a palette of graphical objects. Each object can be laid out in a graphical dashboard and bound to Apama events

Users can customize the look and feel of all widgets, and specify which Apama event scenarios to visualize.

• Real-time variables and analytics can be visualized using graphs, charts, tables etc.

• Enables event-driven Apama logic to be visualized in real-time

• Deployment options from standalone dashboard to thin client portal


Apama

Real-Time Event Processing

Event Store - Capture

• Event Store captures all events in time series order

• Event Store is also able to capture derived events, e.g. Moving Average, from Event Manager

• Event Store can capture all output from Event Manager as an audit trail

EventStore

Historical Event Processing

Apama Dashboards


Apama Research Studio Business Intelligence

Event Store – Replay & Backtesting

• Enables “pre-flight testing” of Apama scenarios with historical data sequences

•Output of runs can be captured in Event Store and charted

•Also enables “digital forensics” – using recorded information to tune future performance

Apama Dashboards

Apama

Real-Time Event Processing

EventStore

Historical Event Processing


Integration Adapter Framework

Purpose of an adapter• Receive an event stream

or create an event stream– Intercept transactions– Use database triggers– Subscribe to Sonic

messages

• Normalize to Apama event format for efficient analysis & response

• Convert Apama events back to actionable messages OpenEdgeSonic Bus

Normalization Normalization

Native Sonic Messages

Transport Integration

APIPublish/ subscribe

Transport Integration

Event Manager

Mapping

Apama Events

Mapping


Agenda


Definitions



Apama Platform




Conclusions


Building an Example Application

Case Study: Revenue Assurance• Ensuring that all service requests are both fulfilled and

billed correctly and within a timely manner• Monitor via complex KPIs and present an graphical

overview of the status• Alert relevant staff with specific problems• Undertake Autonomous Actions: modify the Service

QoS settings to resolve problems

Same underlying principles are valid for tracking & billing in Supply Chain, ERP, Logistics etc.


A (very) simplified revenue chain

Switch (SMSC, MSC, SIP server)

ApplicationServer(s)

BillingUser’s mobile

1 2

3

45

Processa) User requests a service (1 + 2)b) Service is fulfilled (4 + 5)c) Request is Billed (3)

Problem• Any message (1-5) may be lost, corrupt or delayed• Depends on the throughput and the QoS

Parameters (Complex-KPIs) that we calculate:• Revenue leakage: requests that are not billed• Overcharging: requests that are billed but not fulfilled


Scenario

Smart blocks

Our revenue assurance demo package

1 2

3

45

ReconciliationValue Added

ServiceBilling

- Calculate parameters- Apply treshold and raise alarms- Take corrective actions (e.g. adjust QoS)

Dashboard- Display what’s going on- Monitor the alarms- Notify leakages that can’t be fixed

Switch (SMSC, MSC, SIP server)

ApplicationServer(s)

BillingUser’s mobile


Demonstration


Agenda


Definitions



Apama Platform




Conclusions


Conclusions

ESP & BAM enables• Rules to be defined external to the application to monitor,

alert & respond to key business opportunities & threats• Vizualization of key business events on real-time

dashboards

In many cases events are already flowing through existing applications

Apama enables• Easy connection to existing applications• Graphical ESP rules definition for rapid development• Graphical dashboard creation


Questions?


Thank you foryour time

dr gareth smith

Documents

progress software corporation

event stream processing

business scenario

business conditions

apama scenario creation

credit card transactions

individual events

credit card authorizations