Pa#erns  for  Deploying  Analy2cs  in  the  Real  World  

Anjana  Fernando  Senior  Technical  Lead,  WSO2  

Analy2cs  Pa#erns  

•  Batch  Analy2cs  •  Real-­‐2me  Analy2cs  •  Interac2ve  Analy2cs  •  Predic2ve  Analy2cs  •  Batch  +  Real-­‐2me  •  Real-­‐2me  +  Predic2ve  • WSO2  Solu2ons   Source: data-analytics.github.io

Batch  Analy2cs  

Genera4ng  insight  by  processing  large  amounts  of  stored  data    

●  KPI Statistics ○  Application Statistics Monitoring ○  Network / Service Statistics ○  Sensor Data Aggregation

●  Solving Optimization Problems ○  Urban Planning ○  Revenue Distribution Analysis

Source: www.e-deal.com

Batch  Analy2cs  In  Ac2on:  API  Sta2s2cs  

Batch  Analy2cs  In  Ac2on:  API  Sta2s2cs  

Batch  Analy2cs  In  Ac2on:  HTTP  Monitoring  

Real-­‐2me  Analy2cs  

Making  sense  of  fast  moving  data    ●  Sports

○  Real-time Analysis of Player Performance ○  Real-time Match Analysis

●  Geo-Spatial ○  Traffic Monitoring and Alerting ○  Geo-fencing

●  Anomaly Detection ○  Fraud Detection ○  Network Intrusion Detection ○  Server Health Monitoring

Source: www.promojam.com

Real-­‐2me  Analy2cs  In  Ac2on:  TFL  

Real-­‐2me  Analy2cs  In  Ac2on:  TFL  

Interac2ve  Analy2cs  

Near  Real-­‐4me  Indexed  Data  Search      

●  Log Analysis ○  Application / System Logs

●  Activity Monitoring ○  Tracking Message Flows

●  Fraud Detection ○  Executing queries to lookup related

data in a detected fraud situation ●  HL7 Data Exploration

○  ESB HL7 Transport Interfaced with DAS

Source: befoundonline.com

Interac2ve  Analy2cs  In  Ac2on:  WSO2  DAS  

Predic2ve  Analy2cs  

Analyze  Exis4ng  Data  to  Predict  Future  Events    ●  Next Value Prediction

○  Sales Forecasts ○  Electricity Loads

●  Classification ○  Product Categorization ○  Customer Segmentation

●  Anomaly Detection ○  Fraud Detection ○  Preventive Maintenance

Predic2ve  Analy2cs  In  Ac2on:  WSO2  ML  

Batch  +  Real-­‐2me  Analy2cs  

Using  batch  processing  results  in  real-­‐4me  flows    

●  Batch Processing done for processing historical data ●  Real-time Processing done at the same time for

understanding present snapshot situation ●  Overlay summarized historical data with real-time

data to compare current vs past behavior

Source: marketplace.rackspace.com

Batch  +  Real-­‐2me  Analy2cs  In  Ac2on:  CEP  Event  Tables  

●  WSO2 DAS Analytics Event Tables ○  CEP Event Table extension used to lookup

persisted analytics table data in DAS ○  Used in CEP execution plans to lookup/join/

enrich real-time event streams

Batch  +  Real-­‐2me  Analy2cs  In  Ac2on:  No2fica2ons  ●  WSO2 DAS Spark Event Publish Relation Provider

○  Spark SQL jobs are able to send batch job results as events back to the server as events, to be processed by CEP and send notifications

Real-­‐2me  +  Predic2ve  Analy2cs  

Using  Predic4ve  Analy4cs  results  in  real-­‐4me  flows    

●  Machine learning models first created ●  The ML model is used in real-time execution by

applying the model with parameters extracted from the flow

●  Use Case: Unknown Fraud Modelling ●  WSO2 CEP meets ML

○  WSO2 CEP StreamProcessor extension for ML provides a way to apply ML models in execution plans

e.g. :- from InputStream#ml:predict('registry:/_system/governance/ml/indian-diabetes-model', 'double') select * insert into PredictionStream;

WSO2  Solu2ons  Based  on  the  Analy2cs  PlaOorm  

●  WSO2 Fraud Detection Solution ○  Built for detecting credit card fraud ○  The rules extensible with customized Siddhi

execution plans for any type of fraud detection ○  Currently uses Real-time and Interactive

Analytics features ●  WSO2 Log Analytics Solution

○  Distributed indexing and searching of any type of logs stored in the system

○  Notifications support with Real-time event processing features

○  Application / Server health prediction with Machine Learning

○  Uses Interactive + Real-time Analytics + Machine Learning features Source: www.retrospective.centeractive.com

Source: multichannelmerchant.com

Thank  You