real- time analytics – process automation
TRANSCRIPT
This document is offered compliments of BSP Media Group. www.bspmediagroup.com
All rights reserved.
First National Bank – a division of FirstRand Bank Limited. An Authorised Financial Services and Credit Provider (NCRCP20).
Real-Time Analytics Process Automation
Nico Coetzee
Big Data
• Volume
• Amount of data
• Velocity
• speed of data in and out
• Variety
• Range of data types and sources
Volume
• Examples:
• User profile database
• Inventory
• Real Time Analytics Scenarios:
• Measure changes over time
• Assist ERP systems with automation (order
stock)
• Stock load balancing (automatically redistribute
stock from one area to another)
Velocity
• Examples:
• POS Transactions (think big national retailers)
• Logs (firewall logs)
• Real Time Analytics Scenarios
• Sudden unexpected patterns (a region
experiences an outage)
• Attacks and other anomalies that can be picked
up from logs
Variety
• Examples:
• End-to-end transaction flows through Web
logs, Application server logs and database logs
• Real Time Analytics Scenarios:
• Continues monitor response times (very handy
for Cloud type solution where decisions to start
more VM’s may be required)
• Context required for making intelligent
decisions, for example in anti-fraud systems
The “Missing” V’s
• Viability
• The secret is uncovering the latent, hidden
relationships among these variables.
• Value
• Remember: Correlation does not mean
causation
• Realistic scenarios: Was your marketing
campaign successful?
Technologies to Consider
• MongoDB
• NoSQL DB
• Distributed operations (Grid FS)
• Built-in Map-Reduce engine
• Syslog-ng
• Real time decisions on log events
• Granular control over log routing
• Rules based on regular expressions
The Future
• DevOps and Agile methodologies can
benefit from the inputs from real time Big
Data analytics
• Identified (potential) defects should naturally
flow back into the backlog.
• Infrastructure and resource management
Thank You