why cloud, iot, machine learning and cybersecurity...
TRANSCRIPT
© 2017 Striim, Inc. All rights reserved.
Steve WilkesStriim Co‐Founder and CTO
Why Cloud, IoT, Machine Learning and CybersecurityAll Need Streaming Integration & Analytics
© 2017 Striim, Inc. All rights reserved.
Enterprise, Cloud and IoT Are Not Islands
Enterprise Cloud IoT
© 2017 Striim, Inc. All rights reserved.
They Are Part of A Connected Eco‐System
Enterprise
Cloud IoT
© 2017 Striim, Inc. All rights reserved.
Part of a Digital Transformation that Includes AI
Enterprise
Cloud IoT
Predictive maintenance/ part management
Cybersecurity Analytics CGM predictive monitoring
Automating customer engagement through smart botsAI driven
adaptive AML
NLP Call Center/Sentiment AnalysisRetail Banking
Machine Learning
© 2017 Striim, Inc. All rights reserved.
Every Industry Is Undergoing Digital Transformation
Financial Services Healthcare Manufacturing
Retail Communication Transportation/Logistics
ITInsurance Public Sector
© 2017 Striim, Inc. All rights reserved.
Data Generation Rates Are Growing Exponentially
0
20
40
60
80
100
120
140
160
180
2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025
Data Generated and Stored Annually (Zettabytes) *
Other Data Realtime Data Available Storage
Today We Generate Around 16ZB Data Annually
By 2025 This Will Leap to 160ZB
By 2025 25% Of All Data Will be
Real-Time
Only A Small Percentage Of This Data Can Be Stored
About 5% Of This Is Real-Time Data
* Data Age 2025: The Evolution of Data to Life-Critical. An IDC White Paper, Sponsored by Seagate
95% Of Real-Time Data Will Be
Generated By IoT
© 2017 Striim, Inc. All rights reserved.
If You Can’t Store All Data – What Can You Do?
0
20
40
60
80
100
120
140
160
180
2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025
Data Generated and Stored Annually (Zettabytes) *
Other Data Realtime Data Available Storage
* Data Age 2025: The Evolution of Data to Life-Critical. An IDC White Paper, Sponsored by Seagate
© 2017 Striim, Inc. All rights reserved.
Not Just IoT Data is Massive – Cybersecurity For Example
SERVERS &SERVICES
NETWORKDEVICES
SECURITYDEVICES
How do you correlate all events for immediate insights and proactive responses?
How do you avoid losing or ignoring valuable data, while still storing only the minimum?
How do you act promptly to better serve customers, protect reputation, and beat competitors?
© 2017 Striim, Inc. All rights reserved.
All Data Arrives in Streams Not Batches
databasehumans
eventsdevices
logsmachines
… stream processing hasemerged as a majorinfrastructure requirement
streaming
© 2017 Striim, Inc. All rights reserved.
Data Modernization
Moving to a “streaming first” data architecture, cloud, streaming analytics, and IoT.
Bridge the old and new worlds of data.
© 2017 Striim, Inc. All rights reserved.
Customer Comments on Data Modernization
Our Legacy Systems Can’t Keep Up…““ ””
Our Legacy Systems Can’t Keep Up…“ ”We Need to Prepare
For a Huge Increase In Data Volumes““
””We Need to Prepare
For a Huge Increase In Data Volumes“
”
We Need to Manage a Massive Up‐Tick in Integration & Analytics Requirements
““””
We Need to Manage a Massive Up‐Tick in Integration & Analytics Requirements
“”
…We Can’t Just Rip and Replace Systems““ ””
…We Can’t Just Rip and Replace Systems“ ”
New and Old Systems Need to Co‐Exist Until The Old Systems can be Retired. With Failover…
““””
New and Old Systems Need to Co‐Exist Until The Old Systems can be Retired. With Failover…
“”
Our Current Latency is Way Too High…““ ””
Our Current Latency is Way Too High…“ ”
© 2017 Striim, Inc. All rights reserved.
Part of Overall Data Architecture
Multiple data sources
(Existing) ETL Jobs
Streaming Integration & Analytics
Bat
ch /
Hig
h-La
tenc
yR
eal t
ime
/Lo
w-L
aten
cy ODS/ EDW
Real-Time Applications
Legacy Applications
HadoopSpark Hive
Big DataApplications
Users
MachineLearning
MLAnalytics
© 2017 Striim, Inc. All rights reserved.
Drag and Drop UI+ Command Line Interface
Distributed High Speed Message Infrastructure
Distributed In‐Memory Data Grid for Metadata / Control
Distributed In‐Memory Data Grid for Context Data
Distributed Persistent High Speed Message Infrastructure
Distributed Results Storage
…
Real‐Time Streaming Dashboardsto Surface In‐Memory Analytics
TQL / JDBC / ODBC / REST / WS APIsTQL / JDBC / ODBC / REST / WS APIs
SQL‐Based ProcessingAnd Analytics
CLUSTER
ContinuousData Collection
Databases (CDC)FilesMessagingCloudBig DataDevices
ContinuousData Delivery
DatabasesFiles
MessagingCloud
Big Data
Kafka
Scalability, Distribution, Clustering & Failover
Reliability, Recovery & E1P
Role‐Based Security & Encryption
Managem
ent & M
onitoring
Sources Targets
Streaming Analytics Platforms Require Lots of Pieces
© 2017 Striim, Inc. All rights reserved.
IoT Requires a Smart Data Architecture
© 2017 Striim, Inc. All rights reserved.
That Bridges the Edge, On‐Premise and Cloud
Cloud IoT Apps
On-Prem ServerEdge Server
Devices Edge
Cloud Server
Build & Deploy
Visualize&
Monitor
UI
Feedback
Scale / Speed Intelligence / Depth
© 2017 Striim, Inc. All rights reserved.
Integrate Machine Learning
Collect Data to Train Model and Use Model in a Streaming Fashion to Perform Real-Time Anomaly Detection or Predictions• Source Data From Databases, Files, Kafka, etc.• Process and Prepare the Data and Write to Disk to Train Machine Learning• Export Trained Model and Use in Streaming Analytics Real-Time Scoring• Present Results on Dashboard and Alert on Anomalies
ServerReal-Time
Dashboard
Training Files
Process & Prepare Data
MachineLearning
ExportedModel
WrapperFunction
Streaming Analytics
CQ Real‐TimeScoring
© 2017 Striim, Inc. All rights reserved.
Different Categories of Business Use Cases
Real-Time Data Integration
Data Lake Integration Offloading Reporting Operational DS/ DW
Cloud MigrationData Distribution
IoT Edge Processing
Detecting Patterns & Anomalies
Fraud DetectionCyber Security
Anti-Money Laundering Location-based Offers
Predictive MaintenanceIoT Analytics
Monitoring Real-Time Metrics and KPIs
Call Center Quality Network QualityPOS MonitoringSLA Monitoring
Replication MonitoringDevice Monitoring
BusinessUse Case
Technology Solution
© 2017 Striim, Inc. All rights reserved.
Example Use Cases
Hybrid‐Cloud IntegrationHybrid‐Cloud Integration
Real‐Time Streaming IntegrationReal‐Time Streaming Integration
Cyber SecurityCyber Security
Health Care Device MonitoringHealth Care Device Monitoring
Location TrackingLocation Tracking
© 2017 Striim, Inc. All rights reserved.
Hybrid‐Cloud Integration
• Approach– Use Initial Load
+ CDC to Move Data• True Real-Time Integration
– CDC pushes new data real-time– Process as necessary– Monitor and alert on issues
• Benefits– Streaming not Batch– Cloud Always Up to Date– Not Limited to Single Target
© 2017 Striim, Inc. All rights reserved.
Real‐Time Streaming Integration
• Approach– Collect, Prepare and Enrich
Streaming Data for Delivery to Multiple Targets
• Simple SQL-Based Processing– Filter, Transform, Aggregate &
Enrich Streaming Data– Many Targets in one flow
• Benefits– Easy to Collect Real-Time Data– SQL Enables Non-Developers– Simple Deployment / Monitoring
© 2017 Striim, Inc. All rights reserved.
Cyber Security
• Approach– Collect and Correlate Data From
Network, VPN, Firewall, Devices, Motion Sensors, etc.
• Identifies Multi-Phase Attacks– Port Scans + External Access– Operationalize AI– Unusual User & Machine Behavior
• Benefits– Instant Insights– Proactive vs Reactive– Not Limited to Single Solution
© 2017 Striim, Inc. All rights reserved.
Health Care Device Monitoring
• Approach– Collect, Analyze, Aggregate
Device Data. Join with Patient Data. Obfuscate for Cloud
• Real-Time Patient Monitoring– Multiple Medical Measurements– Use ML on Anonymous Data– Look for Anomalies / Issues
• Benefits– Doctors Have Real-Time Insights– React Immediately– Large Scale Data for Trends
© 2017 Striim, Inc. All rights reserved.
Location Tracking
• Approach– Collect and Analyze Location Data
Enriched With Contextual Information And Zones
• Real-Time Tracking– 1000s Real-Time Locations– Multiple Active Zones– Identify Entry / Exit / Wait / etc.
• Benefits– Spot Unusual Activity– Integrate With Existing Context– Real-Time Insights & Alerts
+
© 2017 Striim, Inc. All rights reserved.
Key Takeaways
• Customer demand now requires immediate insight and action for operational excellence
• Growing data volumes require pre‐processing in‐flight before storing data
IT’S THE RIGHT TIME FOR STREAMING FIRSTIT’S THE RIGHT TIME FOR STREAMING FIRST
• Streaming data architecture addresses both concerns • Add an enterprise‐grade streaming data platform to existing infrastructure with small use cases and expand gradually
• No need to rip and replace batch solutions
YOU NEED A FULL END‐TO‐END PLATFORMYOU NEED A FULL END‐TO‐END PLATFORM
© 2017 Striim, Inc. All rights reserved.
About Striim
© 2017 Striim, Inc. All rights reserved.
Striim Is a Complete End‐to‐End Platform
ContinuousData Collection
DBs (thru CDC), files, HDFS,system logs, message
queues, sensors
Stream Processing
Real‐Time Filtering, Transformation,
Aggregation, Enrichment
StreamingAnalytics
Correlation, CEP, Statistical, ML, Alerts and Visualization,Trigger External Systems
Continuous Results Delivery
Enterprise & CloudDBs, files, Big Data, Blob Storage, Kafka, etc.
Enterprise Grade Streaming First ArchitectureClustered, Distributed, Scalable, Reliable and Secure
Streaming Integration & Analytics PlatformSupporting Enterprise, Cloud and IoT
Flexible Architecture With Deployment On‐Premise / At The Edge / In The Cloud
Integration With Existing Enterprise Software
© 2017 Striim, Inc. All rights reserved.
Integration and Analytics Through Data Flows
© 2017 Striim, Inc. All rights reserved.
Visualization Through Streaming Dashboards
© 2017 Striim, Inc. All rights reserved.
Striim’s Key Differentiation
Striim is unique in the market by providing all 4 of the following in a single platform.
End-to-End Easy to Use
Enterprise Grade
Easy to Integrate
• Log-based Change Data Capture• Deep integration with Kafka• Integrates with other technologies
easily to collect data and distribute• Top 3 Cloud Platforms• Top 3 Big Data Platforms• Major Enterprise Databases• Multiple Open Source Solutions
• Single Platform for Collection, Processing, Analysis, Delivery and Visualization of Streaming Data
• Supports wide variety of data sources, targets, and data types
• Converged In-Memory Platform• Consistent end-to-end UI
• Low configuration installation• Fast to build and deploy
apps in days• Easy to iterate using
SQL-like language• Continuous ingestion and
processing• Multi-stream correlation• Time series/windowing
• Secure with built-in authentication, protectionand encryption
• High performance and highly scalable with distributed architecture
• Reliable with fault-tolerant architecture and “exactly once” processing
© 2017 Striim, Inc. All rights reserved.
Get Started Today
@StriimTeam facebook.com/Striim
Resources
www.striim.com/resources/
Product Page
www.striim.com/product/
Download the Striim Platform
www.striim.com/download-striim/
Share