in-memory computing webcast. market predictions 2017
TRANSCRIPT
1
+
Jason Stamper and Gary Orenstein
IN-MEMORY COMPUTING WEBCASTMARKET PREDICTIONS 2017
Market Predictions 2017: In-Memory Computing Jason Stamper, Analyst, 451 Research@jasonstamper
3
451 Research is an information technology research & advisory company
Founded in 2000
210+ employees, including over 100 analysts
1,000+ clients: Technology & Service providers, corporate advisory, finance, professional services, and IT decision makers
10,000+ senior IT professionals in our research community
Over 52 million data points each quarter
4,500+ reports published each year covering 2,000+ innovative technology & service providers
Headquartered in New York City with offices in London, Boston, San Francisco, and Washington D.C.
451 Research and its sister company Uptime Institute comprise the two divisions of The 451 Group
Research & Data
Advisory Services
Events
4
451 Research’s view of the ‘Total Data’ Model
5
Growth?
2015 2016 2017 2018 2019 2020
$69,612$79,612
$90,777
$103,126
$116,796
$132,049
14%2015-20 CAGR
Source: 451 Research, Forecast: Total Data 2016 - Data Platforms and Analytics Market Sizing and Forecasts.
284 Vendors included in this analysis
9 Market segments included
6
Defining the in-memory database“An in-memory database (IMDB; also main memory database system or MMDB or memory resident database) is a database management system that primarily relies on main memory for computer data storage. It is contrasted with database management systems that employ a disk storage mechanism.
Main memory databases are faster than disk-optimized databases because the disk access is slower than memory access, the internal optimization algorithms are simpler and execute fewer CPU instructions. Accessing data in memory eliminates seek time when querying the data, which provides faster and more predictable performance than disk.
Applications where response time is critical, such as those running telecommunications network equipment and mobile advertising networks, often use main-memory databases. IMDBs have gained a lot of traction, especially in the data analytics space, starting in the mid-2000s - mainly due to multi-core processors that can address large memory and due to less expensive RAM.”- Wikipedia
7
In-memory systems come in several guises
Pure in-memory database
Disk-based/persistent database with an in-memory ‘option’ or column store
In-memory data grid or fabric
8
What’s driving in-memory adoption?
• Increasing # of users & transactions• More data!• Growing # of writes• Insufficient capacity• Declining throughput• Performance inconsistencies• Cost of ETL processes
9
How are different data platform and analytic technologies shaping up?
$120,000
$100,000
$80,000
$60,000
$40,000
$20,000
$0
2015 2016 2017 2018 2019 2020
Event/Stream ProcessingData ManagementDistributed Data Grid/CacheAnalytic DatabaseSearchPerformance ManagementReporting and AnalyticsHadoopOperational Databases
$140,000
10
Some questions to ask Will my data fit in memory?! Is the platform optimized for analytics, transactions or both? Is the platform durable (ACID compliant)? What about
restores? What is the programming model – does it support SQL? Is there an open source or community edition? Does it support production requirements such as high
availability, cross data center replication, granular user permissions, and SSL?
Can I run it in the cloud, on-premises or both?
11
Some potential solutions, and their pros and cons
• In-memory ‘options’ added to existing relational databases
• Pure in-memory databases
• Data streaming offerings
• Analytics as a service or database as a service – on-prem/hybrid/cloud
• In memory data grid/cache
12
Some predictions for 2016/7• Multi-modal databases – that can handle both transactions (OLTP) and analytics
(OLAP) become the norm (with in-memory being a key enabler)
• In-memory databases continue to grow – both pure and ‘hybrid’
• E-commerce, ad-tech, gaming, financial services, high tech grow in-memory use particularly fast, but all businesses waking up to ‘latency sensitivity’
• From a slow start, the Internet of Things (IoT) gathers pace, thanks to both demand, and the ability to do something about it
• A new President of the US, and Andy Murray to become world #1
13
Twitter.com/jasonstamper
Architecting with In-Memory
Gary Orenstein
15
The nature of transactions has changed.
15
16
Traditional Transactions Modern TransactionsBATCH• Exactly-Once• Governed• Structured• ERP/CRM Applications
REAL TIME• Duplicates• Optional auditing• Unstructured• Social and Sensor
feeds
17
Modern TransactionsREAL TIME• Duplicates• Optional auditing• Unstructured• Social and Sensor
feeds
Traditional TransactionsBATCH• Exactly-Once• Governed• Structured• ERP/CRM Applications
Converged TransactionsREAL TIME• Exactly-Once• Governed• Any Structure• All Sources
A Real-Time Data Platform
Processing real-time and batch datato maximize traditional
and modern transactions
18
19
Architecting A Real-Time Data Platform
Database Workloads
Data WarehouseWorkloads
Real-Time Streaming
20
Architecting A Real-Time Data Platform
Orchestration / Containers
Cloud / On-Premises Platform
MessagingInputs Real-Time Applications
Business Intelligence Dashboards
Relational Key-Value Document Geospatial
Existing Data Stores
Database Workloads
Data WarehouseWorkloads
Real-Time Streaming
Hadoop Amazon S3MySQL
Transformation
21
A Deeper Look at Two Industry Sectors
Customer 360
Supply Chain Logistics
22
23
DellEMC Adopts MemSQL for Customer 360 Application
+
The Need for “CALM”Customer Asset Lifecycle Management
Forenterprise salesWho needaccurate and timely customer informationCALM is areal-time applicationProvidingup to the moment customer 360 dashboards
For enterprise salesWho need accurate and timely customer information
CALM is a real-time applicationProviding up to the moment customer 360o dashboards
Install Base
Pricing
Device Config
Contacts
Contracts
Analytics Contracts
Component Data
Offers
Scorecard
Source: DellEMC
25
Data Lake Architecture
D A T A P L A T F O R M
V M W A R E V C L O U D S U I T E
E X E C U T I O N
P R O C E S S GREENPLUM DBSPRING XD PIVOTAL HD
Gemfire
H A D O O P
ING
ES
TIO
ND
AT
A G
OV
ER
NA
NC
E
Cassandra PostgreSQL MemSQL
HDFS ON ISILONHADOOP ON SCALEIO
VCE VBLOCK/VxRACK | XTREMIO | DATA DOMAIN
A N A L Y T I C S T O O L B O X
Network WebSensor SupplierSocial Media MarketS T R U C T U R E DU N S T R U C T U R E D
CRM PLMERP
APPLICATIONS
Apache R
angerA
ttivioC
ollibraR
eal-T
ime
Mic
ro-B
atch
Bat
ch
Source: DellEMC
26
BUSINESS BENEFITS
Deliver real-time customer updates to EMC Sales Department Scale customer analytics to increase sales productivity Drive real-time personalization for operational efficiency
TECHNICAL ACHIEVEMENT Execute joins across multiple distributed data sets
26
27
The Pace of Supply Chain SuccessAmazon
30 minute post-order drone deliveries
FedEx
317 million packages shipped over Xmas with real-time tracking
Uber Freight
Matching shippers with trucks in real time
28
| MemExAn IoT showcase application that powers supply chain
monitoring and management through predictive analytics.
3030
TECHNICAL BENEFITS Processes 2 million data points, based on 2,000 sensors across
1,000 warehouses Two million reads and writes per second Combines MemSQL, Apache Kafka, and Streamliner with
machine learning, IoT sensors, and predictive analytics Enables enterprises to predict throughput of supply warehouses
| MemEx
32
Data Producers(simulating
sensor activity)
Raw Sensor 1 + Predictive Score 1
S1 P1
1
| MemEx
Real-TimeScoring
MemEx UI
33
THANK YOU!
Follow our speakers on Twitter@jasonstamper @garyorenstein
Try MemSQL today at memsql.com/download