data warehousing in the age of in-memory computing and

Data Warehousing in the Age of In-Memory Computing and Real-Time Analytics

Erich Schneider, Daniel Rutschmann June 2014

©  2014 SAP AG or an SAP affiliate company. All rights reserved. 2

This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.

Disclaimer



Key Trends Impacting Data Warehousing


Machine Data Run Connected: B2C and IoT


Un-Structured Data Run Connected: B2C


More Data in More Areas e.g. Healthcare - Genomics, DNA analysis



Big Data Introduces More Complexity to Traditional

System Architectures


Challenges and Inefficiencies

Analysts: Talent Shortage

Fragmented Point Solutions

Usability Shortcomings

Lack of Visualization Model Proliferation

High Latency

Operational Datastore Sensors Mobile Archives Social & Text

Order Processing

Operational Reporting

RT Risk & Fraud

Trend Analysis Sentiment Analytics

Predictive Analytics

Pattern Recognition

Spatial Processing

Analyze

Data Stores Integrate/Load Staging

Collect

Clean-Data Quality

Transact

Report Explore

Communicate Monitor Predict Planning

1

0

0

1

0

0

1

0

0

1

Data Warehouse

Geo-Spatial

Cache Cache Cache Cache Cache Cache

Business & IT: Segregated Organization Structure

Lack of Decision Support

Lack of Data Governance

Complex Slow Costly


ERP Business Suite

Traditional Data and Information Architecture (example) RDBMS-based

Non-SAP ERP

Planning Systems

Predictive OLAP Systems

Custom OLTP

Custom BI Systems

OLAP DW

Operational Data Stores

Transactional Systems Analytical Systems Data Access systems

Data Mart #N

Data Mart #2

Data Mart #1

GRC Systems

3rd party BI Systems

SAP BI Systems

Data Mart #3

Data Mart #4

ETL

Sentiment OLAP Systems

3rd party ETL

DB DB DB

DB DB

DB DB DB

DB DB

DB DB DB

DB DB

DB DB DB

OLAP EDW Events

DB DB DB DB DB DB

DB DB DB

DB DB

DB DB

DB DB

DB DB

DB DB

DB DB DB

DB DB DB

DB DB DB

DB DB

DB DB DB

DB DB

DB DB

DB DB DB

DB DB

DB DB DB

ETL

DB DB DB

DB DB DB

EIM

Machine Data

Social Data

DB DB

DB

Legend

Traditional RDBMS

Big Data File System



Simplification with SAP In-Memory Computing


SAP HANA (DRAM)

An innovative data management and

application approach for transactions, analytics and custom development using

an in-memory platform

One in-memory atomic copy of data for Transactions + Analysis

!  Eliminate unnecessary complexity and latency !  Accelerate through simplification

Re-think IT landscape simplicity with SAP HANA in-memory Eliminate redundant data copies and simplify applications

Transact

ETL

Analyze

ETL

Accelerate

Cache

!  Redundant data in and across applications !  Inherent data latency

Separated Transactions + Analysis + Acceleration processes

VS


Simpler landscape Integration of data types, data operations and applications processing in on platform

Any Apps SAP Business Suite & SAP BW JSON R

Open Connectivity MDX SQL

SAP HANA Platform SQL, SQLScript, JavaScript

Integration Services

Spatial

Business Function Library

Search Text Mining

Predictive Analysis Library

Database Services

Stored Procedure & Data Models

Planning Engine Rules Engine

Application & UI Services

SAP HANA

One System


Unification via SAP HANA Live in SAP Business Suite on HANA Operational Reporting and Foundation for new class of applications

13

Purchasing Manager

©  2014 SAP AG or an SAP affiliate company. All rights reserved.

Atomic data set for detailed drill-down information

Pre-defined models across entire suite

Operational data available instantaneously

SAP Business Suite Applications

SAP HANA PLATFORM

Database Layer Physical Tables

HANA Views

Operational Reporting

Zero latency!



So Why an EDW at all?


Some Simple Querying



Simplification with SAP HANA And what it means for Data Warehousing

ERP Data Warehouse

BI

Historical Reporting Planning

CRM …

Legacy

DB DB DB HANA HANA HANA

Consolidation

Integration / Harmonization

Focus on Data Warehousing

•  Integration / Harmonization of diverse sources and multiple technologies

•  Governance

•  Enterprise-wide master data like hierarchies, time-dependent data etc.

•  Information Lifecycle Management

•  Etc.

Operational Reporting ERP

Operational Reporting CRM

Operational Reporting…


Traditional EDW’s can be Streamlined by Focusing on Original Intent Eliminate the misappropriation

Provide a single source of truth ! Data harmonization and integration capabilities for heterogeneous data ! Audit proof “Sealed, Signed and Delivered” data persistence (instead of Excel spreadsheets)

– Regulatory – Legal – Enterprise-compliant

Enable + Design + Maintain + Govern consistent meta data, master data and KPI’s from ! Diverse Information sources ! Multiple Technologies ! SAP Data ! non-SAP Data


EDWs provide Data Management and Transformation Capabilities In addition to on-the-fly Analytics

Centralized processes to move/manage data flows and transformations to harmonize ! Enterprise-wide master data like hierarchies, time-dependent data etc. ! Calculated and restricted KPI’s

Data Snapshots in general, e.g.: ! Inventory by time period ! Data from batch interfaces ! Consistent Real-Time data (for example Headcount KPI’s, reports for Board

meetings etc.) ! Any versions of data based on simulations or manipulations which need to be

shared across the enterprise, as results of on-the-fly calculations


Enterprise Data Warehouse Capabilities will be Required in Support of Real-time Enterprises

Provide Information Lifecycle Management ! Data from Legacy systems as from Mergers & Acquisition ! Non-real-time required data from real-time systems as ERP, CRM etc. ! Corporate Memory data required to adjust historical information to new business rules

– Data not ready to be archived yet ! Streamed data like un-structured social media data, machine data storage

– Not required for real-time business processes – Long-term trending analysis

Optimize overall TCO ! Manage multi temperature storage media ! Minimize hot data back-up

– Accelerate time to restore mission-critical data from “hot” data back-up in real-time systems



The Logical Data Warehouse is the New EDW


Logical Data Warehousing (LDW) for Big Data and Business Data Traditional EDWs have outlived their purpose

Business in Real-Time: •  Volume •  Variety •  Velocity •  Value

Batch Real-Time

Stru

ctur

ed D

ata

Un-

Stru

ctur

ed

In-Memory Traditional Data Warehouse on

RDBMS

Real-Time Connected SAP HANA Platform


SAP HANA Platform

Journey to an SAP HANA based LDW Logical EDW for SAP and non-SAP platforms powered by SAP IMDF*

Microsoft

IBM Netezza

SAP HANA

SAP BI 4 SAP UI HTML5 Mobile

CR

M

SC

M

SR

M

PLM

ER

P

SAP

BW

Custom

Apps

SAP

EIM

SAP Business Suite HANA

Native

Apps

Fiori

Oracle

Teradata SAP ASE SAP IQ SAP ESP SAP SA

Smart Data Access (SDA) Virtual Data consumption of SAP and non-SAP data across different data bases

and storage media

SD

A

Hot

Warm

Cold

* In-Memory Data Fabric

IBM DB2


smarter

Information Architecture with the SAP HANA Platform


HANA Studio

SAP HANA and SAP ESP Streaming Data as IoT Enabler

Limited value in isolated events

Traditional Data Warehouse

History

Event window – e.g. 30 min

Continuous Sensor readings - single server 1 Mio/sec constant stream – 2 Mio/s (peak) – multi server: 5 Mio+/sec

Analyze in Real-Time Long-Term Trending

ESP

Analyze after 24h

vs.


Information Architecture for Internet of Things - IoT Streaming Data using the SAP HANA Platform

BI Tool

Many Sources

Real-Time Analysis without latency and redundancy

Data Exploration and Visualization

Streaming Real-time Replication

Data Federation Transformation

Loading

Real-Time Access and Action

SAP IQ

SAP ESP Engine

SLT / SRS

Data Services

SDA

Data P

rovisioning W

orkbench

Warm / Cold Data Management (NLS / Extended Storage / SDA)

SAP HANA SA

P Logical D

ata Warehouse

SAP HANA Studio SAP PowerDesigner

SAP Business Warehouse …


faster

Information Architecture with the SAP HANA Platform Velocity aspect of Big Data


SAP Logical Data Warehouse on HANA Load more data in less time

Faster Data Loads ! Faster Activation on database level

– Less data layers to be propagated – Elimination of data aggregation layers

! Less Redundancies – Real-Time replication for immediate consumption

in Mixed Scenarios ! Petabyte-Scale Data Management Elimination of Data Loads because of SDA Smart Data Access


SAP Logical Data Warehouse on HANA Consume more data in almost real-time

Faster BI ! Faster Reporting ! Faster Analytics ! Faster Data Exploration ! Faster Planning ! Faster Financial Consolidation

Across – Real-time data flow via streaming engine – In-Memory Hot Storage – Warm storage – Cold storage – Hadoop


Information Architecture with the SAP HANA Platform

simpler


SAP Logical Data Warehouse on HANA Lower TCO with more Agility

Simpler Data Model with BW and HANA ! No Aggregates ! No Indices ! No separate layers for performance

On-the-fly Transformation ! Master Data ! Data Model

On-the-fly BI ! Data exploration of SQL models and BW data models ! On-the-fly joins using CompositeProviders and SQL data

models for (near-)real-time reporting

Less IT Involvement


Data Warehousing in the Age of In-Memory Computing Bridging the separation between SQL and BW data modeling

SAP BW on HANA = BW + HANA Studio ! BW Enterprise-grade Governance ! SQL Data Mart Agility

Mixed Scenarios ! Agile SQL data modeling complementary to BW ! Virtual data model across SQL and BW data model

Enables BW and SQL skills ! Single environment ! Extract once – Use multiple times ! Shared master data for both types of data models


EDW Landscape Consolidation Logical Data Warehouse with BW and HANA Platform

Less Data redundancy

Less Data persistency

Fewer Data transfers

Less Data latency

Less Data reconciliation

Less Data correction

Less Data confusion

Fewer Data back-ups

Less IT involvement

SAP HANA ... Runs smarter… Runs faster… Runs simpler

HTML5 Mobile

SAP LDW

SAP HANA Platform

Fiori

DB DB DB DB DB

DB DB DB DB DB

DB DB DB DB DB

DB DB DB DB DB

DB DB DB DB DB DB DB DB

DB DB

DB DB DB DB DB


DB DB


DB DB

DB DB DB DB DB


DB DB

DB DB DB DB DB


SAP Data Warehousing Applications*

* Some restrictions depending on release level might apply, please refer to official SAP roadmap for details

JSON R Open

Connectivity MDX SQL

SAP HANA Platform SQL, SQLScript, JavaScript

Integration Services

Spatial

Business Function Library

Search Text Mining

Predictive Analysis Library

Database Services

Stored Procedure & Data Models

Planning Engine Rules Engine

Application & UI Services

SAP HANA

One System

SAP PowerDesigner SAP Business Intelligence

SAP Business Warehouse SAP Information Steward

SAP Event Stream Processor SAP DataServices

SAP HANA Studio

SAP LT Replication Server

SAP InfiniteInsight - KXEN

SAP 3rd-party SAS, Cognos, ….


THE BW and HANA EDW STRATEGY

All customers adopting SAP BW on HANA are on the right track

SAP takes care that all customers will be guided to the HANA future

Every new BW release will make progress on the HANA roadmap


Options TODAY – All Options Converge

•  SAP data dominates, external data augmented into BW

Migrate to BW on HANA and be happy

•  SAP data and external data equally important, e.g. FI/CO data and POS data

Migrate to BW on HANA and leverage HANA natively

•  Non-SAP data dominates, e.g. Health Care, Research, TelCo, Sports,…

Build a Data Warehouse natively with HANA


Thank You! Daniel Rutschmann Global HANA Center of Excellence [email protected] RutschmannD

Erich Schneider SAP HANA Solution Management ErichSap Erich Schneider


THANK&YOU&FOR&PARTICIPATING!!

Please!provide!feedback!on!this!session!by!comple6ng!a!short!survey!via!the!event!mobile!applica6on.!

!SESSION&CODE:&0411&

&For&ongoing&educaAon&on&this&area&of&focus,&

visit&www.ASUG.com&


Useful Links

SAP Community Network (SCN) www.SCN.com SAP HANA website www.SAPHANA.com SAP BW on HANA FAQ http://spr.ly/bwonhanafaq SAP Suite on HANA FAQ http://spr.ly/sohfaq

data warehousing in the age of in-memory computing and

Documents