real-time data warehousing with sap lt replication server ... · real-time data replication into...

Real-time Data Warehousing with SAP LT

Replication Server (SLT) for SAP NetWeaver BW

Marc Hartz (TIP CSA - Customer Solution Adoption)

March 2013

© 2013 SAP AG. All rights reserved. 2

Legal Disclaimer

The information in this document is confidential and proprietary to SAP and may not be disclosed without the permission of SAP. This presentation is not subject to your license agreement or any other service or subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation and SAP's strategy and possible future developments, products and or platforms directions and functionality are all subject to change and may be changed by SAP at any time for any reason without notice. The information on this document is not a commitment, promise or legal obligation to deliver any material, code or functionality. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. This document is for informational purposes and may not be incorporated into a contract. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.

All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.


Real-time data replication into SAP BW Comparison to existing data staging scenarios – technical characteristics

DTP

Index Handling

Rollup

Change Run

DTP

Activation

DataStore

Object

InfoCube

DataSource

DTP?

DTP

Activation

DataStore

Object

InfoCube

Extraction

DataSource

Extraction DB Replication

DataSources

DTPs (RDA?)

DTP?

DataStore

Object(s)

InfoCube

BW 7.x on RDBMS BW 7.30 on HANA

using extraction

BW 7.30 on HANA

using replication

Obsolete data

staging processes Obsolete data

staging processes

Revisit InfoCube

use cases Revisit InfoCube

use cases

Optimized data

staging processes

Inbuild extractor /

staging logic

Optimized data

staging processes

Instant replication

of single tables


Real-time data replication into SAP BW Comparison to existing data staging scenarios – typical use cases

DTP

Index Handling

Rollup

Change Run

DTP

Activation

DataStore

Object

InfoCube

DataSource

DTP?

DTP

Activation

DataStore

Object

InfoCube

Extraction

DataSource

Extraction DB Replication

DataSources

DTPs (RDA?)

DTP?

DataStore

Object(s)

InfoCube

BW 7.x on RDBMS BW 7.30 on HANA

using extraction

BW 7.30 on HANA

using replication

Report Availability

• >= Every x hours

Operational

Reporting

Report Availability

• >= Every minute

Source

• many tables

• complex join and

transformation logic

Source

• few tables

• simple / no join and

transformation logic

Corporate /

Strategic Reporting

and

and

and

and


Real-time Data Replication into SAP BW

Data Replication to SAP BW using SAP LT Replication Server

Consumption of HANA data in BW,

with real-time replication into HANA:

SLT replicates into HANA Table

BW Objects can consume HANA Views

directly

Enabling real-time data supply into

SAP BW (PSA) via WebService

DataSource:

SLT replicates into BW DataSource (PSA)

table directly, staging via RDA

LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

View

Trans./ Virt.

Provider

LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

DataSource

SLT offers new real-time capability for SAP NetWeaver BW on HANA

which can be deployed via two approaches:



Data Replication to SAP BW using SAP LT Replication Server

Use case of SLT for BW:

Replication of Non-SAP source data or custom tables of e.g. ECC

Adding delta capabilities where no standard extractor offered

Reduce admin effort for high frequent master data updates

Remark: Don’t try to replace complex extractors

– Inbuilt extractor logic has to be considered, consistency for delta uploads has to

be guaranteed *

– Logic has to be rebuild in BW (transformations / query) or via HANA modeler

(SQL Script, Calc. Views)

LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

View

Trans./ Virt.

Provider

LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

* Differences and Details are explained on the next slide.



Usage of BW extractors with SLT

VBAP T001

2L

IS_11_V

_S

CL

B

W E

xtra

cto

r S

LT

Re

plic

atio

n

VBUP VBEP

VBAK VBKD

targ

et

targ

et

SLT Replication of single extractor source table

• target needs to know:

− table associations

− table content logic

• delta: 1:1 replication of DB-operations

Complex to re-design by yourself

SAP Delta Queue Business View Extractors

• Delivered data are data mart ready

(joined information for extraction)

• inbuilt associations & content logic

• delta (SAP ∆ queue, full)

ABCD 0A

BC

D_

AT

TR

B

W E

xtra

cto

r

∆

∆

full

targ

et

SAP Generic Table/ View Extractors

• source table / view level

• delta (complete ? / full - lost deletes)

• often only full loads

Well fitting Use Case for SLT

∆

full

Use case of SLT for BW:

Replication into

a HANA table

consumed by BW

LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

View

Trans./ Virt.

Provider


SAP HANA system SAP non-SAP source system

Overview of SAP HANA provisioning

SAP LT Replication Server

Application Table

Trigger Based

Delta Recording

Replication

Engine

Application Table

RFC

Connection

HANA Studio

DB

Connection

Replication Configuration

Connection(s) between source system and SAP HANA system are

defined as “Configuration” on the SAP LT Replication Server

Data load and replication are

triggered via SAP HANA Studio


Overview

SAP HANA acts as platform to deploy as database for SAP NetWeaver BW and

for native HANA modeling as well (via two separated HANA schema)

SLT writes into a HANA table which can be consumed via HANA Views of

corresponding new BW InfoProvider

– Direct access from BW to HANA data

Real-time Data Replication into SAP BW Replication into HANA table consumed by BW

LT - Server

Structure

mapping &

Transform

ation

Write module

SAP NetWeaver ECC

AnyDB*

Application table

Logging table DB trigger

Read module

DB Connection

SAP NetWeaver BW

SAP HANA

HANA View BW Schema

*Any SAP supported DB


Real-time Data Replication into SAP BW Replication into HANA table consumed by BW

SAP NetWeaver BW Architecture in Detail

• TransientProvider based on HANA Model

For ad hoc scenarios without need to create

InfoObjects first

• VirtualProvider based on HANA Model

For a flexible integration of HANA DataMarts with

metadata persistency in BW

Support of Analytical Views and Attribute Views

• Composite Provider

The CompositeProvider is a new InfoProvider type

introduced with BW7.30. It offers the possibility join

in between InfoProviders.

SAP NetWeaver BW

SAP HANA

HANA View BW Schema


SAP NetWeaver BW

SAP HANA

HANA View BW Schema

Consuming SAP HANA models in

SAP Netweaver BW on HANA

Transient Provider Virtual Provider

InfoObjects optional InfoObjects

mandatory

Limited integration

with BW master data

with assigned

InfoObjects (No Nav.

Attributes)

Full Integration with

BW master data

BW Authorizations for

assigned InfoObjects

All BW Authorizations

Combining HANA data

with BW data in a

CompositeProvider

Combining HANA data

with BW data in a

MultiProvider /

CompositeProvider

Consumption of external managed

HANA data in BW via two different

objects:


Transient Provider Virtual Provider

Consuming SAP HANA models in

SAP Netweaver BW on HANA


SAP NetWeaver BW – SAP HANA Interoperability

Combination of real-time data

replication and staging for historic

purposes possible:

SAP HANA Data Load into

SAP NetWeaver BW

Complementary to DB Connect

Based on new source system type:

Operational Data Provider (ODP) with

the context ‘HANA’

Enable direct loading from HANA Views via

DTP into BW InfoProvider with storage in BW

managed schema

Data loading directly into InfoProvider,

persistency in PSA optional

Mass data loading enabled

SAP NetWeaver BW

SAP HANA

HANA View BW Schema

ODP source system,

context ‚Hana‘

Info

Package

DTP DataSource (PSA)

DSO (Optional)

Trans./ Virt.

Provider

Replication into a

BW DataSource

LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

DataSource


Real-time Data Replication into SAP BW Replication into a BW DataSource

Overview

Replicating Data of any SAP System (or non-SAP) into SAP BW

Real-Time data processing via SAP BW Realtime Data Acquisition (RDA)

Daemon into a DataStore Object (DSO) or master data tables (MD)

LT - Server

Structure

mapping &

Transfor-

mation

Write module

SAP NetWeaver ECC

AnyDB

Application table

Logging table DB trigger

Read module

RFC-BAPI

SAP NetWeaver BW

SAP HANA

Row & Column Store

SQL/Calc/Planning/Aggr. Engine

WebService

DataSource

RDA

Daemon

MD DSO


Real-time Data Replication into SAP BW Replication into a BW DataSource

SAP NetWeaver BW Architecture in Detail

• WebService DataSource used as a entry object

Generated DataSource RFC BAPI as interface

PSA as persistency for single requests

• Daemon continuously uploading data

Common RDA technique is uploading data into targets

Upload cycles ~ 1 per minute

• Several InfoProvider can be used for upload

MasterData objects as well as BW InfoProvider can be

used as data target

SAP NetWeaver BW

SAP HANA

Row & Column Store

SQL/Calc/Planning/Aggr. Engine

WebService

DataSource

RDA

Daemon

MD DSO


Maintain

table

IUUC_

BW_LO

G_SYS

Real-time Data Replication into SAP BW - Details (1/4) Replication into a BW DataSource

Processing Steps in SLT:

Define RFC connections

Define a Configuration: Establishes the

connections between source system(s), SLT and

BW system(s)

Map logical system and MT ID: in table

IUUC_BW_LOGSYS

Start data load / replication for (source) tables: Automatically creates logging tables and DB triggers

in the source system and related Web Service Data

Source in the BW system

Define

Confi-

guration

Start

Replication Web Service Data

Source in PSA

DSO

Query MD

SLT BW

Processing Steps in BW:

Define each affected source system as

system of type “Web Service”

Data is written in PSA via function module

Data distribution via RDA: RDA can acquire

data in real-time and load requests to

DataStore Objects or master data tables

Data can be queried in ~1 min. after data

change in source

Log. table + DB trigger

RDA

Daemon

Source


Real-time Data Replication into SAP BW – Details (2/4) Replication into a BW DataSource

SLT / BW integration Steps: SAP LT Replication Server steps are now automated and integrated with SAP NetWeaver BW

(as of DMIS 2011 SP4 / DMIS 2010 SP9)

Create a configuration in the SLT system with connections

to source system and SAP BW system

RFC connection to SAP BW system


Real-time Data Replication into SAP BW – Details (3/4) Replication into a BW DataSource

• Start data provisioning via t-code: LTRC


Real-time Data Replication into SAP BW – Details (4/4) Integration of SLT and BW

SLT will create BW DataSource automatically

according to SLT naming conventions: SLT_<TABLENAME> in target system

no manual work necessary

Field RECORDMODE is appended to each

data source and automatically filled according

to existing naming conventions

Action RECORDMODE SLT operations mode

Insert N 'I'

Delete D 'D'

Update ' ' 'U'


Technical Prerequisites in Detail

SAP BW system LT Replication Server Source system

System Requirements:

SAP Basis: SAP Netweaver 7.02 or higher

Filesystem: 100 GB

RAM: 16-32 GB

CPU: 2-4 cores

Number of background jobs: 10

RFC

Connection

RFC or DB

Connection


SAP sources: SAP Basis 4.6C and higher

non-SAP: all SAP supported DB versions

(with respective SAP Kernel installed on

LT Replication Server)

Installation (if SAP source system!):

Respective DMIS_2011 add-on version*

Minimum DMIS_2011 SP level: SP02

Installation:

Add-on DMIS 2011_1_700 (or higher)**


Apply SAP Note 1810627

Installation:

Respective DMIS_2011 add-on version


Apply SAP Note 1808251 (BW specifics!)

Basic Configuration:

Define RFC connection to source system

Define RFC connection to BW system

Maintain table IUUC_BW_LOG_SYS (after creation

of a configuration)


Optional: define separate table space

for logging tables

Define user with appropriate authorization

for remote connection – see SLT Security

Guide


Define each affected source system as

logical system – type “Web Service”

Define RFC user with appropriate

authorization

* For R/3 4.6C source systems or if other DMIS_2010 based applications are in use, install DMIS_2010 (part of SAP LT Replication Server 1.0) ** The use of SAP LT Replication Server 2.0 (DMIS_2011) is recommended, in case other DMIS_2010 based applications (such as TDMS 3.0) are in use, you may run LT Replication Server also with DMIS_2010_1_700 SP09 (or higher)


SAP BW 7.0: min. SP level – SP17

SAP BW 7.01 and 7.40: min. SP level – SP00

SAP BW 7.02 – 7.31: min. SP level – SP01

https://service.sap.com/sap/support/notes/1810627

http://service.sap.com/sap/support/notes/1808251


Installation / Deployment Options

As long as the technical prerequisites are met, there are 3 potential options to

deploy SAP Landscape Transformation Replication Server

For productive use - a separate SLT system is recommended to stay

flexible for software maintenance and scale out scenarios and

minimize impact on other applications

Source System (if SAP) BW System Dedicated System

Advantages Simplified landscape and

administration

Re-use of existing SAP ERP instance

Simplified landscape and

administration

Re-use of existing

NW instance

No software maintenance

dependencies

Flexibility

Disadvantages Performance impact

Potential software maintenance dependencies

Performance impact

Potential software

maintenance dependencies

Investment and maintenance

effort for separate server / NW instance

Differences of both approaches: LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

View

Trans./ Virt.

Provider

LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

DataSource



Considerations between both approaches

LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

View

Trans./ Virt.

Provider

LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

DataSource

DB calls / records: INSERT

UPDATE

DELETE

Update of DML via

0recordmode values

DOC CST WGHT DB

CALL

4711 1 400 INS

4711 1 500 UPD

4711 DEL

Source Table DML

DataSource View

Example of different record handling:

Delete record will

deleted record in

target table directly

Delete record will be

tracked and handled

flexibly depending on

EDW needs




No BW request handling, data is

accessed directly via HANA table

HANA Authorization has to be created,

Studio access for modeler necessary

Flexibility due to HANA modeling which

can aligned in BW objects

Replication (provisioning) integrated in

HANA Studio

Fully integrated in BW request handling,

delta will be loaded as requests in PSA

No additional HANA authorizations /

access needed

Data structure is controlled by BW

InfoObjects (usage of PSA)

Replication for BW (provisioning) will be

integrated in SLT Transactions

LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

View

Trans./ Virt.

Provider

LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

DataSource




Write Performance of SLT / HANA

secondary DB Connection

HANA Modeling & EDW Skills

Real-time master data scenarios via

virtual InfoObjects (BW 7.30 SP8)

HANA Studio can be used to transform

data (SQLScript, Calc.View)

Mixing of different calculations

engine (HANA calc. engine/BW OLAP)

Data quality has to be handled HANA

Write Performance of BW WebService

DataSource

EDW Skills

(Near) Real-time master data scenarios

via BW RDA requests

BW transformation can be used to

transform data (ABAP, routines)

Data quality handled by BW InfoObjects

LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

View

Trans./ Virt.

Provider

LT

Server

SAP NetWeaver

ECC

AnyDB

SAP NetWeaver BW

SAP HANA

DataSource


Real-time data replication into SAP NetWeaver BW Commercial Aspects and Software Shipment

Commercial Aspects

If SAP BW is not running on a SAP HANA Enterprise

Edition, the following license for SAP LT Replication

Server is required:

SAP LT Basis (Material Number 7010685)

SAP LT IT Consolidation (Material Number 7010688)

SAP LT Value chain harmonization (Material Number 7010687)

Software Version and Download

SAP LT Replication Server 2.0 (based on DMIS_2011) is

the recommended version for the replication into SAP

NW BW, find at http://service.sap.com/swdc.

http://service.sap.com/swdc

http://service.sap.com/swdc

Thank You!

Contact information:

Marc Hartz

TIP CSA – Customer Solution Adoption

[email protected]

mailto:[email protected]

real-time data warehousing with sap lt replication server ... · real-time data replication into...

Documents