dmm105 15586 antonellot 1

78
Public Anthony (Tony) Antonello / Global HANA Centre of Excellence DMM105 A Synopsis of SAP Data Movement Strategies for HANA

Upload: konduru25

Post on 08-Nov-2015

26 views

Category:

Documents


4 download

DESCRIPTION

DMM105

TRANSCRIPT

  • Public

    Anthony (Tony) Antonello / Global HANA Centre of Excellence

    DMM105 A Synopsis of SAP Data Movement Strategies for HANA

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 2 Public

    Disclaimer

    This presentation outlines our general product direction and should not be relied on in making a

    purchase decision. This presentation is not subject to your license agreement or any other agreement

    with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to

    develop or release any functionality mentioned in this presentation. This presentation and SAP's

    strategy and possible future developments are subject to change and may be changed by SAP at any

    time for any reason without notice. This document is provided without a warranty of any kind, either

    express or implied, including but not limited to, the implied warranties of merchantability, fitness for a

    particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this

    document, except if such damages were caused by SAP intentionally or grossly negligent.

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 3 Public

    Agenda

    1. Overview of SAP Data Movement & Transformation Strategies for HANA

    2. Real Time Data Movement Strategies

    1. SAP ESP (Event Stream Processor) Into HANA

    o DEMO - ESP (Streaming) into HANA demo

    2. SAP LT Replication Server (SLT) Into HANA

    o DEMO SLT (Replicaton Server) into HANA demo

    3. SAP SRS Into HANA

    o DEMO - SRS (Replication) into HANA demo

    3. Batch Data Movement Strategies

    1. SAP DXC (Direct eXtractor Connection) into HANA

    2. SAP HANA Studio Bulk Loader

    o DEMO HS Bulk Loader

    4. Data Synchronization into HANA

    1. SAP Mobilink into HANA

    o DEMO - Mobilink (Synchronization) into HANA

    5. Data Movement with Transformation

    1. SAP Data Services into HANA

    o DEMO Data Services and SRS (CDC) into HANA

  • Overview of SAP Data Movement &

    Transformation Strategies for HANA

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 5 Public

    An Enterprise Worth of Data

    SAP HANA In-memory

    Platform

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 6 Public

    SAP Data Movement and Transformation for HANA

    SAP Event Stream Processing (ESP) Analyze and act on fast moving data

    SAP Landscape Transformation (SLT) Designed to work with SAP Netweaver

    SAP Replication Server (SRS) Can create complex replication landscapes

    SAP Mobilink Designed for occasionally connected replication

    Data Services (DS) Data cleansing and transformation prior to loading

    Direct eXtractor Connection (DXC) Loading from SAP sources using DataSource extractors

    Hana Studio Bulk Loader Load external data in flat file format into an SAP HANA database

    Integration Feature Data Sources

    Streaming Data Data Feeds, SCADA

    Replication (Triggers) RDBMS

    Replication (T-Log) RDBMS

    Synchronization Ultralite, SQL

    Anywhere

    Transformation and

    Loading

    Structured and Un-

    structured data

    Application Data

    Extractors

    SAP Applications

    Bulk Loaders Flat Files

    SAP Data Movement Strategies

  • REAL TIME DATA MOVEMENT STRATEGIES

    SAP ESP (Event Stream Processor) Into HANA

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 8 Public

    Our World is Full of Simple EventsAnything Can Happen

    Plane Landing Lightning Strike

    Barcode Scanned

    Temperature Reading

    Transaction Created

    Stock Price Change

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 9 Public

    Insight from EventsAggregation in Time

    Time Window

    Raw Events

    We look at the stream of events

    watching for pre-defined patterns or

    trends over a period of time.

    We generate an alert (complex event) if the

    required pattern is detected.

    Temperature is dropping for 5 mins

    More than 100 parcels are delayed for 10 mins

    Time

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 10 Public

    ESP - SAP Event Stream Processor

    Event Stream

    Processor

    Unlimited number of input streams

    Incoming data is processed as it arrives, according

    to the business logic defined using Data Flow

    Authoring tools

    Send output to apps, dashboards

    Query on demand

    Capture data in SAP HANA or Sybase IQ for

    historical analysis

    ?

    INPUT

    STREAMS

    Market Data

    Transactions

    Sensors

    Alerts

    Applications

    Studio

    (Authoring)

    Reference

    Data

    SAP HANA,

    Sybase IQ

    Dashboard

    Message

    Bus

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 11 Public

    ESP Key Terms and Concepts Basics of data-flow programming in Event Stream Processor

    Data-Flow Programming

    ESP uses data-flow programming for processing event streams.

    Events

    A business event is a message that contains information about an actual business event that occurred. Many business systems produce streams of such events as things

    happen.

    ESP Projects: Streams, Windows, Adapters, and Continuous Queries

    Is like an application, consisting of a set of event streams, any other required data sources, and business logic applied to incoming event data to produce results.

    Streams and Windows

    Both streams and windows process events. The difference is that windows have state, meaning they can retain and store data, while streams are stateless and cannot.

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 12 Public

    ESP Key Terms and Concepts Basics of data-flow programming in Event Stream Processor

    Data-Flow Programming

    ESP uses data-flow programming for processing event streams. Define a set of event streams and the connections between them, and apply

    operations to the data as it flows from sources to outputs

    Breaks a potentially complex computation into a sequence of operations with data flowing from one operation to the next

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 13 Public

    Data Flow Authoring

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 14 Public

    How ESP Can Be Used With HANA

    ESP SAP HANA 1

    2

    1

    1

    2

    Event

    Streams

    High speed smart capture (via SAP HANA Output Adapter): ESP can apply logic

    to transform, filter, aggregate, enrich incoming event streams and capture data

    in HANA over a million events per second

    ESP can query HANA to provide context for processing incoming events

    ESP publishes alerts or triggers response process

    3

    3

    Alerts

    Dashboard

    Applications

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 15 Public

    Further ESP Information

    Watch SAP d-code Online

    www.sapcode.com/online

    Related ESP d-code Sessions

    DMM110 SAP HANA Streaming Introduction to Continuous Queries

    DMM119 Installing and Configuring an SAP HANA Streaming Server in Your Landscape

    DMM121 Integration Scenarios for Stream Processing with the SAP HANA Platform

    DMM165 Incorporating Real-Time Event Stream Processing into SAP HANA Applications

    DMM268 SAP Event Stream Processor and SAP Replication Server

  • Demo

    ESP Into HANA License Plate Scanning

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 17 Public

    Roadside License Plate Reader Streaming into HANA

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 18 Public

    Roadside License Plate Reader Streaming into HANA

    DMV

    Plate scans

    and speed

    SAP Sybase ESP

    Tickets

    SAP HANA In-memory

    Platform

  • REAL TIME DATA MOVEMENT

    STRATEGIES

    SAP LT Replication Server (SLT) Into HANA

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 20 Public

    Enable real-time replication within your entire landscape

    SAP LT Replication Server

    Read Engine

    Mapping & Transformation Engine

    Write Engine

    SAP HANA Appliance

    SAP HANA

    DB

    In-Memory S

    AP

    HA

    NA

    Stu

    dio

    BI Client

    non-ABAP (SAP supported DBs)

    SAP

    Business

    Suite

    SAP

    Business

    Suite

    SAP

    Business

    Suite SAP

    Business

    Suite (>=4.6c)

    SAP

    Business

    Suite

    SAP

    Business

    Suite

    SAP

    Business

    Warehouse

    SAP HANA

    DB

    In-Memory

    Source Systems SAP LT Replication Server Target Systems

    real time real time

    non-ABAP (SAP supported DBs)

    Data Services

    4.2 SP1

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 21 Public

    SAP HANA Application Tables

    Data replication via DB trigger based on

    change capturing concept

    Unlimited release coverage (from SAP R/3 4.6C onwards)

    Automatically non-Unicode to Unicode conversion

    during load/replication

    Handling of cluster and pool tables

    Table settings and transformation capabilities (e.g. data

    filtering, enrich table structure, scramble data, etc.)

    Fast data replication via DB connection,

    integration into SAP HANA Studio

    SAP HANA Studio

    ABAP Source System SAP LT Replication Server Target Systems

    SAP Business Suite

    Application table Logging table

    DB trigger

    Read Engine

    Read Engine

    Mapping & Transformation Engine

    Write Engine

    SAP LT Replication Server

    RFC

    Connection

    DB

    Connection

    Architectural Concept 1/2 Replication from ABAP source system

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 22 Public

    SAP LT Replication Server transfers all metadata table definitions from the non-ABAP source system to the HANA system. From the HANA Studio perspective, non-SAP

    source replication works the same as for SAP sources. When a table replication is started, SAP LT Replication Server creates logging tables in the source system. The

    read engine is created in the SAP LT Replication Server. The connection the non-SAP source system is established as a database connection.

    SAP HANA Application Tables

    Non-ABAP System

    Application table Logging table

    DB trigger

    Non-ABAP Source System SAP LT Replication Server Target Systems

    SAP LT Replication Server

    Read Engine

    Mapping & Transformation Engine

    Write Engine DB

    Connection

    DB

    Connection

    Architectural Concept 2/2 Replication from non-ABAP source system

    SAP HANA Studio

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 23 Public

    Application Tables

    Efficient implementation of data

    replication via DB trigger

    based on change capturing concept

    Highly scalable and reliable replication

    process, including comprehensive data

    transformation capabilities on the fly

    Fast data replication via DB connection.

    ABAP Source System SAP LT Replication Server non-ABAP Target System

    SAP Business Suite

    Application table Logging table

    DB trigger

    Read Engine

    Read Engine

    Mapping & Transformation Engine

    Write Engine

    SAP LT Replication Server

    RFC

    Connection

    DB

    Connection

    ABAP to non-ABAP Replication Architectural Concept

    Non-ABAP

    Start of

    replication has to

    be executed via

    LTRC

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 24 Public

    Setup, Execution & Monitoring of a Replication

    Setup Replication Execute Replication Monitor Replication

    New guided procedure to create a

    configuration between source and

    target system.

    SAP LT Replication Server

    SAP HANA

    DB

    In-Memory

    SAP HANA

    Studio

    Start the replication for the relevant

    tables via SAP HANA Studio.

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 25 Public

    SAP HANA system Source systems

    System A

    System B

    Schema 1 Schema 2

    Source systems are connected to

    separate HANA schema on the

    same HANA system

    SAP HANA system Source systems

    System A

    System B Schema 1

    Source systems are connected to

    same HANA system and also the

    same schema

    SAP HANA systems SAP Source system

    Schema 1

    Schema 2 System A

    SAP source system is connected

    to separate HANA systems or to

    the same system with different

    schema name.

    Multi System Support

    SAP HANA systems Source systems

    System A

    System B

    Schema 1

    Schema 2

    Source systems are connected to

    separate HANA systems. Schema

    name can be equal or different

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 26 Public

    DB

    Connection RFC/DB

    Connection

    Data and Structure Transformation Capabilities

    Target System SAP LT Replication Server Source system

    Reduce Number

    of Records

    by Filter

    Target System SAP LT Replication Server Source system

    Conversion

    of Data

    Target System SAP LT Replication Server Source system

    Adjustment of

    Target Table Structures (i.e. extension/reduction/change of table

    structure and/or adjustment of technical

    table Setting

    Replicating certain data only Only data of specific years, departments, clients, etc. should be used in HANA.

    To make certain fields anonymous i.e. HR reporting To fill initial fields To convert units or currency, and recalculate amounts and values using coded rules

    Remove fields that are required in the HANA system Additional fields are required (for example can be calculated/filled during load/replication) Merging the same table (i.e. MARA) from different systems (and create an additional field, for example a client field in order to avoid duplicate entries)

    Transformation

    of Data

    DB

    Connection RFC/DB

    Connection

    DB

    Connection RFC/DB

    Connection

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 27 Public

    Further LT Replication Information

    Watch SAP d-code Online

    www.sapcode.com/online

    Related ESP d-code Sessions

    DMM108 Real-Time Data Replication with SAP Landscape Transformation Replication Server

    DMM263 SAP Landscape Transformation Replication Server Real-Time Data Integration

    DMM844 Road Map Q&A: SAP LT Replication Server (SLT) realtime replication and CDC

    CR701 SAP Landscape Transformation Replication Server Performance Boost

    CR702 SAP Landscape Transformation Replication Server Replacing SAP BW Extractors

    DMM300 SAP Business Warehouse Data Provisioning from SAP and Non-SAP Sources

    DMM360 SAP BW 7.40 Data Provisioning from SAP and Non-SAP Sources

  • Demo

    LT Replication Into HANA ERP

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 29 Public

    LT from ERP into HANA

  • REAL TIME DATA MOVEMENT

    STRATEGIES

    SAP SRS Into HANA

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 31 Public

    SAP Replication Server Real-time data, delivered reliably, enterprise-wide

    SAP Replication Server is a real-time, low-impact, replication solution with enterprise-level features for both SAP and non-

    SAP databases and applications. It securely, reliably, and rapidly moves data across multiple data sources in real time to

    deliver:

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 32 Public

    Product Architecture

    Change Data Capture Replication Engine Routing & Distribution

    Primary

    Sources*:

    SAP HANA SAP ASE Oracle MS SQL Server IBM DB2/UDB IBM DB2/OS390

    Targets*:

    SAP HANA SAP ASE SAP IQ SAP ESP SAP Data Services Oracle Microsoft SQL Server IBM DB2/UDB IBM DB2/OS390 Message Bus

    Transformation

    Message Repository

    Rules Engine

    Administration &

    Monitoring

    In-b

    ou

    nd

    Co

    nn

    ectivity(L

    TL

    ) T

    arg

    et D

    B C

    on

    ne

    ctiv

    ity

    Catalog DB

    Replication

    Agent

    Secondary

    Replication

    Server

    Replication

    Server

    * Please check user documentation for end-to-end source and target database certification

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 33 Public

    SRS Components

    Replicate

    Queues

    Rep Agent

    Primary

    Rules Engine

    Message Repository

    Change Detection

    Catalog/Control DB

    RSSD

    Stable Devices

    ECDA Gateway

    Sybase ASE

    Oracle

    MS SQL

    IBM DB2/OS390

    IBM DB2/UDB

    RepConnector

    Sybase ASE

    HANA

    Sybase IQ

    Oracle

    MS SQL

    IBM DB2

    Any ODBC

    MQ, JDBC,

    MSMQ, etc.

    Sybase Control Center Monitoring

    Route

    Replication Server

    Replication Server

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 34 Public

    SRS Replication into HANA

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 35 Public

    RCL Used to Define Source and Target Mappings

    From source

    To target

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 36 Public

    Further Replication Server Information

    Watch SAP d-code Online

    www.sapcode.com/online

    Related ESP d-code Sessions

    DMM113 SAP Replication Server Usability Enhancements

    DMM209 Real-Time Heterogeneous Replication with SAP Replication Server

    DMM211 HADR for SAP Business Suite on SAP ASE with SAP Replication Server

    DMM212 Real-Time Transactional Replication for SAP Business Suite into SAP HANA

    DMM213 SAP ASE and SAP Repllication Server with SAP Business Suite

    DMM217 SAP Replication Server for Hadoop

    DMM222 SAP Replication Server Performance Tuning and Troubleshooting

    DMM262 SAP HANA Replication Using SAP Replication Server

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 37 Public

    Further Replication Server Information - continued

    Watch SAP d-code Online

    www.sapcode.com/online

    Related ESP d-code Sessions

    DMM268 SAP Event Stream Processor and SAP Replication Server

    DMM269 Real-Time Change Data Capture with SAP Data Services and SAP Replication Server

    DMM807 Road Map Q&A: SAP Replication Server Technical

  • Demo

    SRS Into HANA Book Store Analytics

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 39 Public

    ASE

    Primary DB

    ASE

    Replicate DB

    HANA

    Replicate DB

    Transformation

    Message Repository

    Rules Engine

    Administration & Monitoring

    LT

    L C

    onnection

    EC

    H

    Catalog DB

    ECH accesses HANA is via

    HANA odbc driver

    SAP HANA In-memory

    Platform

    Topology

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 40 Public

    Replication Test using rs_ticket

    40

  • BATCH DATA MOVEMENT STRATEGIES

    SAP DXC (Direct eXtractor Connection)

    into HANA

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 42 Public

    SAP HANA Direct Extractor Connection (DXC)

    DXC provides low TCO data acquisition for SAP HANA leveraging existing

    delivered foundations for data models oCustomer projects may face significant complexity in modeling entities in SAP Business Suite

    systems

    oData from some areas in SAP Business Suite systems needs application logic to appropriately

    represent business documents

    oSAP Business Content DataSource Extractors have been available for many years as a basis for data

    modeling and data acquisition for SAP BW; customers want a simple, low-TCO data acquisition

    mechanism to use SAP Business Content DataSource Extractors in SAP HANA data mart scenarios

    o In many use cases, batch-driven data acquisition is sufficient (e.g. every 10 min)

    oDXC became generally available with SAP HANA SP4

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 43 Public

    SAP Delivered Business Content DataSource Extractors

    SAP Business Suite

    Proprietary Extraction Technology

    Application based change data capture (aka delta capabilities)

    Extractors are application based and take data from the context of the application itself

    Extraction is event/process driven and (in some cases) is accomplished with publishing new or changed data in ERP based Delta Queues for receiving systems (e.g. BW or HANA)

    Extract structures can easily be enhanced using append structures

    Transformations can be implemented at the time of extraction using Business Add Ins (BADIs)

    Extract Structures based on entities in the Business Suite

    Asynchronous, mass data capable extraction

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 44 Public

    SAP HANA DXC Overview

    SAP ERP

    Extractor

    DataSource (flat structure)

    ERP data

    Embedded BW

    DataSource

    PSA

    generic data transfer

    InfoCubes

    Data Store Objects

    InfoObjects

    generic data transfer

    SAP HANA

    Data load into HANA activation queue

    Separate activation step

    Scheduled batch jobs

    Activation

    Queue

    Active version

    In memory DSO

    Status

    Activation

    Processing

    Data flow redirected from embedded

    BW: transferred over http connection

    Data models in HANA are built

    using active data table of In-

    Memory DataStore object

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 45 Public

    business documents master data

    transactional data master data attributes master data text master data hierarchy

    Transactional data Master data Texts Hierarchies

    ERP & R/3 900 700 2300 100 4000

    CRM 160 280 700 40 1180

    SRM 30 30 60 10 130

    Others 360 210 330 30 930

    GRC 20 30 120 10 180

    1470 1250 3510 190 6420

    Thousands of SAP Business Content DataSources Exist

    Totals

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 46 Public

    SAP Business Suite DataSource Extractors Example from Sales Order Item Content

    HANA Data Models - virtual

    SAP HANA

    SAP ERP

    Application Tables

    Sales Order

    Communication Structure

    Setup Table

    Update Methods

    Update Mode

    DataSource for transactional data

    2LIS_11_VAITM

    In-Memory Data Store Object Tables:

    Active table: /BIC/A 2LIS_11_VAITM00

    Activation Queue table: /BIC/A 2LIS_11_VAITM40

    Transfer Structure

    Extract Strukture MC11VA0ITM

    Application logic

    Activation Processing

    Data flow redirected from embedded BW

  • BATCH DATA MOVEMENT

    STRATEGIES

    SAP HANA STUDIO BULK LOADER

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 48 Public

    1. The IMPORT FROM statement imports

    external data from a file into an existing

    table

    2. Can append contol features on the Import

    statement or use a control file to specify

    file characteristics

    3. Can be run from any HANA editor (HANA

    Studio SQL Editor, hdbsql)

    4. Data files must reside on the HANA Server

    Control File

    HANA Table

    Flat File HANA SQL

    Import cmd

    Import into HANA using SQL Import Command

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 49 Public

    1. Uses a rich data mapping toolset within

    HANA Studio

    2. Data files can reside on the client host or

    the HANA server

    3. The target table can already exist, or be

    created by the import

    4. Broad array of schema objects beyond flat

    files

    Import into HANA using SQL Import Command

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 50 Public

    Further HANA Studio Data Provisioning

    Watch SAP d-code Online

    www.sapcode.com/online

    Related ESP d-code Sessions

    DMM271 Developing Custom Adapters Using the SAP HANA Data Provisioning Adapter SDK

  • Demo

    Import using HANA Studio and hdbsql

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 52 Public

    Importing into HANA with hdbsql and HANA Studio

  • DATA SYNCHRONIZATION

    SAP MOBILINK INTO HANA

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 54 Public

    What is Mobilink?

    1. A bi-directional synchronization

    technology for large scale mobile (IoT)

    database deployment o Remote database (mobile, embedded, or workgroup

    database server)

    o Consolidated database (enterprise, workgroup, or

    desktop database)

    2. A server that processes synchronization

    requests from SAP mobile databases

    (SQLAnywhere and Ultralite)

    3. Maintains the integrity of transactions

    across the entire synchronization process.

    HANA, DB2

    ASE, IQ, MSSQL

    Oracle, MySQL,

    SQLAnywhere

    Consolidated

    Database

    Communication Infrastructure

    (Internet / Wireless)

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 55 Public

    Mobilink Synchronization

    1. MobiLink uses system tables in the Consolidated

    database to store

    o Synchronization scripts

    o Synchronization progress & status information

    o Synchronization User information

    2. A Database Object in the Remote Database data to be

    synchronized is Published

    3. During synchronization MobiLink progresses through

    a series of events synchronization scripts that direct

    the data flow during the process

    4. Written in the native SQL of the Consolidated

    Database

    5. MobiLink uses ODBC to connect to the Consolidated

    DB

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 56 Public

    MobiLink SQLAnywhere Synchronization

    HANA

    MobiLink

    Ultralite or

    SQLAnywhere

    Remote

    ODBC

    dbmlsync

    1. Upload

    2. Download

    3. Ack Download

    1. Upload

    SQLAny or UltraLite keep track of all changes since previous synchronization

    All changes are sent in a single upload stream

    MobiLink applies changes in a single transaction

    2. Download

    Scripts in the consolidated database determine changes that need to be downloaded

    All changes are sent in a single download stream

    3. Acknowledge download

    Optional

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 57 Public

    DBMLSYNC

    Synchronization is initiated by a client application

    via the dbmlsync utility from SQL Anywhere

    Remote databases

    TCP/IP, SSL HTTP/HTTPS ActiveSync

    Reads the Database engine transaction log

    Builds an upstream with the a summary of the

    changes to the specific publication since the last

    synchronization

    Applies the downstream from the MobiLink server

    to the Remote Database

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 58 Public

    SQL Anywhere Architecture Scenario Remote Sync

    Remote Sync

    Synchronize HANA database with SQL Anywhere/Ultralite databases

    running on smart systems, kiosks,

    intelligent appliances, and mobile

    devices

    MobiLink Server

    ODBC Driver

    ODBC/Network interface

    Database Server

    HANA

    MobiLink Server

    Machine

    HANA

    Server Machine

    SQL Anywhere Server or UltraLite Server

    MobiLink Client

    Remote

    Machines

    Database

    File

    Database API

    Application

    SQL Anywhere/UltraLite Interface API

    HANA

    Network

    DMZ

    Machine

    Firewall

    Firewall

    Web Server Relay Server

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 59 Public

    Further SQL Anywhere and Mobilink Information

    Watch SAP d-code Online

    www.sapcode.com/online

    Related ESP d-code Sessions

    DMM106 Introduction to SAP SQL Anywhere

    DMM114 Synchronizing Data in SAP HANA Using SAP SQL Anywhere

    DMM116 From Raspberry Pi to SAP HANA SAP SQL Anywhere and the Internet of Things

  • Demo

    Telematics Demo

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 61 Public

    Architecture

    On board

    SAP HANA Cloud

    Platform

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 62 Public

    Vehicle Fleet Management with SAP HANA and SAP SQL Anywhere

  • DATA MOVEMENT WITH

    TRANSFORMATION

    SAP DATA SERVICES INTO HANA

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 64 Public

    SAP Data Services and HANA RTDP

    SAP Data Services provides batch Data Integration. It offers HANA optimized capabilities for Transforming,

    Cleansing* and Integrating (bulk or delta) structured and unstructured* data from many different Sources

    (SAP and non-SAP) to the Target (SAP HANA)

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 65 Public

    SAP Data Services Source Connectivity Broadens Enterprise Data Access

    HANA

    Tables

    views

    Oracle

    DB2

    Sybase ASE

    Sybase IQ

    SQL Server

    Informix

    MySQL

    Teradata

    ODBC

    Netezza

    SAP Bus Suite Extractors

    ABAP

    BAPI

    Idoc

    SAP NW BW

    JD Edwards

    Oracle Apps

    PeopleSoft

    Siebel

    Salesforce.com

    SuccessFactors

    Text delimited

    Text fixed width

    EBCDIC

    XML

    Cobol

    Excel

    HTTP

    JMS

    Web Services

    HADOOP

    ADABAS

    ISAM

    VSAM

    Enscribe

    IMS/DB

    RMS

    Both direct and changed data

    Extracts context from any text or binary file types (docs, pdfs, ppts)

    Understands context in 31 languages

    Databases Applications Files/Transport Mainframe (with partner) Unstructured

    Content

    SAP Data Services has a long history of providing broad connectivity to databases,

    applications, legacy systems, file formats

    PAM info posted at:

    https://service.sap.com/sap/support/pam?hash=s%3Ddata%2520services%26o%3Dmost_viewed%257Cdesc%26st%3Dl%26rpp%3D20%26page%3D1%

    26pvnr%3D01200314690900003943%26pt%3Dg%257Cd

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 66 Public

    For typical database

    targets, Data

    Services transformations are

    done in the DS engine and then

    loaded into the target

    database

    Source Target

    Extract Transform Load

    E-T-L moves to E-L-T How Data Services Leverages HANA for Transformations

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 67 Public

    For HANA, DS

    stages data and then

    pushes the execution of any

    transformation to

    HANA and creates

    Calculation Views for

    execution and

    consumption HANA

    HANA as Target Source

    Extract Load Transform

    E-T-L moves to E-L-T How Data Services Leverages HANA for Transformations

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 68 Public

    Data Services for HANA and RTDP Enhancements with Data Services 4.1+

    Additional Sources Support with Data Services

    HANA stored procedures, calculation, attribute, and analytical views as a Source

    Optimized Hadoop extraction and loading

    SuccessFactors source and target

    Binary files (PDF, Doc) and 31 additional languages supported for Text data processing

    Loading Performance

    50% improvement in loading flat file and database to HANA (via DS-HANA pipe)

    Increased E-L-T and push down optimizations

    Sybase IQ support for bulk updates and binary format data transfer

    10x improvement in BW extract and load to HANA, Sybase

    Parallelized reading from Business Content Extractors

    Modeling, Metadata, and Repository

    New Eclipse DS Workbench for very fast, simplified data movement into HANA

    Full support for HANA views for metadata impact and lineage analysis

    HANA as a repository for Data Services and Information Steward

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 69 Public

    Integrated Data Movement Strategy

    SAP SLT / Sybase Rep Server + Data Services

    Common runtime environment provides

    enriched CDC data provisioning across SAP

    ERP and non-SAP data replication

    Enables changed data sharing between SAP Sybase Rep Server, SAP SLT, and Data Services

    Leverage native CDC reading from Rep Server and SAP SLT

    With the

    Complex data transformation and data quality management of Data Services

    Complex Operational Data

    Note: SAP SLT & DS integration is targeted for 4.2 SP1 release, ~Q4 2013

    New

    with 4.2

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 70 Public

    Data Services Designer

    drag-n-drop tables,

    files, transforms

    from Local Object

    Library onto Work

    Canvas

    Work Canvas

    click-n-drop

    objects from

    Palette onto

    Work Canvas

    click-n-drag from

    source point to target

    point to create link

    Double-click to drill

    into object/transform.

    (All objects/transforms

    are movable)

    Normal/Best practice for

    flows to be Left-Right

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 71 Public

    Further Data Services Information

    Watch SAP d-code Online

    www.sapcode.com/online

    Related ESP d-code Sessions

    DMM269 Real-Time Change Data Capture with SAP Data Services and SAP Replication Server

    DMM805 Road Map Q&A: SAP Data Services

    DMM104 Whats New in SAP Data Services

  • Demo

    Bookstore BI into HANA Demo

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 73 Public

    Bookstore BI into HANA demo

  • Question ?

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 75 Public

    SAP d-code Virtual Hands-on Workshops and SAP d-code Online Continue your SAP d-code education after the event!

    SAP d-code Online

    Access replays of keynotes, Demo Jam, SAP d-code live interviews, select lecture sessions, and more!

    Hands-on replays

    http://sapdcode.com/online

    SAP d-code Virtual Hands-on Workshops

    Access hands-on workshops post-event

    Starting January 2015

    Complementary with your SAP d-code registration

    http://sapdcodehandson.sap.com

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 76 Public

    Further Information

    SAP Education and Certification Opportunities

    www.sap.com/education

    Watch SAP d-code Online

    www.sapcode.com/online

    SAP Public Web

    scn.sap.com

    www.sap.com

  • 77 2014 SAP SE or an SAP affiliate company. All rights reserved.

    Feedback Please complete your session evaluation for

    DMM105.

    Thanks for attending this SAP TechEd && d-code session.

    2014 SAP SE or an SAP affiliate company. All rights reserved. 77 Public

  • 2014 SAP SE or an SAP affiliate company. All rights reserved. 78 Public

    2014 SAP SE or an SAP affiliate company. All rights reserved.

    No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an

    SAP affiliate company.

    SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE

    (or an SAP affiliate company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark

    information and notices.

    Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.

    National product specifications may vary.

    These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its

    affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or

    SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing

    herein should be construed as constituting an additional warranty.

    In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or

    release any functionality mentioned therein. This document, or any related presentation, and SAP SEs or its affiliated companies strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for

    any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-

    looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place

    undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.