48 data provisioning

Used with permission of SAP SE

Data Provisioning

Unit Overview

There are several ways of loading data into SAP HANA. This unit describes the

different methods.

Unit Objectives

After completing this unit, you will be able to:

• Understand the capabilities and positioning of the Flat file data load

functionality

• Load data from Flat Files into the SAP HANA Database

• The main features of the SAP Data Services solution for SAP HANA

• Describe the process of loading data from ECC to SAP HANA using the

ETL method

• Understand the architectural foundation of Landscape Transformation

Replicator and its technical pre-requisites

• Configure Landscape Transformation Replicator for connectivity to the

source SAP ERP system and the target SAP HANA Database

• Configure data provisioning in SAP HANA Studio and trigger an initial

load and/or replication

• Explain an additional data acquisition technique for working with data from

SAP Business Suite systems that has been added to the existing techniques

for HANA data acquisition

Unit Contents

Lesson: Uploading Data from Flat Files ......................................................... 2

Lesson: SAP Data Services .............................................................................. 1

Lesson: SAP Landscape Transformation Replication Server (SLT) ........... 20 Lesson: SAP Direct Extractor Connection (DXC)......................................... 31

© 2013 SAP AG or an SAP affiliate company. All rights reserved. 1

Lesson: Uploading Data from Flat Files

Lesson Overview

One of the options available for data provisioning is to simply upload data from

flatfiles. The lesson shows you the steps required to upload your own data from a

csv file.

Lesson Objectives

After completing this lesson, you will be able to:


functionality


Business Example

You want to upload Sales Organisation texts from a flat file to an SAP HANA

table, so you can use the data in modeling afterwards.

Figure 1: Data Replication – Definition

There are different technologies for loading data into SAP HANA (different

replication scenarios).

2 © 2013 SAP AG or an SAP affiliate company. All rights reserved.


Figure 2: Data Replication Methods

There are different technologies how to load data into SAP HANA (different

replication scenarios).

Features of uploading data from Flat Files

• If the required table for loading the data does not exist in SAP HANA

database, it’s necessary to create a table structure that is based on the flat files

• The application suggests the column names and data types for the new tables

and it’s possible to edit them

• The new table always has a 1:1 mapping between the file and table columns

• When loading new data in the table, it gets appended to the existing data

• The application does not allow to overwrite any column or change the data

type of existing data

• The supported file types are: .csv, .xls, and .xlsx

Figure 4: Process Flow: Uploading Data from Flat Files


Figure 3: Select Import Source

Figure 5: Select Target System



Figure 6: Select File for Upload

• In the Flat File Upload screen, browse for the file which should be uploaded

into SAP HANA database

• If a .xls or .xlsx file has been selected, choose the corresponding worksheet

• If a .csv File has been selected, select a delimiter

• If a header row exists in the flat file, select Header row exists and enter row

number

• If only a specific row range should be relevant for the import, remove check

for Import all data and enter the start / end line

Hint: A delimiter is used to determine columns and pick correct data

against them. In a .csv file, the accepted delimiters are: , ; :


Figure 7: Select Target Table

• For the Target Table, two options are available:

New: When selecting New, a new table with the name entered will be

generated within the schema chosen.

Existing: When selecting Existing, data will be appended to an existing table.

• Choose Next



Figure 8: Manage Table Definition and Data Mapping (1)

• In the Manage Table Definition and Data Mapping screen it’s possible to map

the source and the target columns

• The application proposes a mapping structure automatically based on the

naming

• Additionally it’s required to select a Key

Note: Only 1:1 column mapping is supported. Additionally, it’s possible to edit the table definition by changing the store type, data types,

renaming, adding or deleting columns.


Figure 9: Manage Table Definition and Data Mapping (2)

• It’s possible to preview the data based on the flat file chosen

• Select Finish to finalize the creation process

Figure 135: Check Target Table



Figure 10: Check Target Table


Lesson Summary You should now be able to:


functionality



Lesson: SAP Data Services


Lesson Overview

SAP Data Services 4 enables you to integrate disparate data sources to deliver

more timely and accurate data that end users in an organization can trust. This

lesson will look at the fundamentals of how to provision data from SAP Data

Services into SAP HANA.

Lesson Objectives




ETL method

Business Example

You are working in an organization where data is stored in various disparate

databases like Oracle, DB2 and other legacy systems. You are asked to recommend

the best application for consolidating and replicating data into SAP HANA from

SAP and Non SAP sources using the ETL method. Therefore, you need to

understand the benefits of using SAP Data Services over other methods.

Figure 11: Data Replication – SAP Data Services


Figure 12: Data Landscape

Most likely you face similar data challenges limiting your ability to know your

business.

Siloed – With data scattered across your organization in different ERP, database,

or homegrown systems; you may likely find different versions of the truth limiting

your ability to gain a complete view of the business.

Inaccurate – Have you ever received a promotion mail addressed to someone

in your mailbox? Consider the wasted marketing costs (in printed material

and postage) for a company that leverages customer promotions as a source of

revenue generation. Data is inherently inconsistent because things change and

your business requirements continue to evolve to meet your goals. As a result,

you need ongoing maintenance of your data quality by both the business and

IT stakeholders. Common issues like incorrect customer names, addresses, and

product names only add to the challenge for organizations to resolve before they

can leverage their corporate data as an enterprise asset.

Inconsistent – Definitions of common business entities like customers, products,

supplier, material names and codes vary from system to system creating

inconsistencies that data access alone can not address. You need a better way

to reconcile this.

Incomplete – Another common data challenge is incompleteness. A customer

record may be missing a postal code or country code and would be unusable unless

it is appended with the correct data.

Inaccessible – Connectivity will allow you to get access to the data in your

enterprise systems but often times, this is only the beginning. Making inaccurate

data accessible does not necessarily help you leverage that information for a



business goal. Sometimes the data is in a format that is unstructured like a free

form text coming from a CRM call log. The challenge lies in how to unlock

insights and the potential from all of your data sources.

Figure 13: Challenge: Disparate – Information Management Tools

Yet, the main challenge faced by organizations is the explosion of tools managing

the explosion of data sources. In this scenario, we see a typical IT environment

where multiple information management tools from multiple vendors increase

the complexity of managing information consistently and effectively across the

enterprise. Each tool has its own metadata repository, development environment,

administrative environment, runtime architecture requirements that you need to

support and maintain making the overhead of keeping up with all of these tools

extremely costly from a time and resource perspective. Many of these tools are

commonly used in an end-to-end solution supporting the requirements from data

extraction, transformation, cleansing, matching, and metadata management.

For example, data integration and data quality have a deep symbiosis that needs

coordination. Data integration ferrets out data quality issues, whether problems

to be fixed or opportunities to be leveraged for improvement. Likewise, data

quality reaches more data and systems that need improvement when it employs

data integrations numerous interfaces.


Figure 14: Data Services Is Your One-Stop Solution – Data Integration, Data

Quality, Data Profiling, Metadata Management, and Text Analytics

SAP Data Services is the first and only, all-in-one solution for data integration

(ETL), data quality management, information stewardship (data profiling and

metadata management), and text analytics.

With Data Services, you have a one-stop shop solution for 5 core critical

information management capabilities (ETL, DQ, Profiling, Metadata Management,

Text Analytics) that helps you:

Move and integrate enterprise data to and from almost any data source and target

with market-leading extract, transform, and load (ETL) capabilities.

Improve data from any data domain (e.g. customer, product, supplier, material)

with market-leading data quality management to cleanse, enrich, match, and

consolidate data.

Govern data with a new information stewardship solution providing a business

user interface for understanding and measuring the quality of data using integrated

data profiling, data quality scoring, and metadata management capabilities.

Unlock insights from structured and unstructured sources including text data

sources using text analytics.

Technically, this is underpinned by a development user interface, a metadata

repository, a data connectivity layer, a runtime environment, and a management

console, so IT groups can lower total cost of ownership (TCO) and accelerate time

to value by using one, integrated, all-encompassing tool for all of the above tasks.



Figure 15: Native Integration with the SAP – Stack Open and Agnostic

to non-SAP

Data Services is a recognized as leader in the Magic Quadrants for Data Integration

and Data Quality, proving that its industry-leading in both these respects for ANY

landscape. But for SAP landscapes, a special factor comes in:

• Within integration to increase efficiency, standardize processes, support

growth, increase effectiveness, and minimize risk, SAP Data Services is

the preferred data management solution for SAP environments. If your

organization depends on SAP to run its business, this software is your best

bet to manage you critical data in the most efficient, cost-effective way

available today.

• Whats more, SAP Data Services serves as the only engine available that

works with a wide range of supplementary data management offerings from

SAP. These include SAP High-Performance Analytic Appliance software,

SAP NetWeaver Business Warehouse, SAP NetWeaver Business Warehouse

Accelerator software, SAP BusinessObjects Rapid Marts packages, and SAP

Test Data Migration Server software.


Figure 16: Data Services Improves your Analytics – Data Lineage Helps

Users Make Confident Decisions

The data lineage function enabled by Data Services is directly integrated into

the BI workspace of the BI end-user. By right-clicking on a number, users can

choose to trace the origin of that figure back to where it comes from, how it was

computed, combined, enriched etc. This level of transparency boost trust in

your BI initiative and helps users make confident decisions since they have full

certainty of why a given figure is correct and what it really implies. There is also

Information steward that manages all your metadata and data governance.

Figure 143: Process Flow: Data Services



Figure 17: Import the Created Table Structure into Data Services

Figure 145: Create and Execute a Data Services Job to Populate HANA


Figure 18: Execute the Job to Populate the HANA Target Table and Monitor

the Load

Figure 147: View the Data Uploaded by Data Services






ETL method


Lesson: SAP Landscape Transformation Replication

Server (SLT)

Lesson Overview

This lesson gives an overview to SAP Landscape Transformation Replication

Server in connection with SAP HANA.

Lesson Objectives








Business Example

You are a business in a market with fast changes. Your HANA Information

Models need to have realtime information in order to keep up with market

changes. As such, you have decided to deploy SAP Landscape Transformation

Replication Server to achieve this.

Figure 19: SAP Landscape Transformation Replication Server

SAP Landscape Transformation Replication Server (aka “SLT”) is for all SAP

HANA customers who need real-time or scheduled data replication, sourcing

from SAP and NON-SAP sources, with the option to accomplish complex data

transformations on the fly.


Lesson: SAP Landscape Transformation Replication Server (SLT)

Figure 20: SAP Landscape Transformation Replication Server for SAP

HANA Appliance

Figure 21: DB Supportability Matrix (HANA 1.0 SPS06): SLT Replication

Server for SAP HANA


Figure 22: SAP Landscape Transformation Replication Concept:

Trigger-Based Approach – Architecture and Key Building Blocks


Trigger-Based Approach – Key Building Blocks in Detail




Trigger-Based Approach – Key Building Blocks in Detail

Figure 25: Landscape Transformation Replication Concept: Trigger-Based

Approach – Replication from non-SAP Sources

Landscape Transformation replication server transfers in a first step all metadata

table definitions from the non-SAP source system to the HANA system into the

tables DD02L and DD02T. From the HANA perspective everything looks like

you work with a SAP source. When a table replication is started, Landscape

Transformation replication server creates logging tables within the source system.

Only the read modules are created in the Landscape Transformation replication


server (for SAP sources the read modules are located in the source system only).

The connection from Landscape Transformation replication server to the non-SAP

source system is established by means of a database connection.

Figure 26: Installation

(*) SAP Landscape Transformation Replication Server can run on any SAP

system with SAP NetWeaver 7.02 ABAP stack or higher, for example on Solution

Manager 7.1 or the source system – it does not have to be a separate SAP system!

(**) A few new SLT features available since HANA SPS05 may require

DMIS_20 SP09 / DMIS_201 SP04 (***) SAP customers who run other

DMIS-based applications can apply DMIS_20 in the source and SLT system.

Hint: Always apply all related SAP Notes mentioned in Installation

Guide and SLT General Note 1605140!



Figure 27: Configuration & Monitoring Dashboard

With the Configuration & Monitoring Dashboard the Landscape Transformation

replication server will be able to provide different status information (for example

are triggers active, job monitor, status load and replication with error alert, system

connection) and statistical information (for example lowest/highest/average speed

rate of a replication) to the user.


Trigger-Based Approach – Set-up of Data Replication in SAP HANA System

(1)



Trigger-Based Approach – Set-up of Data Replication in SAP HANA System

(2)

Figure 30: Stop/Suspend Replication – Data Provisioning at a Glance



Figure 160: Multi System Support 1/2 – Positioning and Key Concepts (1)

Figure 31: Multi System Support 1/2 – Positioning and Key Concepts (2)

Figure 162: SLT and SAP HANA in the Cloud – Architecture and Integration

with SLT


When using SLT with SAP HANA in the Cloud, data is transferred from the SAP

source system to HANA using an RFC Connection via Wide Area Network.

SAP Landscape Transformation Replication Server – Development Roadmap

Key Goals and Objectives:

• Continuous improvements – especially in operations aspects

• Evolve core features – namely for operations and non-SAP aspects

• Serve new scenarios using trigger-based changed data capturing capabilities

• Contribute to strategic SAP initiatives like Suite on HANA and RTDP

Figure 32: SAP Landscape Transformation Replication Server –

Development Roadmap



Figure 33: Summary and Key Take Aways


Lesson: SAP Direct Extractor Connection (DXC)


Lesson Overview

This lesson gives a high level overview of SAP Direct Extractor Connection

(DXC), which is another data acquisition method. SAP HANA. DXC is a

batch-driven data acquisition technique which allows ETL from BW Extractors.

Lesson Objectives


• Explain an additional data acquisition technique for working with data from

SAP Business Suite systems that has been added to the existing techniques

for HANA data acquisition

Business Example

You work for ABC organization, where BW is heavily utilized. You want to load

BW data into SAP HANA by re-using exisiting extractors which have already

have the business logic applied.

Figure 165: Overview - SAP HANA Direct Extractor Connection

SAP HANA Direct Extractor Connect (DXC) is a means for providing

out-of-the-box foundational data models to SAP HANA, which are based on

SAP Business Suite entities.

DXC is also a data acquisition method. The rationale for DXC is essentially

simple: low TCO data acquisition for SAP HANA leveraging existing delivered

data models.

Customer projects may face significant complexity in modeling entities in SAP

Business Suite systems.

In many cases, data from different areas in SAP Business Suite systems requires

application logic to appropriately represent business documents.


SAP Business Content DataSource Extractors have been available for many years

as a basis for data modeling and data acquisition for SAP Business Warehouse;

now with DXC, these SAP Business Content DataSource Extractors are available

to deliver data directly to SAP HANA.

DXC is a batch-driven data acquisition technique; it should be considered as a

form of extraction, transformation and load although its transformation capabilities

are limited to user exit for extraction.

Overview of the DXC Rationale:

Leverage pre-existing foundational data models of SAP Business Suite entities for

use in SAP HANA data mart scenarios:

• Significantly reduces complexity of data modeling tasks in SAP HANA

• Speeds up timelines for SAP HANA implementation projects

Provide semantically rich data from SAP Business Suite to SAP HANA:

• Ensures that data appropriately represents the state of business documents

from ERP

• Application logic to give the data the appropriate contextual meaning is

already built into many extractors

Simplicity / Low TCO:

• Re-uses existing proprietary extraction, transformation, and load mechanism

built into SAP Business Suite systems over a simple http(s) connection to

SAP HANA

• No additional server or application needed in system landscape

Change data capture (delta handling):

• Efficient data acquisition – only bring new or changed data into SAP HANA

• DXC provides a mechanism to properly handle data from all delta processing

types



Figure 34: Overview SAP HANA DXC Concept: Illustration

An SAP Business Suite system is based on SAP NetWeaver. As of SAP NetWeaver

version 7.0, SAP Business Warehouse (BW) is part of SAP NetWeaver itself,

which means a BW system exists inside SAP Business Suite systems such as ERP

(ECC 6.0 or higher). This BW system is referred to as an “embedded BW system”.

Typically, this embedded BW system inside SAP Business Suite systems is

actually not utilized, since most customers who run BW have it installed on a

separate server, and they rely on that one. With the default DXC configuration, we

utilize the scheduling and monitoring features of this embedded BW system, but

do not utilize its other aspects such as storing data, data warehousing, or reporting

/ BI. DXC extraction processing essentially bypasses the normal dataflow, and

instead sends data to SAP HANA. The following illustration depicts the default

configuration of DXC.

An In-Memory DataStore Object (IMDSO) is generated in SAP HANA, which

directly corresponds to the structure of the DataSource you are working with. This

IMDSO consists of several tables and an activation mechanism. The active data

table of the IMDSO can be utilized as a basis for building data models in SAP

HANA (attribute views, analytical views, and calculation views).

Data is transferred from the source SAP Business Suite system using an HTTP

connection. Generally, the extraction and load process is virtually the same as

when extracting and loading SAP Business Warehouse – you rely on InfoPackage

scheduling, the data load monitor, process chains, etc – which are all well known

from operating SAP Business Warehouse.

DXC does not require BW on SAP HANA. Also with DXC, data is not loaded

into the embedded BW system. Instead, data is redirected into SAP HANA.


Figure 167: SAP HANA Direct Extractor Connection

Figure 168: Setup & Configuration – Relevant Notes

You must read the following SAP Notes before you start the installation. These

SAP Notes contain the most recent information on the installation, as well as

corrections to the installation documentation.

Make sure that you have the up-to-date version of each SAP Note, which you can

find on SAP Service Marketplace at http://service.sap.com/notes.


http://service.sap.com/notes


SAP Note Number:

• 1583403 Direct extractor connection to SAP HANA

Main note for setup steps required in the source SAP Business Suite system.

• 1670518 SAP HANA Direct Extractor Connection: Monitoring

Provides information on how to monitor SAP HANA Direct Extractor

Connection (DXC), in particular the Activation processing for In-Memory

DataStore Objects (IMDSOs).

• 1688750 DataSource: Reading a property in the source system

Apply this note to the source SAP Business Suite system only if you have the

“sidecar” scenario described in section Appendix – DXC System Landscape

Variants: The “Sidecar” Approach.

• 1701750 DataSource: Secondary Index on the PSA

If your DataSource is missing a key, apply this note to any BW systems

connected to the SAPBusiness Suite system you are using with DXC.

• 1677278 DataSource: Changing the Key Definition (A version)

Provides a report where you can define a semantic key for any DataSources

that are missing keys. DataSources without keys will cause an error when

you try to generate the In-Memory DataStore Object in SAP HANA. Before

applying this not to your SAP Business Suite system, first apply SAP note

1701750 to any BW systems connected to the SAP Business Suite system

you are using with DXC.

• 17 236 SAP HANA DXC: DataSource Restrictions

Lists specific DataSources not supported by DXC.

• 1714852 Troubleshooting SAP HANA DXC issues

Guidance for troubleshooting DXC issues.



• Explain an additional data acquisition technique for working

with data from SAP Business Suite systems that has been added

to the existing techniques for HANA data acquisition

Copyright Copyright © 2013 SAP AG or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of

SAP AG. The information contained herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software

vendors.

Trademarks Adobe, the Adobe logo, Acrobat, PostScript, and Reader are trademarks or registered trademarks of Adobe Systems Incorporated

in the United States and other countries.

Apple, App Store, FaceTime, iBooks, iPad, iPhone, iPhoto, iPod, iTunes, Multi-Touch, Objective-C, Retina, Safari, Siri, and

Xcode are trademarks or registered trademarks of Apple Inc.

Bluetooth is a registered trademark of Bluetooth SIG Inc.

Citrix, ICA, Program Neighborhood, MetaFrame now XenApp, WinFrame, VideoFrame, and MultiWin are trademarks or

registered trademarks of Citrix Systems Inc.

Computop is a registered trademark of Computop Wirtschaftsinformatik GmbH.

Edgar Online is a registered trademark of EDGAR Online Inc., an R.R. Donnelley & Sons Company. Facebook, the Facebook and

F logo, FB, Face, Poke, Wall, and 32665 are trademarks of Facebook.

Google App Engine, Google Apps, Google Checkout, Google Data API, Google Maps, Google Mobile Ads, Google Mobile

Updater, Google Mobile, Google Store, Google Sync, Google Updater, Google Voice, Google Mail, Gmail, YouTube, Dalvik, and

Android are trademarks or registered trademarks of Google Inc.

HP is a registered trademark of the Hewlett-Packard Development Company L.P.

HTML, XML, XHTML, and W3C are trademarks, registered trademarks, or claimed as generic terms by the Massachusetts

Institute of Technology (MIT), European Research Consortium for Informatics and Mathematics (ERCIM), or Keio University.

IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, z10, z/VM,

z/OS, OS/390, zEnterprise, PowerVM, Power Architecture, Power Systems, POWER7, POWER6+, POWER6, POWER,

PowerHA, pureScale, PowerPC, BladeCenter, System Storage, Storwize, XIV, GPFS, HACMP, RETAIN, DB2 Connect, RACF,

Redbooks,

OS/2, AIX, Intelligent Miner, WebSphere, Tivoli, Informix, and Smarter Planet are trademarks or registered trademarks of IBM

Corporation.

Microsoft, Windows, Excel, Outlook, PowerPoint, Silverlight, and Visual Studio are registered trademarks of Microsoft

Corporation.

INTERMEC is a registered trademark of Intermec Technologies Corporation. IOS is a registered trademark of Cisco Systems Inc.

The Klout name and logos are trademarks of Klout Inc.

Linux is the registered trademark of Linus Torvalds in the United States and other countries. Motorola is a registered trademark of

Motorola Trademark Holdings LLC.

Mozilla and Firefox and their logos are registered trademarks of the Mozilla Foundation. Novell and SUSE Linux Enterprise

Server are registered trademarks of Novell Inc.

48 data provisioning

Documents

new data

data types

data type of existing

sap data services solution

sap hana table

sap ag

data replication methods

data replication definition