all about data transfer process.docx

8/14/2019 All about Data Transfer Process.docx

1/14

All about Data Transfer Process (DTP) -

SAP BW 7

Data Transfer Process (DTP)DTP determines the process for transfer of data between two persistent objects within BI.

As of SAP NetWeaver 7.0, InfoPackage loads data from a Source System only up toPSA. It is DTP that determines the further loading of data thereafter.

Use

Loading data from PSA to InfoProvider(s). Transfer of data from one InfoProvider to another within BI. Data distribution to a target outside the BI system; e.g. Open HUBs, etc.

In the process of transferring data within BI, the Transformations define mapping and

logic of data updating to the data targets whereas, the Extraction mode and Update mode

are determined using a DTP.

NOTE: DTP is used to load data within BI system only; except when they are used in the

scenarios of Virtual InfoProviders where DTP can be used to determine a direct

data fetch from the source system at run time.

Key Benefits of using a DTP over conventional IP loading

1. DTP follows one to one mechanism between a source and a Target i.e. one DTP sourcesdata to only one data target whereas, IP loads data to all data targets at once. This is oneof the major advantages over the InfoPackage method as it helps in achieving a lot ofother benefits.

2. Isolation of Data loading from Source to BI system (PSA) and within BI system. Thishelps in scheduling data loads to InfoProviders at any time after loading data from thesource.

3. Better Error handling mechanism with the use of Temporary storage area, Semantic Keysand Error Stack.

ExtractionThere are two types of Extraction modes for a DTPFulland Delta.

Full:

Update mode full is same as that in an InfoPackage.


2/14

It selects all the data available in the source based on the Filter conditions mentioned in

the DTP.

When the source of data is any one from the below InfoProviders, only FULL Extraction

Mode is available.

InfoObjects

InfoSets

DataStore Objects for Direct Update

Delta is not possible when the source is anyone of the above.

Delta:

Unlike InfoPackage, delta transfer using a DTP doesnt require an explicit initialization.

When DTP is executed with Extraction mode Delta for the first time, all existing requesttill then are retrieved from the source and the delta is automatically initialized.

The below 3 options are available for a DTP with Extraction Mode: Delta.

Only Get Delta Once. Get All New Data Request By Request.

Retrieve Until No More New Data.

I Only get delta once:
http://scn.sap.com/servlet/JiveServlet/showImage/102-31781-3-137885/Delta.jpg


3/14


4/14


5/14

A on the right side of the Semantic Groups button indicates the Semantic keys existfor the DTP.

Update

Error Handling

Deactivated:

If an error occurs, the error is reported at the package level and not at the data

record level.

The incorrect records are not written to the error stack since the request is

terminated and has to be updated again in its entirety.

This results in faster processing.

No Update, No Reporting:

If errors occur, the system terminates the update of the entire data package. The

request is not released for reporting. The incorrect record is highlighted so that

the error can be assigned to the data record.
http://scn.sap.com/servlet/JiveServlet/showImage/102-31781-3-137890/update.jpghttp://scn.sap.com/servlet/JiveServlet/showImage/102-31781-3-137889/check.jpghttp://scn.sap.com/servlet/JiveServlet/showImage/102-31781-3-137890/update.jpghttp://scn.sap.com/servlet/JiveServlet/showImage/102-31781-3-137889/check.jpg


6/14

The incorrect records are not written to the error stack since the request is

terminated and has to be updated again in its entirety.

Valid Records Update, No Reporting (Request Red):

This option allows you to update valid data. This data is only released for

reporting after the administrator checks the incorrect records that are not updated

and manually releases the request (by a QM action, that is, setting the overallstatus on the Status tab page in the monitor).

The incorrect records are written to a separate error stack in which the recordsare edited and can be updated manually using an error DTP.

Valid Records Update, Reporting Possible (Request Green):

Valid records can be reported immediately. Automatic follow-up actions, such as

adjusting the aggregates, are also carried out.

The incorrect records are written to a separate error stack in which the records

are edited and can be updated manually using an error DTP.

Error DTP

Erroneous records in a DTP load are written to a stack called Error Stack.

Error Stack is a request-based table (PSA table) into which erroneous data records from a

data transfer process (DTP) are written. The error stack is based on the data source (PSA,DSO or Info Cube), that is, records from the source are written to the error stack.

In order to upload data to the Data Target, we need to correct the data records in the Error

Stack and manually run the Error DTP.

Execute


7/14

Processing Mode

Serial Extraction, Immediate Parallel Processing:

A request is processed in a background process when a DTP is started in a process chain

or manually.

Serial in dialog process (for debugging):

A request is processed in a dialog process when it is started in debug mode from DTPmaintenance.

This mode is ideal for simulating the DTP execution in Debugging mode. When this

mode is selected, we have the option to activate or deactivate the session Break Points at

various stages likeExtraction, Data Filtering, Error Handling, Transformation and Data

Target updating.

You cannot start requests for real-time data acquisition in debug mode.

Debugging Tip:
http://scn.sap.com/servlet/JiveServlet/showImage/137891/Execute.jpg


8/14

When you want to debug the DTP, you cannot set a session breakpoint in the editor

where you write the ABAP code (e.g. DTP Filter). You need to set a session break

point(s) in the Generated program as shown below:

No data transfer; delta status in source: fetched:

This processing is available only when DTP is operated in Delta Mode. It is similar toDelta Initialization without data transfer as in an InfoPackage.

In this mode, the DTP executes directly in Dialog. The request generated would mark thedata found from the source as fetched, but does not actually load any data to the target.

We can choose this mode even if the data has already been transferred previously usingthe DTP.
http://scn.sap.com/servlet/JiveServlet/showImage/137892/Execute.jpg


9/14


10/14


11/14

Why Data Transfer Process (DTP) over Info Package (IP)

in SAP BI 7.0Posted byVenkateswarlu NandimandalaminSAP NetWeaver Business Warehouseon Aug 23, 2011 3:23:54 PM

inShareDTP (Data Transfer Process):

Data transfer process (DTP) loads data within BI from one object to another object with respect to transformations andfilters. In short, DTP determines how data is transferred between two persistent objects. It is used to load the data from

PSA to data target (CUBE or DSO or Info Object or External systems) thus, it replaced the data mart interface and theInfo Package. In BI 7.0 Info Package pulls the data only till data source/PSA from source system, thereafter we need tocreate the transformation and DTP to pull the data further in BI system and to external systems.

IP (Info Package):The Info Package is an entry point for SAP BI to request data from a source system. Info Packages are tools fororganizing data requests that are extracted from the source system and are loaded into the BW system. In short, loadingdata into the BW is accomplished using Info Packages.

Where DTP comes in Data Flow:

DTP can be used to load the data in the following situations:
http://scn.sap.com/community/data-warehousing/netweaver-bw/blog/2011/08/23/why-data-transfer-process-dtp-over-info-package-ip-in-sap-bi-70http://scn.sap.com/community/data-warehousing/netweaver-bw/blog/2011/08/23/why-data-transfer-process-dtp-over-info-package-ip-in-sap-bi-70http://scn.sap.com/community/data-warehousing/netweaver-bw/blog/2011/08/23/why-data-transfer-process-dtp-over-info-package-ip-in-sap-bi-70http://scn.sap.com/people/venkateswarlu.nandimandalam2http://scn.sap.com/people/venkateswarlu.nandimandalam2http://scn.sap.com/people/venkateswarlu.nandimandalam2http://scn.sap.com/community/data-warehousing/netweaver-bw/bloghttp://scn.sap.com/community/data-warehousing/netweaver-bw/bloghttp://scn.sap.com/community/data-warehousing/netweaver-bw/bloghttps://lh6.googleusercontent.com/-QIqrU19Gb9Q/Tk59z950LgI/AAAAAAAAAHg/dDiOAr8C6RM/s576/DTP1.GIFhttp://scn.sap.com/community/data-warehousing/netweaver-bw/bloghttp://scn.sap.com/people/venkateswarlu.nandimandalam2http://scn.sap.com/community/data-warehousing/netweaver-bw/blog/2011/08/23/why-data-transfer-process-dtp-over-info-package-ip-in-sap-bi-70http://scn.sap.com/community/data-warehousing/netweaver-bw/blog/2011/08/23/why-data-transfer-process-dtp-over-info-package-ip-in-sap-bi-70


12/14

1) Loading data from DSO to DSO/CUBE, CUBE to CUBE2) Real time Data Acquisition3) For Data Direct Accessing.

4) Loading data from BI to External systems (for example Open Hub Destinations)

Advantages of DTP:1) Delta Management

2) Handling duplicate records3) Filters

4) Parallel processing5) Error Handling6) Debugging

1) Delta Management:i) DTP follows one to one mechanism, i.e. We have to create one DTP for each data target, whereas IP loads to all data

targets at once.

Advantage:Let's take an example where we are loading delta data using IP to 5 data targets. The data was successfully loaded into4 targets but failed in 1 target.In this case we have to delete the request from all 5 targets and need to re-load the IP to load the data.If we use DTP (one to one mechanism), we need to just delete the request from failed targets and re-load to the failed

target, which is very easier than loading to all 5 targets once again.I) One DTP can be used for Full and Delta loads, whereas in IP we have to create different Info Packages for Full andDelta loads,iii) No Full Repair/Repair full request concept in DTP as it has one to one delta mechanism.

Note: Data Mart and Delta for DTP is maintained in table RSMDATASTATE.In Table RSMDATASTATE

i) Field DMALL is used to keep track of Delta. It increases when Delta completesii) Field DMEXIST is used to prevent deletion of Source request.Deleting data request deletes the entry in table RSSTATMANREQMAP which keep track of data marts

2) Handling duplicate records:While we perform attributes or text loading, we will get the records with same key fields. Sometimes based on the

properties of the fields we have to load this records to the target (Info object).This can be easily achieved by DTP.We have to set the option Handle Duplicate Record Keys indicator in update tab of DTP to get this feature enabled.

3) Filters:
https://lh3.googleusercontent.com/-qKhdNF6-X8Q/Tk599hwW_XI/AAAAAAAAAH0/Wy3svfQycOk/Delta%2520DTP2.JPG


13/14

DTP filter's the data based on Semantic Key which is not possible on IP. We can use filters between DSO and CUBEalso.

4) Parallel processing:The request of a standard DTP should always be processed in as many parallel processes as possible. There are 3

processing modes for background processing of standard DTPs.i)Parallel extraction and processing (transformation and update)

The data packages are extracted and processed in parallel, i.e. The parallel process is derived from the mainprocess for each data package.

ii) Serial extraction, immediate parallel processingThe data packages are extracted sequentially and processed in parallel, i.e. The main process extracts the data

packages sequentially and derives a process that processes the data for each data package.

iii) Serial extraction and processing of the source packagesThe data packages are extracted and processed sequentially.

We can change this setting in the execution tab of DTP.The maximum number of background parallel processes we can set for a DTP is 99.

5) Error Handling:The error DTP concept is introduced to handle the error records in DTP. While loading the data using DTP, errorrecords will move to the error stack and the correct records will load to the target. We can edit this error records in errorstack and load to the target using Error DTP, which is not possible in IP.

6) Debugging:
https://lh3.googleusercontent.com/-Tn9RjjYcNo8/Tk50FMbMr_I/AAAAAAAAAGo/-1Mly6J7Zaw/Error%2520handling.JPG


14/14

Simulating the data update and debugging helps us to analyze an incorrect DTP request. In this way we can alsosimulate a transformation prior to the actual data transfer if we would like to check whether it provides the desiredresults.

We can define break points for debugging by choosing change break points, which was not available in Info Package.

Debugging can be done in 2 waysi) Simple Simulation

ii) Expert Mode

For more information about DTP, go through the below linkhttp://help.sap.com/saphelp_nw2004s/helpdata/en/84/497e4ec079584ca36b8edba0ea9495/frameset.htm
http://help.sap.com/saphelp_nw2004s/helpdata/en/84/497e4ec079584ca36b8edba0ea9495/frameset.htmhttp://help.sap.com/saphelp_nw2004s/helpdata/en/84/497e4ec079584ca36b8edba0ea9495/frameset.htmhttp://help.sap.com/saphelp_nw2004s/helpdata/en/84/497e4ec079584ca36b8edba0ea9495/frameset.htm

all about data transfer process.docx

Documents