bi2012 christensen modelling v1
DESCRIPTION
bvbTRANSCRIPT
-
*
In This Session
Tips and techniques for more effective SAP NetWeaver BW data modeling
This session will cover the top 20 data modeling best practices for an SAP NetWeaver BW landscape. Explore the benefits of techniques such as semantic partitioning and dimensional modeling, and receive proven strategies for performing them. Receive best practices for optimizing data models specifically for SAP NetWeaver BW Accelerator. Examine options for modeling InfoCubes for enhanced system performance, and learn how to improve memory utilization by caching. Get tips for leveraging system diagnostics to identify and troubleshoot SAP NetWeaver BW performance issues. Learn how and when to apply new data modeling techniques delivered with SAP Netweaver BW 7.3, such as the Semantic Partitioning Object (SPO) and cube-less SAP NetWeaver BW accelerator indices. Take home a list of best practices for optimizing InfoCube dimensions. -
*
What Well Cover
Dimensional modelingEnterprise Data Warehouse ModelingModeling for performanceModeling for in MemoryWrap-up -
*
1: Dimensional Modeling Business requirements
Start with 9 questions about the requirementDescribe the business processes and analysis/reporting requirement that define this subject area.
What granularity is required
What Dimensions are required
What Facts/measures
What additional information is required (Dimensional Attributes)
Define frequency of dimension updates
How should the measures be aggregated/summarized
How much History is required
How frequent should the data be updated
-
*
1: Dimensional Modeling - ERM
Start the dimensional modeling with an entity relationship model (ERM) based on the 9 decision pointsSource: SAP
-
*
1: Dimensional Modeling ERM (Cont.)
Add more details to the model based data analysis and relationshipsSource: SAP
-
*
1: Dimensional Modeling Star Schema
Turn the ERM into a dimensional modelThe fact table contains the measures
The dimension tables are de-normalised
Source: SAP
-
*
1: Dimensional Modeling The Info cube model
The SAP Netweaver BW Infocube model is an extended star schemaThe dimension table link to infoobjects that can have master data, texts and hierarchies maintained that can be used across the EDW
Source: SAP
-
*
What Well Cover
Dimensional modelingEnterprise Data Warehouse ModelingModeling for performanceModeling for in MemoryWrap-up -
*
Enterprise Data Warehouse modeling
Enterprise data warehouse modeling is all about reusability and modeling based on current and future requirementsReuse master data, texts and hierarchies
Reuse transaction data
Simple and standard dataflow
-
Dataflow in SAP NetWeaver BW
*
Source: SAP
*
-
*
2: Simplify the data flow
Denormalise the data during the extraction processingCombine header and item for documents
Standardize dataflows to use DSO objectsEnsures that data can be easily reloaded
Allows for improved flexibility for the future (HANA)
An info cube should only be updated from one DSOSimple to identify datasource
Allows for simple dataload recovery procedures
Enables immediate compression of data
-
*
3: Counter in DSO objects and cubes
Always add a counter infoobject in all DSO and Info cube infoproviders.This allows for
Counting no of documents etc.
Definition of process metrics
Avoids exception aggregation key figures at very granular level
-
4: Modeling Multi-Provider
Multi-Provider should always be used for reporting even when only 1 data target is needed for reporting, especially if there is a possibility of logically partitioning the data target.Multi-Providers should be used with multiple data targets when:Business requirements combine loosely related data with different dimensions. This data is also analyzed in separate InfoCubes.
the number of data sources have different levels of granularity
Keep the # of data targets in a multi-provider to 10 or less.
Use Variables on 0INFOPROV or another model specific infoobject to restrict the query to hit a specific underlying data target.
Multi-Provider
InfoCube 1
InfoCube 2
-
5: Info object modeling Business content
SAP Business Content objects will be used wherever possible to speed up development time for the data warehouse.Attributes should not be deleted from Business Content objects.
Time dependency should not be changed on Business Content objects
Enhancements should be made to Business Content without making a copy.
Changes to length or datatype should be made in a copy of the Business Content.
ALPHA conversion routines should not be altered
When to create a new custom Info ObjectWhen there is no existing active or inactive Info Object with the correct master data
-
6: Info object - Master Data
Reference characteristics must be used to limit the amount of redundant master dataExample: 0SHIP_TO is referencing 0CUSTOMER
Define master data and texts only if needed. Master data and text tables are not needed for info objects like document number, free text info objects like city and keys such as zip code.Texts should always be loaded if available in the source system
Attributes should always be loaded if available in the source system
Hierarchies should be loaded only when required for reportingUse upper case values for characteristic info objects that have master data, text or hierarchy tables.Time dependent master data, texts, and hierarchies should not be used unless specifically required by the business.
-
6: Info object - Master Data (Cont.)
Analyze the source data and set the Text properties (short, medium and long)Text should be:Marked language dependent if the source systems data has language associated with the text.
Extracted for English only, unless there is a business requirement to extract text in other languages.
Navigational attributes should be limited for performance reasons. If an attribute is marked navigational in Business Content and is not needed, this should be turned off before activating the InfoCube to improve load performance. -
7: Info object F4 help
To improve query selection filter performance, set the Query Execution Filter Val. Selectn for each infoobject as follows:= M (Values in Master Data Table), if master SID records 100. Make this the default.
Note: User can always override to Q at query execution if accuracy is required. -
8: Info object Key Figures
Key Figure data type must be valid according to data in the source system.Use data type of number when there is no relation to currency.Limit the use of floating point data type as it can cause problems for BWA and HANADo not use fixed currencies or unit of measures unless a fixed currency or Unit is determined in a routineUse summation aggregation in order to utilize the aggregate functionality of SAP BW.Use summation for exception aggregation to control the calculation of the key figure value in BEX.If summation is not used in the aggregation or exception aggregation, it is recommended to use the lowest grain of time characteristic of the InfoCube as the reference characteristic. -
8: Info object Key Figures (Cont.)
All non-cumulative key figure info objects should be approved by the ArchitectCalculated Key figures: Do not save a key figure as ratios. Instead save numerator and denominator and calculate the ratio in BEx. Most of the time business wants ratio of sum (not sum of ratio). -
*
9: Use Navigational Attributes for Security
Use specific Security InfoObjects in your SAP Netweaver BW systemE.g., Do not use 0COMP_CODE, instead create a reference InfoObject (e.g., SECCOMPCD) that you add as a navigational attribute of 0COMP_CODE
It has the same values as the base object but can be chosen to be assigned only in the InfoProviders that require security by the object
-
*
10: Enhancing and changing the data model
Reporting and analytical systems undergo constant changeUse SAP Netweaver BW infocubes and DSO (7.3) remodeling tool
-
*
11: Enhancing and changing the data model - reloads
Use a custom datasource when reloading additional new fields for an existing Logistics datasource to avoid having to load all the data columns and avoid the setup in ECCExample: A custom datasource to reload sales order can be defined as a view across tables VBAK and VBAP.
This approach can also be used for other master data reloads
DSO
Cube
Data-source
Reload Data-source
-
*
What Well Cover
Dimensional modelingEnterprise Data Warehouse ModelingModeling for performanceModeling for in MemoryWrap-up -
12: OLAP Cache
OLAP Cache is the first storage that is checked for a query result during query runtimeIf the result is available in cache it is read from there and results in the best query performance
The recommended settings are
Read Mode H
Cache Mode 5
Set the default cache settings in the infoprovider properties.
Additional Cache settings can be maintained in the query and cache monitors t-codes RSRT and RSRCACHE
-
12: OLAP Cache (Cont.)
Note: Only new queries will take these settings made for the Data Target. Settings made for the data target will not affect existing queries; they will continue to have the old settings. To check the current OLAP Cache settings check the system table RSRREPDIR and fields CACHEMODE and PERSISTMODE.If these fields are empty that means that query is not being cached.System transparent tables to hold the OLAP Cache data: RSR_CACHE_DBS_BL for Across systems BLOB table RSR_CACHE_DB_BL for Application server specific BLOB table -
13: SAP NetWeaver BW 7.x Statistics
Define standard measure that can be monitored on a daily, weekly, and monthly basis to evaluate data load performance trendsRecords processed per minute or Time to process 1 million records
Time spent on extraction
Time spent in transformations
Top 10 long running loads
Total time spent for Attribute and Hierarchy change runs
Use the standard queries and reports as a starting point*
*
-
13: See Details About Performance in the Monitor
The load monitor transaction code RSMO gives more details about the processing stepsInfoPackage details
Data Transfer Process (DTP) details
*
*
-
*
14: Use SE30 to Test Performance
Transaction code SE30 ABAP Runtime Analysis gives a detailed view of performanceRemember to set the accuracy to Low
Run transaction code RSA3
Note: SE30 can also be used for transformations by simulating the DTP run
-
14: Use SE30 to Test Performance (cont.)
Detailed Runtime will show you the bottlenecksSort descending based on Net Time and you will see your bottleneck on the top
-
*
15: Optimize Info cube Dimensions
Use as many dimensions as possibleSeparate common filter characteristics into own dimension
Use line-item dimensions for high cardinality characteristics such as document numbersDo not set the high cardinality flag as this changes the index to a b-tree index which cant be used in star transformation queries
Define related characteristics in the same dimensionCalculate expected number of dimensional entries
Try not to exceed 10% of expected fact table entries
-
*
15: Optimize Info cube Dimensions (Cont.)
Verify the dimension design after the first dataloads using program SAP_INFOCUBE_DESIGNSAllows for analysis of dimension design
-
*
15: Checklist for info cubes
1Are there large dimensions (>100k records)This should be avoided2Is the cube partitionedCubes should always be partitioned by the most used time characteristic (0CALMONTH or 0FISCPER) try to have less than 3M records in each partition3Is the cube large (> 30M records)Consider semantic partitioning into smaller cubes by e.g. year or geography4Do you have line item dimensions?These should only be used for true line items like document numbers.5Do you have high cardinality dimensionsShould not be used!6All cubes must be designed in a way that they can be compressed without changing the reporting results -
*
16: Implement Semantic Partitioning
What is it?An architectural design to enable parallel data loading and query execution
Partitioning criteria: Year, Region, or Actual/Plan
Source: SAP
-
*
16: Implement Semantic Partitioning (cont.)
Benefits of semantic partitioning:Reduction in SAP NetWeaver BWA footprint (when partitioned by year)
Parallel data loading (when not partitioned by year)
Parallel query execution
Best case when partitioning criterion is set as constant
Almost as good to create variables to filter on 0INFOPROV
Archival of a single InfoCube does not impact others
Easier DB maintenance
Performance benefits are so significant semantic partitioning should be deployed on virtually every data model!
-
*
16: Implement Semantic Partitioning (cont.)
Example: Semantic partitioning by yearSource: SAP
MultiProvider
DataSource
Ex: Current Year + 1 = 2010
Current Year = 2009
Current Year - 1 = 2008
Current Year - 2 = 2007
Current Year - 3 = 2006
Current Year - 1
Current Year
Current Year + 1
Current Year - 2
Current Year 3
Current Year - 1
Current Year
Current Year + 1
Current Year - 2
Current Year 3
ALL years
Write-Optimized (No SIDs)
History
(Summarized)
-
*
17: SAP Netweaver BW 7.3 SPO
Using the Semantic partitioning object in SAP Netweaver BW 7.3 offers an easy way to maintain semantic partitioningSource: SAP
-
*
What Well Cover
Dimensional modelingEnterprise Data Warehouse ModelingModeling for performanceModeling for in MemoryWrap-up -
*
18: Optimal model for SAP Netweaver BWA
SAP NetWeaver BWA does not support exception aggregation key figure is releases prior to BW 7.30 / BWA 7.20Ensure that all key figures can be aggregated
Calculate exceptions during data loading if possible
Ensure that the infocubes can be compressed and archived to minimize the footprint in SAP Netweaver BWAIt is good practice to rebuild the BWA indices on a quarterly basis for cubes where the data is changing frequently e.g. Sales orders
*
-
18: SAP NetWeaver BW Accelerator persistence
In BW 7.3 it is possible to load data into BWA without building an infocubeRequires definition of a hybrid provider based on a DSO
The hybrid provider is then defined with all the infoobjects of the DSO and is marked to only store the data in BWA
Note: BWA 7.20 is required.Latest BWA revision is recommended for increased stability
*
*
-
*
19: Optimal model for SAP HANA
SAP HANA is available as the DBMS for SAP Netweaver BWThis allows for additional data modeling options:Model without cubes (Even hybrid provider is not required)
An additional info cube model is available with SAP HANAIt eliminates the dimension tables and link the fact tables directly to the SID tables of the infoobjects
Existing infocubes can be converted using t-code RSMIGRHANADB
*
-
*
20: BW Work Spaces in SAP Netweaver 7.3
BW work spaces are new in BW 7.3Allows for combining local development with the corporate governed Data Warehouse
Local data files can be loaded
The work spaces are defined by IT but can be managed and changed by business users
Source: SAP
-
*
20: BW Work Spaces in SAP Netweaver 7.3 (Cont.)
Workspace Designer for key users in business departmentsBrowser-based tool running in SAP NetWeaver Business Client or SAP NetWeaver Portal
Easy upload of local flat files, Query results and BW DataSources into Local Provider
Merge data of Local Providers with data from SAP NetWeaver BW to create new data models (CompositeProvider)
Monitor Workspace with regards to size and number of objects created
Source: SAP
-
*
What Well Cover
Dimensional modelingEnterprise Data Warehouse ModelingModeling for performanceModeling for in MemoryWrap-up -
*
Additional Resources
Data Modeling in SAP NetWeaver BW - Frank K. Wolf, Stefan Yamadahttp://www.sap-press.com/products/Data-Modeling-in-SAP-NetWeaver-BW.html
Using Semantic Partitioninghttp://help.sap.com/saphelp_nw73/helpdata/en/4a/0905c133e00890e10000000a421937/frameset.htm
Multi-Dimensional Modeling with BIhttp://www.sdn.sap.com/irj/scn/index?rid=/library/uuid/6ce7b0a4-0b01-0010-52ac-a6e813c35a84
-
*
7 Key Points to Take Home
Data modeling should be based on business requirements and good analysis of the underlying dataModeling an enterprise data warehouse requires standards and best practices to be appliedAll reporting should be based on a multiproviderInfo Object modeling is very important to an enterprise data warehouse model as they should be reused across all modelsUse the remodeling tool to change DSO and cube modelsReview datamodels for performance prior to go liveImplement BWA or HANA to increase reporting performance and apply additional time saving modeling options -
*
Your Turn!
How to contact me:
Jesper Moselund Christensen
-
*
Disclaimer
SAP, R/3, mySAP, mySAP.com, SAP NetWeaver, Duet, PartnerEdge, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Wellesley Information Services is neither owned nor controlled by SAP.