obiee10g metadata modeling basics

OBIEE10g

OBIEE10g Metadata Modeling

Basics Training

Michal Zima Killorglin, 8.10.2013

OBIEE10g Metadata Modeling Basics 2

Get Introduced (trainer, participants) Training :

Audience Prerequisites Objectives Agenda

Training Introduction/Logistic

Lesson Agenda


Trainer introduction : Over 10 years of experience with DWH/BI Oracle technologies and building solution using those technologies 6 years of experience with OBIEE (past Siebel Analytics) technology

Worked for global companies (Siebel, Oracle) in consulting practice

A few words about you (participants) : Name Role Prior experience with BI (particularly OBIEE) and DWH


Trainer, participants introduction


Training is aimed mainly to: Business Intelligence developers DWH/Data designers BI Architects Business analysts


Training Audience


Desirable experience/knowledge for participants: Some BI/DWH experience Knowledge of Data modeling principles, ideally Kimball Dimensional Modeling Knowledge (at least basic) of SQL language


Training Prerequisites


Gain basic knowledge of OBIEE10g architecture and its components Gain knowledge of metadata structure (layers) Gain basic knowledge of BI Administration Tool (client sw, used for designing metadata) Learn basic administration routines and best practices Gain (using practical labs) essential basic knowledge for being able to independently build OBIEE10g on top of relational data sources


Training Objectives


Day 1: Training Introduction/Logistic OBIEE10g architecture/Repository Introduction Physical Layer of a Repository Business Model and Mapping Layer of a Repository Presentation Layer of a Repository

Day 2: Testing and Validating a Repository Adding Multiple Logical Table Sources Adding Calculations to a Fact Creating Dimension Hierarchies and Level-Based Measures


Training Agenda


Day 3: Using Aggregates Using Repository Variables Modeling Time Series Data Security Basics


Training Agenda


Become familiar with OBIEE10g architecture and its components Become familiar with 3 layers of OBIEE10g repository (metadata) Learn about BI Administration Tool

OBIEE10g architecture / Repository Introduction

Lesson Objectives



OBIEE10g architecture

Ad-hoc Analysis

Proactive Detection and Alerts

MS Office Plug-in

Reporting & Publishing

Interactive Dashboards

Disconnected Analytics

Oracle

BI Server Multidimensional Calculation and Integration Engine

Intelligent Caching Services

Enterprise Business Model and Abstraction Layer

Web Services

OLTP & ODS Systems

Data Warehouse Data Mart

SAP, Oracle PeopleSoft, Siebel, Custom Apps

Files Excel XML

Business Process

Physical Layer

Oracle BI Server

Oracle BI server generates one or more Physical SQL queries


Main OBIEE10g architecture components: Clients BI Presentation Services BI Server Data sources


OBIEE10g architecture


Provide access to BI content (requests, dashboards) within OBIEE10g UI: Answers tool for creation/modification/view of requests (synonym for a

Interactive Dashboards for displaying Answers requests along with other content on dashboard pages (also allowing interactivity for exposed requests


Clients


Takes care of rendering OBIEE10g web based UI : Answers Interactive Dashboards Delivers Administration

dashboards..) Presentation Catalog Gets the data from Oracle BI server and provides it to the clients


BI Presentation Services


Uses metadata (repository) during data processing Communicates with underlying data sources For relational data sources it generates dynamic SQL select statement to get the data

connect to data sources It provides results of the queries (eventually enriched by its own calculations) to

Acts as a virtual data source to its surroundings (accessible via ODBC interface)


BI Server


Contains metadata used by BI Server It has physically the form of binary file with RPD extension

semantic model (business model) Repository is created/modified by client tool BI Administration Tool (available on Windows platform only)


BI repository (metadata)


Contains data/information relevant to business users for analysis/reporting BI Server is getting data from the data sources BI Server can work with different data sources:

Relational databases (RDBMS) most common OLAP data sources (Essbase, MSAS, SAP BW..) XML files Excels (on Windows platform only, mainly for demonstration purposes)

be able to generate most effective SQL statements

of suitable form Kimball dimensional model)


Data sources


1 - User submits a request (report) 2 - BI Presentation Services asks for a data (in form of logical SQL) to BI Server 3 - BI Server uses a metadata (repository) to address correct data source and generates optimized SQL to get the data 4 - BI Server receives the data from the data sources and perform any post-processing if needed (internal calculation, internal joins of data sets....) 5 - BI Server passes the data to Presentation Services (result of logical SQL send to BI Server) 6 - BI Presentation Services formats the data and sends it to the client (Answers, Dashboards)


Request Execution Data Flow


Tool for building BI Server metadata divided into 3 layers: Physical Business Model and Mapping (BMM) Presentation


BI Administration Tool



Metadata Layers

Presentation Layer

Physical Layer

Semantic Object Layer

Simplified view Logical SQL interface

Dimensions Hierarchies Measures Calculations Aggregation Rules Time Series

Map Physical Data Connections Schema



Physical Layer

Physical Layer

Multiple sources Optimized SQL generation Regardless of Schema Function ship to appropriate data sources/Compensation

DB2

Supply

Chain

DM

Teradata

OLAP

Oracle

ERP.

XML Data

Source

SQL Server

Acxiom

Siebel

Operational


Contains objects representing physical data sources used for reporting/analysis by BI Server Can multiple (even disparate) data sources First layer built in the repository


Physical Layer



Physical Layer Objects

Data Source Connection Pool

Aliases with specific prefixes

Actual tables

Schema folder

Extra aliases to prevent circular joins

Extra aliases for

reporting

Canonical time dimension



Business Model and Mapping Layer

Business Model Layer

Physical complexity converted to logical subject areas Drill-Paths Complex/Derived Measures (Level-based, time series, dimension-specific, nested) Aggregate/Fragment Aware



Business Model and Mapping Layer transformation into dimension model

XML

CSV

ODBC

OLTP

OLAP

Dimensions

Facts


(semantic layer) understandable by end/business users is performed It has a form of logical star schemas according to dimensional modeling methodology (R. Kimball)


Business Model and Mapping Layer



Business Model and Mapping Layer Objects

Dimensions (Hierarchies)

Dimension Logical Tables

Fact Logical Tables

Logical Table Sources

Logical Columns

Business Model

Measures (Fact Logical Columns )



Business Model Mappings

BMM layer objects map to data objects in the Physical layer Mappings is usually not one-to-one:

Business models may map to multiple data sources Logical tables may map to multiple physical tables. Logical columns may map to multiple physical columns



Presentation Layer

Presentation Layer Role-based, in context, personalized presentation Oracle Answers


Only metadata layer directly exposed to end users Contains objects, providing view to a business model for end users Exposes just objects, relevant to users (simplification)


Presentation Layer



Presentation Layer Objects Presentation Catalog

Presentation Folder

Presentation Column

Subject Area

Folder

Column



Presentation Layer Mappings

Presentation layer objects map to objects in BMM layer Subject area/Presentation Catalog maps to Business Model Presentation Folder may map to logical table Presentation Column maps (strict mapping) to logical column



Repository Directory

\OracleBI\server\Repository



Repository Modes

Repository files can be opened for editing in offline or online mode Offline Mode:

Repository file (RPD) is open from file system Online Mode:

Connection is made (via ODBC driver) to live BI Server We are working against repository loaded into Oracle BI Server memory Administrators can perform tasks not available in offline mode:

Manage scheduled jobs Manage user sessions Manage the query cache Manage clustered servers Stop Oracle BI Server



NQSConfig.ini configuration file

One entry in configuration file NQSConfig.INI (\OracleBI\server\Config) instructs BI server, which RPD file to use for its operation



Summary

In this lesson, we have learned about : Key components of OBIEE10g architecture Three layers of BI Server repository and their relations BI Administration Tool and its purpose



Labs - Overview

This lesson is accompanied with following labs: Exploring BI Server repository


Learn about the objects in Repository Physical layer Create Repository Physical layer

Physical Layer of a Repository

Lesson Objectives



Contains objects representing physical data sources used for reporting/analysis by BI Server Can multiple (even disparate) data sources First layer built in the repository

Physical Layer



Physical Layer Objects

Database object

Connection Pool

Aliases with specific prefixes

Actual tables

Schema folder

Extra aliases to prevent circular joins

Extra aliases for

reporting

Canonical time dimension



Represents the highest-level object in Physical layer Specifies data source, which is used by BI Server for querying (database type, etc...)

Database Object

Database object



Name DB instance name) Database type of the data source (influences queries generated by BI server) :

Relational Multidimensional File based (XML)

Database Object General tab



You can set the SQL features, BI Server can leverage when querying this data source OBIEE has "knowledge base" for default features for registered data source types (like Oracle Db) Be careful when overwriting the default values (can considerably influence generated SQL statements) You can "Ask DBMS" for setting DB features while working in online mode

Database Object Features tab



Connection Pool Defines connection of BI Server to a data source Could be either generic ODBC or native connectivity (Oracle Db) Uses principle of "connection pooling" - multiple users share pool of connection to a data source Connection

Pool Name

Connectivity Type (Native/ODBC)

Max no of connections

For querying Db objects in different owner schemas

Data source name (ODBC name or connect descriptor)

Shared logon username/password

Connection Pooling Enabled



Physical Schema Optional object Container object for definitions of tables, columns For Oracle Db it represents Db schema, owning imported database objects definitions (tables, views) its name is used in generated SQL - .

Physical Schema



Physical Table Corresponds to a table (or view) in a physical data source Most typically the definition is imported from a database or other data source (although can be created manually as well) Provides metadata needed for BI Server to access the tables/views with SQL queries

Physical Table



Physical Table Properties Table Type:

Physical Table (most common) Stored Proc (never used from my experience)

source) - functionality

Name

Table Type

Dynamic Name (using variables)

Caching Properties



Alias Represents "pointer" to physical table definition (created by right-clicking on physical table definition) Used mainly to avoid circular join definition in physical layer (for example role playing dimensions in Kimball methodology) Alias structure (columns) is "read-only", always reflects structure of original table definition Appears with different icon in physical layer

Alias Name

Source Table



Physical Column Object that corresponds to a column in a physical database Child object of physical table It has data type (internal BI Server, corresponding to Db data types) and nullable property

Columns



Key Column Defines relationships between tables Primary key:

Uniquely identifies a single row of data Consists of a column or set of columns Is identified by a key icon

Foreign key: Refers to the primary key columns in another table Is composed of a column or set of columns

Key Column

Foreign Keys within table definition



Joins Represent primaryforeign key relationships between tables in the Physical layer Usually defined in Physical diagram Join Properties define:

Tables joined Cardinality Join Expression

Join Properties

Physical Diagram



Labs Scenario

in a schema SH containing data about : Sales figures (amount, quantity) by day, product, promotions, channels and customers Unit price and costs by day , product, promotions, channels



Physical Layer Implementation Steps

1. Import the physical schema from data source 2. Select tables (and columns) to be imported 3. Import keys and joins (optional) 4. Check the import 5. Edit connection pool properties 6. Define physical keys and joins



Import the physical schema

Using BI Administration Tool menu File -> Import -

Connect descriptor from tnsnames.ora

Connection Type (OCI for Oracle Db)

Credentials for connection to Db



Select tables (and columns) to be imported

In import wizard (next step) select tables/columns needed for building physical model in physical layer (you can add tables later on)

Filtering tables for import

Tables selected for import

Objects for import



Import keys and joins

You can import keys, foreign keys and corresponding joins, if they are defined in a data source This selection is optional, you can define keys, joins "manually" later on (to have better control over the build physical data model)



Check the import

Check, whether correct schema, tables, columns, and keys has been imported You can use View Data option to check the connection to data source



Edit connection pool properties

After import (or during import when importing for the first time), you check or modify the connection pool properties using the Connection Pool properties dialog box Connection

Pool Name

Connectivity Type (Native/ODBC)

Max no of connections

For querying Db objects in different owner schemas

Data source name (ODBC name or connect descriptor)

Shared logon username/password

Connection Pooling Enabled



Define physical keys and joins

In case keys and joins has not been imported automatically (either intentionally or they do not exists in Db) from data source (Db), you can define them in Admin Tool:

Define keys (and also joins) using the Physical Table properties dialog box Define joins and keys using the Physical Diagram



Defining Keys Using the Table Properties

Key Name

Select columns, constituting a key



Using the Physical Diagram

Physical Diagram can be open by selecting tables in physical layer and then Right- Or use



Physical Diagram Foreign Key

in a toolbar

Drag from table with cardinality 1 and drop to table with cardinality N Define join property (name, join expression) in Physical Foreign Key dialog window

FK Name

Join Expression



Summary

In this lesson, we have learned : About objects constituting Physical layer of a repository How to create Physical layer of a repository



Labs - Overview

This lesson is accompanied with following labs: Become BI solution (metadata) Create empty repository and import tables Define keys and joins


Learn about the objects in Repository Business Model and Mapping (BMM) layer Create a business model Create measures in a business model

Business Model and Mapping

Layer of a Repository

Lesson Objectives


(semantic layer) understandable by end/business users is performed It has a form of logical star schemas according to dimensional modeling methodology (R. Kimball) Measures and (Dimensional) attributes are clearly defined in this layer



Business Model and Mapping (BMM) Layer




Business Model and Mapping Layer Objects

Dimensions (Hierarchies)

Dimension Logical Tables

Fact Logical Tables

Logical Table Sources

Logical Columns

Business Model

Measures (Fact Logical Columns )




Business Model Mappings

BMM layer objects map to data objects in the Physical layer Mappings is usually not one-to-one:

Business models may map to multiple data sources Logical tables may map to multiple physical tables. Logical columns may map to multiple physical columns

Mapping of logical columns can also include transformations/formulas




BMM Layer Objects

Business model Logical tables (logical dimensions/logical facts) Logical table sources Logical columns Measures (Special form of logical column) Logical primary keys Logical joins




Business model

Is the top-level object in the BMM layer Contains the business model definitions (logical star schemas) and the mappings from logical to physical tables Business Model uses terminology familiar to business users (not "cryptic" physical object names)

Business Model

Business Terms




Logical Table

Logical Table contains logical columns that can be mapped to one or more physical columns from the physical layer Represents logical fact or logical dimension Can be created:

Automatically by dragging tables from Physical layer Manually by right-clicking business model and selecting New Object > Logical Table




Logical Table Source (LTS)

Define the mappings from a logical table to a physical table Logical tables may have multiple logical table sources One logical table source may map to many physical sources




Logical Table Source - Column Mappings

Within Logical Table Source (LTS), you define mapping between logical columns of logical table and physical columns from tables, constituting LTS Column Mapping tab within LTS dialog box is intended for building, viewing or modifying logical to physical column mappings




Logical Column

Represent the business element (attribute, measure) of the data Can map to many columns in the Physical layer Can be defined by referring other logical columns in formula (calculation) Can be created:

Automatically by dragging tables or columns from the Physical layer Manually by right-clicking a logical table and selecting New Object > Logical Column (and then defining mapping to physical column within LTS)




Logical Table Primary Key

Define unique identifiers (logical columns) for logical tables (but from business point of view, not for example " generated DWH keys) Define the lowest level of detail of a logical table Logical Keys are only required for logical dimension tables (for repository to be valid), not for logical fact tables




Logical Join

Defines the cardinality relationship between logical tables (derives the logical table role in model - dimension or fact) Is required for a valid business model Help Oracle BI Server understand the relationships between various objects of the business model Examining logical joins is an integral part of how Oracle BI Server figures out how to construct the physical queries There is no "join expression" defined as part of logical join definition (as opposed to join definition on physical layer - FK)




Logical Join




Measure

Represents calculations with measurable quantity Special form of logical columns placed in logical fact tables Have aggregation rule defined in metadata




Scenario

Logical dimensions:

Time (Periods) Products Customers Channels Promotions

Logical Facts : SALES - main fact table containing total sales measures (amount, quantity) by day, product, promotions, channels and customers COSTS - "supporting" fact table with unit price and unit cost measures by day, product, promotions, channels




Scenario




BMM Layer Implementation Steps

1. Create business model 2. Create logical tables and columns 3. Define logical joins 4. Modify logical tables and columns 5. Define measures in logical fact tables




1. Create business model

Right-click in the BMM layer and select New Business Model (more control over the process) Or Dragging selected physical tables into BMM layer




2. Create logical tables and columns

Drag physical table objects from the Physical layer into BM definition fast, but less control over the process (creates automatically also logical columns for all physical columns) Or right-click BM object and select New Object > Logical Table (logical columns needs to be defined afterwards)




3. Define logical joins

Similar to definition of foreign key joins in physical layer Business Model Diagram used for it and complex join used instead of foering key join (with no join expression defined, just cardinality)




4. Modify logical tables and columns

Rename tables and columns Reorder columns Add or delete tables and columns Add, delete and modify logical table sources (LTS) Cut, copy and paste objects (but with




5. Define measures in logical fact tables

Special form of logical columns placed in logical fact tables (with aggregation rule) Right-click the logical column (numeric) in logical fact table and select Properties > Aggregation




Recommended practice

Use only complex joins in BMM layer !!!!!!! Rename logical columns to have meaningful names by end user community (business dictionary) Use (if possible) short enough names (due to the space in requests) Use unique names for logical columns to avoid ambiguity




Summary

In this lesson, we have learned : About objects constituting BMM layer of a repository How to create BM layer of a repository (including simple measures)




Labs - Overview

This lesson is accompanied with following labs: Creating Business Model Creating Simple Measures


Learn about the objects of Presentation layer of a repository Modify the properties of Presentation layer objects Create Presentation layer

Presentation Layer of a

Repository

Lesson Objectives


Only metadata layer directly exposed to end users Contains objects, providing view to a business model for end users Exposes just objects, relevant to users (simplification)


Repository

Presentation Layer


Organize and simplify business model for a set of users Single presentation catalog must be populated from a single business model (presentation catalog cannot "span" across business models) But multiple Presentation catalogs can reference the same business model (pretty common scenario)


Repository

Presentation Catalog


Organize presentation columns into that make sense to the users May contain columns from one or more logical tables Can be modified independently of logical tables (no strict "mapping" between logical table and presentation table) Can be created:

Automatically by dragging logical tables from BMM layer Manually in the Presentation layer


Repository

Presentation Tables


Define the columns used to build requests in the OBIEE user interface Map to logical columns in the BMM layer (strict 1:1 mapping) Usually derive their names from corresponding logical column name (best practice) Can be created:

Automatically by dragging logical tables or columns from BMM layer Manually in the Presentation layer


Repository

Presentation Columns



Repository

Presentation Layer Mappings

Presentation layer objects map to objects in BMM layer Subject area/Presentation Catalog maps to Business Model Presentation Table may map to logical table Presentation Column maps (strict mapping) to logical column



Repository

Defining User Interface in the Presentation Layer

Presentation layer objects define the content (Subject Areas), which end users see to query the data from the data sources



Repository

Nested Presentation Folders/Tables

Give the appearance of nested folders in Answers tool In Description of presentation table put following characters ate the beginning : -> Place presentation table after the table in which you would like to nest it



Repository

Presentation Layer Object Aliases

Keep track of any changes to Presentation layer objects Used in Answers when referencing renamed presentation layer object in Answer request Have additional overhead for BI Presentation Server Avoid renaming after BI outputs has been intensively created (rather perform replacing in Web Catalog)



Repository

Scenario

data



Repository

Presentation Layer Implementation Steps

1. Create new presentation catalog 2. Rename tables 3. Reorder tables 4. Delete columns 5. Rename columns 6. Reorder columns



Repository

1. Create a New Presentation Catalog

Drag SH business model from BMM layer to Presentation layer to automatically create Presentation layer objects (the simplest method)



Repository

2. Rename tables

There are a few methods for renaming tables in Presentation layer: Use the General tab in the properties dialog box Right-click a table and select Rename Double-click a table slowly to make the name editable and then enter a new name



Repository

3. Reorder tables

For reordering of presentation tables in Presentation Catalog, open the Presentation Catalog properties and use the Up and Down buttons or drag/drop



Repository

4. Delete columns

Delete unnecessary columns to simplify the content and make it more understandable to users (don't overcomplicate structure of Presentation Catalog for end users)



Repository

5. Rename columns

Presentation columns use the logical column name by default - this is the recommended approach to "inherit" logical column names in presentation layer to have consistent naming of a single logical column across different Presentation Catalog If there is a special need for renaming presentation column, use General tab in the column properties dialog box to deselect the Use Logical Column Name check box and enter custom name



Repository

6. Reorder columns

For reordering of presentation columns, open Presentation Table properties box and use the Up and Down buttons or drag/drop



Repository

Consideration

Presentation catalogs can map to only one BM Multiple presentation catalogs can map to the same BM Presentation columns in single presentation table can come from multiple logical tables in BM Presentation columns are automatically renamed when corresponding logical column is renamed (with alias created for previous name) Presentation tables cannot have the same name as presentation catalogs Presentation objects can be modified or deleted without affecting corresponding logical objects Be careful with modification/restructuring of presentation objects in case BI "outputs" are already using them



Repository

Recommendation/Best Practice

Use meaningful names

prevents it (also avoid spaces at the beginning/end) Keep presentation object names unique:

Naming presentation columns the same as presentation tables can lead to inaccurate results Uniqueness allows SQL statements to be shorter because qualifiers are unnecessary

Group objects in meaningful ways (grouping in presentation tables) Eliminate unnecessary objects to reduce confusion (KISS principle) Use object description fields to convey information to users (displays as tooltip in Answers) Keep names short to save space on reports



Repository

Summary

In this lesson, we have learned : About objects constituting Presentation layer of a repository How to crete Presentation layer of a repository How to modify the properties of Presentation layer objects



Repository

Labs - Overview

This lesson is accompanied with following labs: Creating Presentation layer objects Modifying Presentation layer objects


Learn about execution of steps for testing and validating a repository

Testing and Validating a

Repository

Lesson Objectives


The following steps validate whether the repository is constructed correctly and whether it produces expected query results:

Checking repository for consistency Enabling logging Loading/Deploying a repository Checking a repository using BI Answers front end tool Inspecting the query log


Repository

Validating a Repository



Repository

Scenario Validate and test SH business model before making it available for using by end users


It is a feature of BI Administration Tool that checks whether a repository has met certain requirements, such as:

All logical columns are mapped directly or indirectly to one or more physical columns. All logical dimension tables have a logical key. All logical tables have a logical join relationship to another logical table. There are at least two logical tables in the business model: logical fact table and logical dimension table. Both can map to the same physical table. There are no circular logical join relationships. At least one Presentation Catalog exists for business model.

Does not guarantee that the business model is constructed correctly


Repository

Consistency Check


Check consistency for the entire repository or for individual repository objects by using either of the following menus (you are also asked when saving RPD):


Repository

Checking Consistency

File Menu Tools Menu BMM layer object right-click menu


Displays consistency check messages: Errors: Must be fixed to make the repository consistent Warnings: Condition that may or may not be an error Best Practices: Condition does not indicate an inconsistency (disabled by default)


Repository

Consistency Check Manager


The Options tab in the Consistency Check Manager allows to enable or disable consistency messages


Repository

Disabling/Enabling Consistency Check Messages


BI Server provides a facility for logging query activity at the individual user level (driven by variable LOGLEVEL set for individual users) Logging is intended for quality assurance testing, debugging, and use by Oracle Support Query logging is normally disabled in production mode (turning logging on causes additional overhead on OBIEE server infrastructure - log file can extensively grow) Query log file is named NQQuery.log and is located in the \OracleBI\server\Log directory Various levels of query logging enables various degree of details logged


Repository

Enabling Logging


Use the Security Manager to enable logging level for individual users Or set LOGLEVEL session variable using Initialization Block (session variables are

Repository


Repository

Setting a Logging Level


There are 7 log levels (higher levels are intended just for communicating issues with Oracle support) Logging level 2 allows you to see the SQL generated (sufficient for testing/debugging)


Repository

Logging Levels


After you build a repository and it is consistent, you need to "point" BI Server to use this RPD - add/modify an entry in the NQSConfig.ini file


Repository

Adding a Repository Entry to an Initialization File


If the repository was edited in offline mode (recommended practice), start or restart BI Server process to load new/modified repository If the repository was edited in online mode (just for small changes, not recommended), check in changes and reload the server metadata in Answers


Repository

Loading Repository


Use Oracle BI Answers to validate Presentation layer by creating testing request (report) and displaying query results


Repository

Validating by Using BI Answers


Use the query log to check query results: Either directly looking at NQQuery.log file on the OBIEE server(located in \OracleBI\server\Log) Or by going into Manage Session in Administration page of OBIEE UI


Repository

Inspecting the Query Log



Repository

Inspecting the Query Log


Basic syntax for an Oracle BI SELECT statement : SELECT . list FROM WHERE GROUP BY ORDER BY . list (optional)

Example of logical SQL : SELECT Calendar."Calendar Year" , Products."Prod Category" , "Sales Facts"."Amount Sold" FROM SH WHERE Calendar."Calendar Year" = 2000 ORDER BY Calendar."Calendar Year" , Products."Prod Category"


Repository

BI Server Logical SQL SELECT


BI Server Logical SQL SELECT statements are different from standard SQL in the following ways: No join information is required (SQL is constructed against Presentation layer objects)

Join conditions are predefined in the repository Any join conditions supplied in a query are ignored

No aggregation functions are required (since measures have their aggregation rule defined in repository and thus performed for them automatically), although aggregation function can be defined in logical SQL No GROUP BY clause is required by default

If aggregated data is requested in the SELECT statement, a GROUP BY clause is automatically assumed by the server

No DISTINCT keyword is required BI Server always issues SELECT DISTINCT to eliminate duplicate rows automatically


Repository

BI Server Logical SQL SELECT SELECT



Repository

Summary

In this lesson, we have learned : How to execute the steps to test and validate a repository



Repository

Labs - Overview

This lesson is accompanied with following labs: Testing and validating new repository (along with deployment) Generating inconsistent repository (business model) and fixing consistency errors


Describe normalized and de-normalized table structures in database designs Add multiple sources to a logical table source (LTS) for a dimension/fact in business model Topic : Adding a second logical table source to logical table - will be covered in Aggregation lesson

Adding Multiple Logical Table

Sources

Lesson Objectives


Normalized table structures (usually transactional systems - OLTP): Consists of many tables where data has been split or normalized Are used for inserts and updates Does not work well for queries that perform business data analysis Could occur also in dimensional model (snowflake)

De-normalized table structures (DWH - dimensional model): Follows more a business model and is easier to understand Has data that may be duplicated in several locations in a database Can take the form of a star schema Provides better query performance


Sources

Table Structures


Data may be spread across several physical tables and needs to be mapped to a single logical table


Sources

Business Challenge


Model multiple physical sources for the logical table: Add multiple sources/tables to an LTS

Where data is not duplicated across tables Add a new logical table source

Where data is duplicated across tables (aggregated tables, fragmented data....)


Sources

Business Solution


Add normalized table that store geographical attributes for customer (country, subregion, region) to Customers dimension in SH business model


Sources

Logical Table Source (LTS)


1. Import additional tables 2. Define keys and joins 3. Identify physical columns for mappings 4. Add sources/tables to a logical table source:

Manual method Drag method

5. Rename logical columns 6. Add columns to the Presentation layer


Sources

Implementation Steps: Adding Multiple Sources to an LTS


Import additional table, that contains geographical attributes for Customers


Sources

1. Import additional tables


Define the keys and joins for the imported table(s)


Sources

2. Define Keys and Joins


Identify columns in the Physical layer with the additional geographical information for customer


Sources

3. Identify Physical Columns for Mappings


There are alternative methods , how you can add sources to an LTS: Manual: Use the Properties dialog box of Logical Table Source (LTS) to map tables and columns Drag: Drag columns from the Physical layer to LTS to automatically map tables and columns


Sources

4. Adding Sources to an LTS


Create a new logical column for a logical dimension table


Sources

4a. Manual: Create New Logical Column


Use the Properties dialog box of a logical table source to add a new physical table to the source


Sources

4a. Manual: Add New Physical Source


Use the Properties dialog box of a logical table source to map a new logical column to a physical column in the new table within LTS


Sources

4a. Manual: Create Column Mapping


LTS CUSTOMERS now maps to two physical tables: CUSTOMERS and COUNTRIES Logical column Country maps to COUNTRY_NAME physical column in COUNTRIES physical table


Sources

4a. Manual: End Result


Drag physical columns to a logical table source


Sources

4b. Drag Method


Dragging physical columns to LTS automatically creates logical columns in BMM layer


Sources

4b. Drag Method: Logical Columns Added


Dragging physical columns to LTS automatically adds physical tables to LTS


Sources

4b. Drag Method: Physical Tables Added


Dragging physical columns to LTS automatically adds column mappings to LTS


Sources

4b. Drag Method: Column Mappings Added


Rename new logical columns with names that are meaningful to users (not necessary for manual method)


Sources

5. Rename Logical Columns


Drag new logical column(s) to Customer presentation table to make it(them) available to users


Sources

6. Add Columns to Presentation Layer



Sources

Summary

In this lesson, we have learned : Describe normalized and de-normalized table structures in database designs Add multiple sources to LTS for a dimension in the business model



Sources

Labs - Overview

This lesson is accompanied with following labs: Importing normalized table that contain additional geographical information for customer to Physical layer, defining physical keys and joins Creating multiple sources (adding table) to LTS using manual method


Describe a calculation measure and its use in a business model Create new calculation measures based on logical columns Create new calculation measures based on physical columns Create new calculation measures by using the Calculation Wizard

Adding Calculations to a Fact

Lesson Objectives


Physical schema may not have everything required for your analysis Adding columns into physical tables may require additional integration or ETL work Deriving a metrics in such scenario may be more efficient than storing values in the database

Average Order Size: Divide Revenue by Number of Orders


Derived Metrics


BI Server provides utilities to create calculation measures in the business model Use the Expression Builder to create new logical columns with a calculation formula Use existing logical columns or physical columns as objects in the formula Use the Calculation Wizard to create calculation measures based on existing logical columns Expose calculation measures to the Presentation layer so that users can formulate business questions in Answers by using familiar terminology


OBIEE solution


"XYZ Selling Tigers" wants to track a calculated measure Gross Profit, that calculates the difference between Amount Sold and Unit Cost (stored in table COSTS) multiplied by Quantity Sold : Gross Profit = Amount Sold - (Unit Cost * Quantity Sold)


"XYZ Selling Tigers" Scenario


Calculation measures can be created using 2 different methods: Existing logical columns as objects in a formula Physical columns as objects in a formula

You can use as well Calculation Wizard to automate the process of creation of comparison measures (comparing 2 existing measures/logical columns)


Implementation Methods


1. Create a new logical column 2. Specify logical columns as the source 3. Build a formula


Steps for Using Existing Logical Columns


Right-click the fact table and select New Object > Logical Column


1. Create a New Logical Column


Select "Use existing logical columns as the source" check box


2. Specify Logical Columns as the Source


Open the Expression Builder and build the calculation formula using existing logical columns (from logical tables)


3. Build a Formula


1. Create a new logical column 2. Map the new column 3. Build the formula 4. Specify an aggregation rule


Steps for Using Physical Columns


Use the Column Mapping tab of LTS dialog box to open the Expression Builder for the new column


2. Map the New Column


Build the calculation formula using physical columns


3. Build the Formula


Click the Aggregation tab and set the aggregation rule for new calculated logical column


4. Specify an Aggregation Rule


1. Open the Calculation Wizard 2. Choose the columns for comparison 3. Select the calculations 4. Confirm the calculation measures 5. New calculation measures are added


Steps for Using the Calculation Wizard


Right-click a logical column that you want to use in the calculation and select Calculation Wizard


1. Open the Calculation Wizard


You can choose more columns to compare with


2. Choose the Columns for Comparison


You can choose different comparison calculations (Change, Percent Change, Index, Percent), by default Change (absolute difference) and Percent Change selected You can name the calculation measures You can also choose NULL value handling


3. Select the Calculations


Final screen of an wizard with summary of what measures will be created


4. Confirm the Calculation Measures


New calculation measures are automatically added to the logical fact table


5. New Calculation Measures Are Added


Expose new calculation measures to Presentation layer regardless of the method used to create the measures


Expose New Measures to Presentation Layer


Expression builder allows you to select various expressions, operators and repository variables and use them to build the column formula


Advanced Functions Expression Builder

Using functions

Variables can be built and used in building formula, e.g. Current Month, Last ETL Run Date etc.


Use physical columns for building calculations that require an aggregation rule (such as sum or average) that is applied after the calculation


Best practice for measures calculated from physical columns

select

T82.ItemType as c1,

sum(T55.UnitOrdd - T55.UnitShpd) as c3

from

D1_products T82,

D1_Orders2 T55

where

T55.ProdKey = T82.ProductKey and

group by

T79.ItemType

Calculates the difference between units ordered and units shipped

Applies aggregation rules on top of the calculated values


Example: To accurately calculate total revenue you would multiply the unit price by the number of units sold and then sum the totals


Example of measures calculated from physical columns


Use logical columns for calculation formulas that require an aggregation rule that is applied before the calculation


Best practice for measures calculated from logical columns

Select

distinct D1.c1 as c1, D1.c2 as c2. D1.c3 as c4,

(D1. c2 D1.c3) as c4

From (

select

T82.ItemType as c1,

sum(T55.UnitOrdd) as c2,

sum(T55.UnitShpd) as c3

from

D1_products T82,

D1_Orders2 T55

where

T55.ProdKey = T82.ProductKey and

group by

T79.ItemType ) D1

First partial measures are aggregated

And then final calculation (difference) is used with aggregated measures


Example: To accurately calculate unit price, sum Total Price and Units Sold first and then divide Total Price by Units Sold to determine Unit Price


Example of measures calculated from logical columns



Summary

In this lesson, we have learned : About a calculation measure and its use in a business model How to create new calculation measures based on existing logical columns How to create new calculation measures based on physical columns How to create new calculation measures by using the Calculation Wizard



Labs - Overview

This lesson is accompanied with following labs: Creating Calculation Measures by Using Physical Columns Creating Calculation Measures by Using Physical Columns


Creating Dimension Hierarchies

and Level-Based Measures

Lesson Objectives

Create dimension hierarchies Create level-based measures Create rank and share measures


Defines parent-child relationships within a dimension Establishes levels for data groupings and calculations Provides paths for drilldown



Dimension Hierarchies

Years

Quarters

Months

Days

Period Dimensional Hierarchy

Le

ve

ls


Level-based hierarchy, general hierarchy style where a dimensional column act as the parent to a child level column

Unbalanced (or ragged) hierarchy, is a hierarchy where the leaves (members with no children) do not necessarily have the same depth. Skip-level hierarchy, is a hierarchy where there are members that do not have a value for a particular ancestor level.

Value-based hierarchy (Parent Child Hierarchy), uses the same dimension column for all levels but based on relational tables (parent-child relationship table) in the data model to identify the parent-child relationship. In OBIEE10g metadata, only level-based (balanced, not ragged) hierarchy is supported for modeling



Types of Hierarchies


Are columns whose values are calculated (aggregated) always at a specific level of hierarchy



Level-Based Measures

Period Dimensional Hierarchy Levels

Grand Total

Years

Quarters

Months

Days

Measures

All Period Revenue

Year Revenue

Quarter Revenue

Month Revenue

Day Revenue




Share Measures

Are calculated by dividing a measure by a level-based measure to calculate a percentage contribution of a measure to its upper level aggregation




Dimension Hierarchy: Example

This is an example of a dimension hierarchy based on Products dimension table




logical dimensions : Dim - Channels Dim - Customers Dim - Products Dim - Promotions Dim - Times




Steps to Create a Dimension Hierarchy

1. Create a dimension object 2. Add a parent-level object 3. Add child-level objects 4. Determine the number of elements 5. Specify level columns 6. Create level keys 7. Set the preferred drill path 8. Create a level-based measure 9. Create additional level-based measures 10. Create share measures 11. Add measures to the Presentation layer 12. Test share measures




1. Create a Dimension Object

Alternative methods are: Right-click the business model object and select New Object> Dimension (recommended) Right-click the logical dimension table and select Create Dimension




2. Add a Parent-Level Object

Create the highest (top) level of the hierarchy first




3. Add Child-Level Objects

Add subsequent(child) levels in the hierarchy




4. Determine the Number of Elements

Part of previous step (3. Add Child-Level Objects) There are two methods for determining the number of elements for a dimension level:

Using result from "Update Row Counts" operation (this operation is usually used just for small amount of data) Estimate Levels - only accessible when working in online mode

Update Row Counts Estimate Levels




5. Specify Level Columns

Specify which columns in the logical dimension table are associated with particular level in the dimension hierarchy




6. Create Level Keys

Level keys: Define the unique identifier for the level Provide context for drilldown (specifies the subset of data to include from the next level down)




7. Set the Preferred Drill Path

Use Preferred Drill Path to specify a drill path outside the normal drill path defined by the dimension level hierarchy (quite rarely used)




8. Create Level-Based Measures

Example: Create a level-based measure for the Grand Total level of particular dimension (Product) that refers to an existing logical fact column




8. Create Level-Based Measures

How level-based measure appears in Answers request




9. Create Additional Level-Based Measures

Create other level-based measures to make measures from various levels of various dimensions (and for various "based" measures) available simultaneously in queries using same process as in previous point




10. Create Share Measures

Create a new logical fact column that calculates the share by dividing the -appropriate measure by a total measure (level-based measure) this is an example of measure, calculated from logical columns




11. Add Measures to the Presentation Layer

Expose new measures to Presentation layer so they can be used by end users




12.Test share measures

In Answers, select measures to test (share measure along with corresponding level-based measure) and then drill down to check relative recalculations




Summary

In this lesson, we have learned : How to create dimension hierarchies How to create level-based measures How to create share measures




Labs - Overview

This lesson is accompanied with following labs: Creating Dimension Hierarchies Creating Level-Based Measures Creating Share Measure


Using Aggregates

Lesson Objectives

Describe aggregate tables and their purpose in dimensional modeling Model aggregate tables Use the Aggregate Persistence Wizard to create aggregates


Using Aggregates

Business Challenge

Data in fact and dimension sources is stored at the lowest level of detail Data often needs to be rolled up or summarized during analysis (example: users often requests total products by region by year ) Based on the amount of data, performing calculations at the time of the query can be resource intensive and can delay results to the user (thus leads to end user dissatisfaction with BI application)


Using Aggregates

Business Solution: Aggregate Tables

Create physical tables that store pre-computed aggregates at required levels Use these "aggregate" tables to process user queries

Eliminates run-time calculations Delivers faster results to the users

Summarized

Office Year Total

Dollars

West 1998 100000

West 1999 200000

Central 1998 50000

Id Office

Key

Time

Key

Product

Key Dollars

122222 1001 19980102 100 100000

133333 1001 19990105 200 200000

144444 1002 19980505 300 10000

155555 1002 19980601 400 20000

166666 1005 19980101 600 20000


Using Aggregates

BI Server Aggregate Navigation

Enables queries to use the information stored in aggregate tables automatically BI Server decides which tables provide the fastest answers Metadata must be configured for aggregate navigation


Using Aggregates

Aggregated Facts

Aggregated sales fact table columns store precomputed results at a given set of levels

Total

Organization

Department

Total

Year

Month

Week

Day

Total

Brand

LOB

Type

Detail

Dim

en

sio

n H

iera

rch

ies

Company

Product Hierarchy Office

Hierarchy

Time Hierarchy


Using Aggregates

Modeling Aggregates

Model aggregate tables in the same way as other source data Physical layer:

Import physical table Create physical joins

BMM layer: Add sources to logical tables Specify aggregation content - identifies the level of aggregation in the table so BI Server can determine when to use it for queries

Presentation layer: No changes: Aggregate navigation is independent of Presentation layer objects

Test the results

New step


Using Aggregates

Uses prebuilt aggregate tables to improve performance Must have matching levels of aggregation for fact and dimensions Simplified scenario:

Materialized View CAL_MONTH_SALES_MV aggregates Amount Sold by Month This MV will serve as LTS for aggregated fact and as well as

Times)


Using Aggregates

Steps to Implement Aggregate Navigation

1. Import tables 2. Create joins 3. Create fact logical table source and mappings 4. Specify fact aggregation content 5. Specify content for the fact detail source 6. Create dimension logical table source and mappings 7. Specify dimension aggregation content 8. Specify content for the dimension detail source 9. Test results for levels stored in aggregates 10.Test results for data above or below levels


Using Aggregates

1. Import Tables

Import fact and dimension aggregates

dimension (so no join between fact and dimension at physical layer will be defined see next step)


Using Aggregates

2. Create Joins

Use the Physical Diagram to create joins between aggregate fact table and aggregate dimension tables

dimension - thus no join between fact and dimension at physical layer will be defined)


Using Aggregates

3. Create Fact Logical Table Source and Mappings

Create new aggregate LTS in the existing logical fact table and map the columns


Using Aggregates

4. Specify Fact Aggregation Content

Specify the aggregation content of the new fact LTS, so that BI Server knows what level of data is stored in the aggregate tables (to make optimal decision, which source to use for query)


Using Aggregates

5. Specify Content for the Fact Detail Source

Set the levels of the fact detail LTS to the lowest levels in hierarchies


Using Aggregates

6. Create Dimension Logical Table Source and Mappings

Create new aggregate LTS in the existing logical dimension tables and map the columns

dimension LTS

obiee10g metadata modeling basics

Documents

participants training

main obiee10g archi

repository day

repository business

repository presentation

trainer introduction

oracle peoplesoft

mapping layer