introduction to msbi by yasir

By - Shaik Yasir Ahmed

Raugh kimball – In simplest terms Data Warehouse can be defined as collection of Data marts. -Data marts : Subjective collection of Data.

Bill Inmon – A data warehouse is a “subject-oriented, integrated, timevariant,and nonvolatile” collection of data in support of management’s decision-making process.”ERP

will Run the Business- like how Tyres Run the Car

BI (Reports,Data mining,Dashboards,kpi’s)will help you to take business decisions based on your historical data. - like Steering, mirrors, breaks, dashboards will help, how smoothly you can run the Car or reach the Destination.

4

Which are our lowest/highest margin customers ?

Which are our lowest/highest margin customers ?

Who are my customers and what products are they buying?

Who are my customers and what products are they buying?

Which customers are most likely to go to the competition ?

Which customers are most likely to go to the competition ?

What impact will new products/services have on revenue and margins?

What impact will new products/services have on revenue and margins?

What product prom--otions have the biggest impact on revenue?

What product prom--otions have the biggest impact on revenue?

What is the most effective distribution channel?

What is the most effective distribution channel?

In What way a Data warehouse helps any BusinessLet’s say A producer wants to know….

5

Data, Data everywhere yet ...• I can’t find the data I need

– data is scattered over the network– many versions, subtle differences

• I can’t get the data I need• need an expert to get the data

• I can’t understand the data I found• available data poorly documented

• I can’t use the data I found• results are unexpected• data needs to be transformed from one

form to other

6

A single, complete and consistent store of data obtained from a variety of different sources made available to end users in a what they can understand and use in a business context.

[Barry Devlin]

7

What are the users saying...

•Data should be integrated across the enterprise•Summary data has a real value to the organization•Historical data holds the key to understanding data over time•What-if capabilities are required

8

A process of transforming data into information and making it available to users in a timely enough manner to make a difference

[Forrester Research, April 1996]

Data

Information

9

Data Warehousing -- It is a process• Technique for assembling and

managing data from various sources for the purpose of answering business questions. Thus making decisions that were not previous possible

• A decision support database maintained separately from the organization’s operational database

10

Data Mining works with Warehouse Data

Data Warehousing provides the Enterprise with a memory

Data Mining provides the Enterprise with intelligence

We want to know ...•Given a database of 100,000 names, which persons are the least likely to default on their credit cards?

•Which types of transactions are likely to be fraudulent given the demographics and transactional history of a particular customer?

•If I raise the price of my product by Rs. 2, what is the effect on my ROI?

•If I offer only 2,500 airline miles as an incentive to purchase rather than 5,000, how many lost responses will result?

•If I emphasize ease-of-use of the product as opposed to its technical capabilities, what will be the net effect on my revenues?

•Which of my customers are likely to be the most loyal?

Data Mining helps to extract such information

Base ProductBase Product$ 25K $ 40K $ 25K

Oracle 10g

IBM DB2

Base ProductBase Product

ManageabilityManageability

(included)(included)

$ 25K $ 40K $ 25K $ 56K $ 35K

Tuning $3K

Diagnostics $3K

Partitioning $10K

Performance

Expert$10K

Base ProductBase Product

ManageabilityManageability

(included)(included)

$ 25K $ 35K $ 154.5K $ 56K$ 116K

Business Business IntelligenceIntelligence

OLAP $20k

Mining$20k

BI Bundle$20k

DB2 OLAP $35KDB2

Warehouse $75K

Cube Views $9.5K

Base ProductBase Product

ManageabilityManageability

(included)(included)

$ 25K $ 154.5K $ 164.5K $ 232K$ 116K

Business Business IntelligenceIntelligence

High AvailabilityHigh Availability

Data Guard $116K Recovery

Expert$10k

Base ProductBase Product

ManageabilityManageability

(included)(included)

High AvailabilityHigh Availability

Business Business IntelligenceIntelligence

Multi-coreMulti-core

$348k - $464k

$ 232K$ 25K $ 164.5K $ 329K

$164.5K$116K - $232K

Additional BenefitAdditional BenefitNumber of UsersNumber of Users

What happened?

What happened?

Why did it happen?Why did

it happen?

What will happen?

What will happen?

What happened why and how?

What happened why and how?

OLTP – Online Transaction ProcessingOLAP – Online Analytical ProcessingMOLAP – Multidimensional OLAPROLAP – Relational OLAPHOLAP – Hybrid OALP Dimensions – De-normalized master tables Attributes – Columns of DimensionsHierarchies – sequential order of attributesFacts (Measure group) – Transactions tables in DWHFact (Measures)Cubes – Multidimensional storage of DataKPI’s – Key performance indicatorDashboards – combination of reports,kpis,chartsData Marts – Subjective Collection of DataSCD’s – Slowly changing DimensionsPerspectives – Child Cube

OperationalData Sources

Data-Migration Middleware (Populations-Tools)

DataStorage

RepositoryRepository

DataAnalysis

Reporting, OLAP,Data Mining

Stage DB Optional

ROLAP

OLTP

MOLAP

O L A P

SSIS

Integration Services Analysis Services

Reporting Services

SSAS

SSRS

SSISData Marts

CUBE

1. OLTP (on-line transaction processing)

2. Day-to-day operations: purchasing, inventory, banking, manufacturing, payroll, registration, accounting, etc.

1. OLAP (on-line analytical processing)

2. Data analysis and decision making

3. The tables are in the Normalized form. 3. The tables are in the De-Normalized form.

5. For Designing OLTP we used data modeling.

5. For Designing OLTP we used Dimension modeling.OLAP is classified into two i.e.,MOLAP & ROLAP

4. We Called the Storage objects as Tables. i.e., All the masters and the Transactions are stored in the tables.

4. We Called the Storage objects as Dimension and Facts. i.e., All the masters Are dimension and the Transactions are Facts.

Product

Prod_Id

Prod_Name

Base_Rate

Cat_IdCategory

Cat_Id

Cat_Name

Cat_Desc

Group_IdGroup

Group_Id

Group_Name

Group_Desc

Product_Dim

Prod_Id

Prod_Name

Base_Rate

Cat_Name

Cat_Desc

Group_Name

Group_Desc

Topics Later We will Cover

2. Slowly changing Dimensions1. Types of Dimensions

3. Hierarchies

Normalized Tables De-Normalized Tables

SalesOrder_Fact

Cust_Id

Prod_Id

Order_Date

Delivery_Date

Unit_Price

Qty

Total_Amount

Tax

SalesOrderDetails

Cust_Id

SalesPerson

Prod_Id

Order_Date

Booked_Date

Delivery_Date

Unit_Price

Qty

Tax

Created_By Qty*Unit_Price+Tax=Total AmountUsually calculate all the calculations before storing into OLAP

Reference keys of Dimensions

Numeric fields called as Fact or measure

Prod_Dim

Prod_Id

………

Cust_Dim

Cust_Id

………

Time_Dim

Date

Year

Month

………

Org_Dim

Org_Id

………SalesOrder_Fact

Cust_Id

Prod_Id

Order_Date

Delivery_Date

Org_Id

Unit_Price

Qty

Total_Amount

Tax

STAR Schema

Product_Dim

Prod_Id

Prod_Name

Base_Rate

Cat_Name

Cat_Desc

Group_Name

Group_Desc

SalesOrder_Fact

Cust_Id

Prod_Id

Order_Date

Delivery_Date

Unit_Price

Qty

Total_Amount

Tax

1. Dimensions will have only relation with the Fact. (Normalized model)

1. Dimension will have a relation other than Fact. (De-Normalized model)

2. One to many or One to One relation will Occur.

2. Used for many to many relation.

3. Performance is fast but required huge storage space.

3. Performance is Low but required Less storage space.