data warehousing & business intelligence introduction what do you think of when you hear the...
Post on 28-Dec-2015
216 Views
Preview:
TRANSCRIPT
Data Warehousing & Business Intelligence
Introduction
What do you think of when you hear the words Data Warehousing ?
Prithwis Mukerjee, Ph.D.
2©© Prithwis Mukerjee
Conceptual DW Definition
Data warehousing is a program dedicated to the delivery of information which advances decision making, improves business practices, and empowers workers.
3©© Prithwis Mukerjee
Data
Structu
re
Technology
Infrastr
ucture
Man
agem
ent
Business
Applicati
ons
Information
Information
Technology
Technology
Proce
ss
Proce
ss
Peopl
e
Peopl
e
The Knowledge Management Framework
4©© Prithwis Mukerjee
Database
How it all fits in ..
CRM : Customer Relationship Management
Transactional Systems ERP : Enterprise Resource Planning
SCM : Supply Chain Management
Data Warehouse
5©© Prithwis Mukerjee
Target Advertising campaigns
Strategic Initiatives
Business Processes
Functions
ProfitabilityAnalysis
Market Basket
Analysis
Product Pricing
Cross-selling and upgrade selling
Just-in-Time Inventory
Category Management
Human Resources Management
Determine Customer Lifetime Value
Predict Customer
Behavior
Management Reporting
Customer Acquisition and Retention
Typical Business Uses of the Data Warehouse
6©© Prithwis Mukerjee
Benefits of the Data Warehouse Program
Improves the way we do business and the bottom line
Revenue Stimulation & Revenue Protection
Cost Reduction and Cost Avoidance
Productivity Improvement
Profitability Enhancement
Performance Analysis
DecisionMaking
Market Response
Competitive advantage
7©© Prithwis Mukerjee
DSSs,Report writers, Excel, databases, etc.DSSs,Report writers, Excel, databases, etc.
Data FeedsData Feeds
BudgetingBudgeting
AnalysisAnalysis
Non-integrated Decision Support Architecture
Inventory System
Order System
Procurement System
Accounting System
Sales ForecastingSales Forecasting
8©© Prithwis Mukerjee
Enterprise DW/ODS
Subject oriented Data Warehouses or Data Marts
Subject oriented Data Warehouses or Data Marts
One Stop Data Shopping
One Stop Data Shopping
Basic Data Warehouse Architecture
Fewer Data FeedsFewer Data Feeds
Inventory System
Order System
Procurement System
Accounting System
9©© Prithwis Mukerjee
Performance Measures : Definition & Examples
Carefully selected set of measures derived from strategies, goals and objectives that represents a tool to communicating strategic direction to the organization for motivating change.These form the basis to plan, budget, structure the organization and to control results.
Innovation & Learning Measures
Innovation & Learning Measures
Customer MeasuresCustomer Measures
Financial MeasuresFinancial Measures
Internal Process
Measures
Internal Process
Measures
% Sales of New ProductsCustomers AcquiredCustomer Satisfaction
Market ShareROI and ROARevenue Growth
Product Time to MarketUnit Manufacturing CostDays Supply to inventory
New Product IntroductionManagement SkillsEmployee Turnover
10©© Prithwis Mukerjee
Differences between OLTP and DW
Data Access, Manipulation and UseData Organisation and IntegrationTime HandlingUsageData Structures and Schemas
Explanations ..Explanations ..
11©© Prithwis Mukerjee
Data access, manipulation and use
Data EntryTransaction OrientedConsistent use patternsData retrievals are lookups of single recordsUsers deal with one record at a timePerformance is criticalReporting is generally table lists
Data QueryBulk data orientedSpiked, uneven use patternsQueries are unpredictable, they change continuouslyData retrievals are summary and sorts of millions of recordsPerformance is relaxed (sec/min)Reporting is primary activity (on line, presented in small chunks)
OLTPOLTP DWDWDifferences between OLTP and DWDifferences between OLTP and DW
12©© Prithwis Mukerjee
Data Organisation And integration
Organized around applicationsUnintegrated dataDifferent key structuresDifferent naming conventionsDifferent file formats
Organized around subject areasIntegrated dataStandardized key structuresStandardized naming conventionsStandardized file formats
OLTPOLTP DWDWDifferences between OLTP and DWDifferences between OLTP and DW
13©© Prithwis Mukerjee
Time Handling
No time series analysisData relationships constantly changeChanges are instantaneousLimited history, 60-90 days
Twinkling Database ….
Time series analysisData is static over timeSeries of data snapshotsSnapshots create historical database, often greater than two yearsQuiet database
OLTPOLTP DWDWDifferences between OLTP and DWDifferences between OLTP and DW
14©© Prithwis Mukerjee
Usage
Place an order for a productLook up price for a productApply discountAssign shipperTrigger inventory pick-listVerify shipment of productCreate invoice for the productApply credit to sales representative
Essential to RUN the company
What type of customers are ordering this product?Who are my top 10% accounts? By name, by revenue, by profitability, by region?How are these different by customer segments? By sales rep? By store?Which shippers have the best on time delivery records ?How does this vary by shipment size? By season of year?
Essential to WATCH the company
OLTPOLTP DWDWDifferences between OLTP and DWDifferences between OLTP and DW
15©© Prithwis Mukerjee
Data Structures & Schemas
Drives out all data redundancy
Improves performance
Divides data into many discrete entitiesTables are symmetrical
Can’t tell most important, largest, which hold measures, which are static descriptors
Lots of connection paths between tables
prefers to use tables individually or in pairs
Too complex for users to understand
Data redundancy is encouraged
Improves table browsing
Subject area oriented. Groups data into categories of business measure and characteristicsTables are symmetrical
Large dominant tables
Clearly defined connection paths for table joinsSimple for users to understand and navigate
OLTPOLTP DWDWDifferences between OLTP and DWDifferences between OLTP and DW
16©© Prithwis Mukerjee
Basic Datawarehousing Topics
The Four Building BlocksDW DefinitionDW Usage and BenefitsDW Vs. the non-integrated DSS environmentPerformance Measures
Dimensional ModelingTechnical InfrastructureKnowledge Mgmt. ArchitectureIT and Business PerspectivesDW Methodology
17©© Prithwis Mukerjee
Dimensional Data Modeling
Dimensional Data Modeling techniques organize the content of the data warehouse. It structures the data according to the way users ask business questions.
18©© Prithwis Mukerjee
The Technical Infrastructure
A technical infrastructure provides the physical framework to support data acquisition, storage, access, and data management. It involves development and integration of hardware and software components.
19©© Prithwis Mukerjee
Metadata
Source Data
Purchasing Systems
General Ledger
Other Internal Systems
External Data
Sources
Data Resource Management And Quality Assurance.
Invoicing Systems Data
Extraction
Integration
and
Cleansing
Processes
Extract ODS
Purchasing
Marketing and Sales Corporate
information Product Line
Location
Translate
Attribute
Calculate
Derive
Summarize
Synchronize
Segmented
Data
Subsets
Summarized
Data
Data Warehouse Applications
Custom
Developed Applications
Data Mining
Statistical
Packages
Query Access Tools
Data Marts
Transform
Knowledge Management Architecture
20©© Prithwis Mukerjee
The Business and The IT Perspective
Business
What will it do?
What value will it bring?
How is it built?
How does it work?
Information Technology
Data Warehouse
21©© Prithwis Mukerjee
The Business Perspective of the Data Warehouse
It takes forever to get the information I need to do my jobWhen I do get it, it’s wrongWe have mountains of data, but I can’t figure out what’s importantIt takes so long to get the data that I don’t have any time left over to analyze itI want it to be easy. Just let me point and click my way to an answerI want to see my data in every possible combinationData is scattered everywhere across our organization. Where do I look ?I want a historical view of the businessI want to predict the future
Focuses on needs and usage
22©© Prithwis Mukerjee
The IT Perspective of the Data Warehouse
Organizes and stores data by subject area rather than applicationExtracts and integrates data from multiple source systems into a single databaseProvides data cleansing, summarization, and calculationUser does not create, update, or delete dataProvides snapshots of data over periods of timeSupports analytical processing, not transactional processingBuilds a technology infrastructure to support data acquisition, data storage, data access, and metadata captureFocuses on database, technology, organizational features
23©© Prithwis Mukerjee
DW Methodology
The methodology provides a detailed roadmap to organize and perform the tasks required in building the data warehouse
24©© Prithwis Mukerjee
Data Warehouse System Development Life Cycle
CONSTRUC-TION
CONSTRUC-TION
IMPLEMEN-TATION
IMPLEMEN-TATION
DESIGNDESIGNANALYSIS
ANALYSIS
PLANNING MANAGING
Business ArchitectureBusiness Architecture
Data ArchitectureData Architecture
Technology ArchitectureTechnology Architecture
Management InfrastructureManagement Infrastructure
top related