sequel 7 signs you need a data warehouse
TRANSCRIPT
7 Signs You Need a Data Warehouse
Mike StegemanSr. Data Access Consultant
Introductions
Heath KathSr. Data Access Consultant
Today’s Speakers:
Data Warehouse
•What is data warehousing (DW)
• Signs you need a data warehouse
• Common pitfalls
• SEQUEL Data Warehouse
Data Warehouse
What is a data warehouse (or business intelligence)?
Sales Data
Budgets
Customers, Inventory, Financials
Industry Data
Why do you need more than just a query tool?• You have multiple application databases• The database is complex and not well designed• Your data contains errors• Your reporting needs are complex • You have many query users
What’s real? Data is growing, data is becoming more complex, and users need and want more reliable information.
SEQUEL Data Warehouse
Multiple Application Databases
Sales (DB2 for i5/OS)
BI Reporting
Financials (DB2 for i5/OS)
POS System (SQL Server) Purchasing (Oracle)
You Need a Data Warehouse: Sign 1
You Need a Data Warehouse: Sign 2
CUSTNO CUSTNAME1001 John Smith1002 Mary Jones1003 Chris Anderson1004 David Perry
Customer File - USCUSTNO CUSTNAME
1001 Harry Potter1002 Jeremy Carr1003 Penny Hayes1004 Debbie Thornton
Customer File - Canada
CUSTID CUSTNAMAA234 Julie JohnsonAA235 Fred HunterAB670 John SmithBD309 Alan Jordan
Customer File - CanadaCUSTNO CUSTNAME
1001 John Smith1002 Mary Jones1003 Chris Anderson1004 David Perry
Customer File - US
Files (tables) are the same, but different… Multiple instances of the same table, with duplicate key values
Or different versions of the same entity with incompatible data types
Poor performance
You Need a Data Warehouse: Sign 3
100M rows
Changing dimensions
You Need a Data Warehouse: Sign 4
100 Smith & Jones ElectricalSmall Retailer Jenny Brown
100 Smith & Jones ElectricalMajor Retailer Rob McAdam
100 Smith & Jones ElectricalMajor Retailer Jenny Brown
2012
2013
2014
You Need a Data Warehouse: Sign 5
Data errors• Failed joins• Invalid dates• Missing values
Difficult dates• Dates are in MDY format,
but you need to sort by date• Separate Year, Month, Day
columns
Hidden meanings and conditional rules• Second character of column X means…• If column Y = ‘C’, value Z must be multiplied by -1• If record type = 1, there must be a matching record in table B
• If type = 2, there may be a record• If type = 3, there must not be a record
• For data older than 7/1/2005 column X will be zero, but it must be a value in the range of 1-5 from that date onwards
You Need a Data Warehouse: Sign 6
A chaotic reporting environment!
SalesSales FinancialsFinancialsPurchasingPurchasing
GL Summary(Excel)Summary Sales by
Customer/Brand
Profitability Extract
Summary Sales by Region
Purchasing extract
(MS Access)
~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~
Joe downloads this manually via Client
Access every Monday.. except when he’s on
vacation or out with the flu!
Mary wrote this extract. She left last year and no-one knows how it
works.
The Net Sales calculation in this
extract is different to
Mary’s
No one has yet realized that this is loaded
incorrectly. The auditors will be the first to discover the problem
These reports don’t
balance with each
other.
These reports don’t
balance with each
other. No one trusts
this reportJohn spends 5 days
every month generating this and
massaging the numbers until he
thinks it is correct
You Need a Data Warehouse: Sign 7
Example of poor data quality:•Property assessment incorrectly changed to $400M •Property tax revenue of $8M was included in the county budget •County had a huge revenue shortfall, resulting in lots of cuts•The school district forced to return $2.7M
Just because of ONE
bad data
value!
Common Results
We found:• 96% of data marts require change in the first year (usually
requiring them to be totally rebuilt)
• 75% of independent data marts do not survive past two years
• 60% of companies without a data warehouse architecture abandon their BI investment within 5 years, citing maintenance complexity and cost as the prime reasons
SOLUTION: SEQUEL Data Warehouse
SEQUEL Data Warehouse
ETL Tool
DataManagement
ManageDevelopment
IBM iEnvironment
MetadataRepository
DataWarehouse
SEQUEL Data Warehouse
Development Client
SEQUEL Data Warehouse
SALES PURCHASING FINANCIALS
O P E R A T I O N A L S Y S T E M S
SEQUEL Implemented Against Operational Data
Simple Implementation
Front End Tools Implemented Against DW/DM Tables
Data Warehouse/Data Marts
O P E R A T I O N A L S Y S T E M S
SALES PURCHASING FINANCIALS
Adding SEQUEL Data Warehouse
SEQUEL Data Warehouse – Data Access
+ Non DB2 Data Sources
SEQUEL Data Warehouse on IBM i
Oracle, MS SQL, MySQL, Sybase
XML FilesText Files (fixed length or delimited)
MS Excel Salesforce Apache Hive, Impala
Share Information with SEQUEL
Key to formatting and distribution of data
Share information with SEQUEL Web Interface
Solves Business Problems•Reduces the time required to deploy•Requires no software to install for end users•Builds web pages without needing another tool•Provides a secure way to view your data
Data Warehouse
A solution to your business needs
Why use a Data Warehouse
We are ready for your questions!
Mike StegemanSr. Data Access Consultant
Heath KathSr. Data Access Consultant
Today’s Speakers:
Thank You for Joining Us Today!
Website: www.helpsystems.com/sequel
Phone: 800-328-1000 or+1 952-933-0609
Email: [email protected]@helpsystems.com